Senior Data Engineer

Role Senior Data Engineer

Sheffield, UK

6 Months contract

Rate: 65 GBP per hour inside IR35

Experience 7 to 11

Responsibilities

• Design build and maintain largescale data pipelines and ETL workflows across enterprise datasets
• Ingest provision and manage raw enriched and curated data assets for analytics platforms
• Improve reliability scalability and realtime data ingestion capabilities
• Support and enhance cloudbased data ingestion infrastructure and platform services
• Acquire transport clean transform and integrate data from internal and external sources
• Perform exploratory data analysis and onboard new data sources
• Develop automated data engineering workflows and optimize data processing pipelines
• Manage data modelling data warehousing data integration and data cataloguing activities
• Support realtime analytics and streaming data platforms
• Contribute to cloud infrastructure engineering platform operations and production support
• Implement cloud cost optimization and platform efficiency improvements
• Collaborate with engineering analytics infrastructure and cybersecurity teams
• Support DevOps automation monitoring and operational excellence initiatives
• Maintain documentation governance standards and operational procedures

To be successful in this role you should have

• Experience with SRE principles and Azure DevOps
• Strong scripting skills using Bash PowerShell and Azure CLI
• Programming experience with Python C Java and objectoriented development
• Strong SQL and Kusto Query Language KQL experience
• Experience with GitHub source control systems and CICD pipelines
• Strong Linux administration and scripting experience
• Knowledge of networking operating systems storage technologies and infrastructure fundamentals
• Experience in Data Acquisition and CloudBased Data Pipelines
• Experience with Data Transport Data Cleaning and Data Integration
• Strong Data Engineering pipeline automation productionization and optimization skills
• Experience designing and maintaining ETL workflows and largescale data pipelines
• Knowledge of Data Warehousing Data Modeling and Data Cataloguing
• Experience with RealTime Analytics and Streaming Data Platforms
• Cloud Cost Optimization experience
• Strong Azure platform knowledge including Azure AD Azure IAM Networking Compute Storage Databases and Containers
• Experience with Azure Data Factory Azure Databricks Unity Catalog Azure Functions Azure Logic Apps Azure Monitor Azure Log Analytics Azure Data Lake Synapse Analytics and Power BI
• Experience with Kubernetes and CloudNative platforms
• Knowledge of Nginx Apache CosmosDB Linux Prometheus Grafana and Elasticsearch
• Experience with Infrastructure as Code tools such as Terraform ARM Chef and Ansible
• Experience with Azure Event Hubs Kafka and Spark Streaming
• Exposure to SIEM and SOAR platforms is advantageous
• Experience working in highly regulated enterprise environments is preferred