51 to 61 of 61 Apache Spark Jobs in the UK excluding London

/ELT pipelines integrating large, complex datasets Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies Develop solutions using Hadoop, Spark, and Splunk in large-scale environments Write maintainable Python code, applying object-oriented and functional programming principles Implement and maintain CI/CD pipelines … sustainability, and recoverability Essential Skills & Experience Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments Hands-on expertise with Hadoop, Spark, and Splunk Advanced Python development skills Experience designing and optimising high-performance data pipelines Strong understanding of CI/CD, source control, and automated ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

Data Engineer Role: Data EngineerLocation: Newcastle Upon TyneSalary: TBC – Depending on experienceLevels: Senior Analyst, Specialist Hybrid Working: 3 days per week in our Newcastle, Cobalt business park office Please Note: Any offer of employment is ...

data engineering projects. Responsibilities Design, build and deliver data solutions using Databricks Develop and optimise ETL/ELT data pipelines using Spark, Python and SQL Build and maintain data lakes and data warehousing solutions Collaborate with stakeholders to deliver scalable and insight-driven data platforms Contribute to projects involving … Snowflake, Azure Data Factory, Synapse or Microsoft Fabric Skills and Experience Strong experience with Databricks with either Azure or AWS Strong SQL, Python and Spark expertise Proven track record delivering data engineering solutions across full project lifecycles Strong understanding of data modelling and modern data architectures Ability to engage ...

Senior Data Engineer- Spark & Databricks Optimisation- Up to £55K North Lincolnshire/Hybrid (2 days onsite) Major performance, optimisation & AI-driven projects Are you a Data Engineer who loves solving performance challenges, optimising compute, and getting stuck into large-scale data environments? This one is very much … understands what "good" looks like across modern data platforms. Someone who knows Python and SQL inside out, and who can improve pipelines, optimise Spark workloads, and make smart architectural decisions that genuinely impact runtime and cost. Tech-wise, you'll have hands-on experience with a modern cloud ecosystem. ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they're looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What's on offer: - The chance to join a business at an exciting stage of growth ...

experience with Cloudera Data Platform (CDP) on AWS or recent Hadoop administration on similar platforms. AWS administration and platform management experience. Strong experience in Spark, Airflow, Apache Ranger, Impala (AWS CDK is a plus). Agile experience with Scrum, and tools including Jira, Slack. Familiarity with PowerBI ...

everything listed below, but experience with a combination of these would be valuable: Power BI, Tableau or other BI/visualisation tools Databricks or Spark (certifications are a plus) AWS (Solutions Architect knowledge desirable) SQL/Spark SQL Snowflake Cloud-based or enterprise-scale data platforms (e.g., Cloudera ...

DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience Hands ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...