101 to 117 of 117 Apache Spark Jobs in the UK

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

production-ready PySpark code. Strong understanding of data and data warehousing concepts, including: ETL/ELT, Data models, Dimensions and facts, Data marts, SCDs Spark Performance & Optimisation Strong knowledge of Spark execution concepts, including partitioning, optimisation, and performance tuning. Experience troubleshooting and improving distributed data processing pipelines. Python ...

data transformation processes. Cloud & Data Engineering (AWS-Focused) Build and deploy data pipelines using AWS services such as EMR, Glue, S3, Athena . Optimise Spark workloads for performance, partitioning, cost efficiency, and scalability. Use CI/CD pipelines and Git-based version control for deployment and automation. Collaborate with … Ability to write production-grade data engineering code. Strong understanding of: ETL/ELT Data modelling Facts & dimensions Data marts Slowly Changing Dimensions (SCDs) Spark Performance & Optimisation Expertise in Spark execution, partitioning, performance tuning, and optimisation. Troubleshooting distributed data pipelines at scale. Python & Engineering Quality Strong Python coding ...

data engineering projects. Responsibilities Design, build and deliver data solutions using Databricks Develop and optimise ETL/ELT data pipelines using Spark, Python and SQL Build and maintain data lakes and data warehousing solutions Collaborate with stakeholders to deliver scalable and insight-driven data platforms Contribute to projects involving … Snowflake, Azure Data Factory, Synapse or Microsoft Fabric Skills and Experience Strong experience with Databricks with either Azure or AWS Strong SQL, Python and Spark expertise Proven track record delivering data engineering solutions across full project lifecycles Strong understanding of data modelling and modern data architectures Ability to engage ...

ensure data quality as the platform scales. The environment is modern and cloud-based, using tools such as AWS, GCP or Azure, alongside Spark, Snowflake or BigQuery. You'll also play a key role in supporting and mentoring other engineers through code reviews and collaboration. You'll be joining … gives engineers the space to contribute in their own way. What we're looking for: Strong Python and SQL experience in production Experience with Spark, Snowflake, BigQuery or similar Cloud experience (AWS, GCP or Azure) Comfortable working with complex, messy data Experience owning pipelines end-to-end Collaborative mindset ...

Senior Data Engineer- Spark & Databricks Optimisation- Up to £55K North Lincolnshire/Hybrid (2 days onsite) Major performance, optimisation & AI-driven projects Are you a Data Engineer who loves solving performance challenges, optimising compute, and getting stuck into large-scale data environments? This one is very much … understands what "good" looks like across modern data platforms. Someone who knows Python and SQL inside out, and who can improve pipelines, optimise Spark workloads, and make smart architectural decisions that genuinely impact runtime and cost. Tech-wise, you'll have hands-on experience with a modern cloud ecosystem. ...

with Azure Cosmos DB including query optimisation and throughput management Thorough understanding of Cosmos DB Change Data Feed (CDF) and integration and debugging of Spark with Cosmos CDF. High technical proficiency with Cosmos DB SDKs (e.g., .NET, Java, Python, Node.js) Thorough understanding of Spark distributed computing concepts. High … technical proficiency in PySpark and Spark Concepts. Experience with concurrency patterns, CLR, and scalable application design. Deep understanding of Azure services including Azure Functions, App Services, AKS, and Logic Apps. Experience with SQL API and familiarity with other APIs. Strong understanding of partitioning, indexing, and consistency levels. Experience with ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they re looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What s on offer: - The chance to join a business at an exciting stage of growth ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they're looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What's on offer: - The chance to join a business at an exciting stage of growth ...

experience with Cloudera Data Platform (CDP) on AWS or recent Hadoop administration on similar platforms. AWS administration and platform management experience. Strong experience in Spark, Airflow, Apache Ranger, Impala (AWS CDK is a plus). Agile experience with Scrum, and tools including Jira, Slack. Familiarity with PowerBI ...

Lead Data Engineer Tech Stack: Python | Spark | Kafka | Airflow | Vector DBs Location: London - office based Salary: Up to £140k + equity We’re partnering with a well-funded AI startup building a category-defining platform within a major UK industry undergoing rapid transformation. This is a first data hire … What we’re looking for: • 7+ years in Data/Backend Engineering • Strong Python and distributed systems experience • Hands-on with modern data tooling (Spark, Airflow, Kafka, etc.) • Experience with relational + NoSQL databases (vector DBs a big plus) • Comfortable in a fast-paced, early-stage environment 💡 High ownership ...

everything listed below, but experience with a combination of these would be valuable: Power BI, Tableau or other BI/visualisation tools Databricks or Spark (certifications are a plus) AWS (Solutions Architect knowledge desirable) SQL/Spark SQL Snowflake Cloud-based or enterprise-scale data platforms (e.g., Cloudera ...

Head of Engineering - Remote - UK - £90,000-£130,000 GenAI, ML, Spark Why Join This Company? This is a leadership role for someone who wants to work in a truly AI-first engineering environment, not a business experimenting at the edges. The company is already well ahead … solving rather than legacy ways of working. You'll Work With Cross-functional teams spanning: Web Engineering (React/TypeScript) Data Engineering (distributed systems, Spark-based platforms) Data Science & Machine Learning A highly technical CTO who is deeply invested in AI adoption and modern engineering Product leadership following significant ...

DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience Hands ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...