76 to 96 of 96 Apache Spark Jobs in England

candidates with experience of one or more of the major cloud platforms. AWS, Azure and GCP depending on the project Databricks, Snowflake and Synapse Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kafka or Kinesis You're expected to pick the right tools ...

ensure A-Level grades are included on your CV) Basic scripting knowledge in Python or Bash Excellent customer-facing skills You have a sales spark - while this role isn't a focused sales role, this is required due to the nature of the role A motivated self-starter with ...

technical delivery of data platforms and pipelines built in AWS-with a focus on AWS Glue, Managed Workflows for Apache Airflow (MWAA), and Python-and collaborate closely with Directors, Senior Architects, and Program Leadership to deliver business outcomes at scale. This is a player-coach role: you will design … environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs, task isolation, best practices) Strong across broader AWS services: S3, Lambda, Step Functions, IAM, CloudWatch, KMS, Secrets ...

technical delivery of data platforms and pipelines built in AWS-with a focus on AWS Glue, Managed Workflows for Apache Airflow (MWAA), and Python-and collaborate closely with Directors, Senior Architects, and Program Leadership to deliver business outcomes at scale. This is a player-coach role: you will design … environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs, task isolation, best practices) Strong across broader AWS services: S3, Lambda, Step Functions, IAM, CloudWatch, KMS, Secrets ...

engineering with stakeholder engagement and solution design. Key Responsibilities Build and optimise data pipelines, ETL processes, and data platforms Develop solutions using Python, Spark, Kafka, Hadoop or similar Work in Agile teams to deliver production-ready systems Translate business requirements into technical data solutions Engage with stakeholders and communicate …/CD Skills & Experience Experience as a Data Engineer/Big Data Engineer Strong coding in Python, Scala or Java Hands-on with Spark, Kafka, ETL/data pipelines Knowledge of cloud platforms (AWS, Azure or GCP) Familiar with Agile and software engineering best practices Strong communication/stakeholder ...

with EKS) Proficiency in Git-based development workflows Experience working independently and communicating effectively in distributed teams Desirable : Familiarity with (or happy to learn): Apache Spark (EMR) and Delta Lake PostgreSQL (Aurora) CI/CD (Github Actions) and IaC (Helm, CloudFormation) Monitoring (Sentry) Knowledge/interest in research … improve the TypeScript/React-based frontend application. Build and manage containerised services deployed on AWS, utilising Kubernetes orchestration. Support data and transformation workflows (Spark) Diagnose, troubleshoot, and improve overall application performance. Handle and resolve live production issues efficiently and under time pressure. Collaborate closely with product and data ...

experience, within tools such as Terraform, ArgoCD, Helm, or other IaC Degree within Computer Science, ideally at Masters or PhD level Experience with Kafka, Spark, Trino would be beneficial If interested, please apply below with your CV. ...

scalable data pipelines and solutions. From a technical standpoint you will ideally possess: Proven experience with databricks Proficiency in programming languages such as Python, Spark, SQL. Strong experience with SQL databases. Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF). Experience with cloud platforms … Azure preferred) and related data services. Knowledge of big data technologies (e.g., Hadoop, Spark, Kafka). Experience of Waterfall and Agile delivery methodologies Contact: Please apply via the link or contact (url removed) for more information. Modis International Ltd acts as an employment agency for permanent recruitment ...

engineering with stakeholder engagement and solution design. Key Responsibilities Build and optimise data pipelines, ETL processes, and data platforms Develop solutions using Python, Spark, Kafka, Hadoop or similar Work in Agile teams to deliver production-ready systems Translate business requirements into technical data solutions Engage with stakeholders and communicate …/CD Skills & Experience Experience as a Data Engineer/Big Data Engineer Strong coding in Python, Scala or Java Hands-on with Spark, Kafka, ETL/data pipelines Knowledge of cloud platforms (AWS, Azure or GCP) Familiar with Agile and software engineering best practices Strong communication/stakeholder ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

Data Engineer Role: Data EngineerLocation: Newcastle Upon TyneSalary: TBC – Depending on experienceLevels: Senior Analyst, Specialist Hybrid Working: 3 days per week in our Newcastle, Cobalt business park office Please Note: Any offer of employment is ...

data engineering projects. Responsibilities Design, build and deliver data solutions using Databricks Develop and optimise ETL/ELT data pipelines using Spark, Python and SQL Build and maintain data lakes and data warehousing solutions Collaborate with stakeholders to deliver scalable and insight-driven data platforms Contribute to projects involving … Snowflake, Azure Data Factory, Synapse or Microsoft Fabric Skills and Experience Strong experience with Databricks with either Azure or AWS Strong SQL, Python and Spark expertise Proven track record delivering data engineering solutions across full project lifecycles Strong understanding of data modelling and modern data architectures Ability to engage ...

ensure data quality as the platform scales. The environment is modern and cloud-based, using tools such as AWS, GCP or Azure, alongside Spark, Snowflake or BigQuery. You'll also play a key role in supporting and mentoring other engineers through code reviews and collaboration. You'll be joining … gives engineers the space to contribute in their own way. What we're looking for: Strong Python and SQL experience in production Experience with Spark, Snowflake, BigQuery or similar Cloud experience (AWS, GCP or Azure) Comfortable working with complex, messy data Experience owning pipelines end-to-end Collaborative mindset ...

Senior Data Engineer- Spark & Databricks Optimisation- Up to £55K North Lincolnshire/Hybrid (2 days onsite) Major performance, optimisation & AI-driven projects Are you a Data Engineer who loves solving performance challenges, optimising compute, and getting stuck into large-scale data environments? This one is very much … understands what "good" looks like across modern data platforms. Someone who knows Python and SQL inside out, and who can improve pipelines, optimise Spark workloads, and make smart architectural decisions that genuinely impact runtime and cost. Tech-wise, you'll have hands-on experience with a modern cloud ecosystem. ...

Developing and maintaining scalable data pipelines using Python - Automating manual Excel/VBA workflows into reliable, production-grade systems - Working with large datasets using Spark and Azure (Databricks/Synapse) - Supporting data modelling and analytics use cases (Crossing over to more Data Science driven work) - Driving best practices including … code reviews - Collaborating with stakeholders to deliver actionable data solutions What they're looking for: - Strong Python and SQL skills - Experience or exposure to Spark, big data tools, or cloud platforms (ideally Azure) What's on offer: - The chance to join a business at an exciting stage of growth ...

experience with Cloudera Data Platform (CDP) on AWS or recent Hadoop administration on similar platforms. AWS administration and platform management experience. Strong experience in Spark, Airflow, Apache Ranger, Impala (AWS CDK is a plus). Agile experience with Scrum, and tools including Jira, Slack. Familiarity with PowerBI ...

Lead Data Engineer Tech Stack: Python | Spark | Kafka | Airflow | Vector DBs Location: London - office based Salary: Up to £140k + equity We’re partnering with a well-funded AI startup building a category-defining platform within a major UK industry undergoing rapid transformation. This is a first data hire … What we’re looking for: • 7+ years in Data/Backend Engineering • Strong Python and distributed systems experience • Hands-on with modern data tooling (Spark, Airflow, Kafka, etc.) • Experience with relational + NoSQL databases (vector DBs a big plus) • Comfortable in a fast-paced, early-stage environment 💡 High ownership ...

everything listed below, but experience with a combination of these would be valuable: Power BI, Tableau or other BI/visualisation tools Databricks or Spark (certifications are a plus) AWS (Solutions Architect knowledge desirable) SQL/Spark SQL Snowflake Cloud-based or enterprise-scale data platforms (e.g., Cloudera ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...