1 to 25 of 30 Remote/Hybrid Apache Spark Jobs in the North of England

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

engineers where needed. Core Data Engineering Strong programming proficiency in Java (preferred) or Python. Hands-on experience with at least one of: Kafka, Flink, Spark (Flink/Kafka preferred for streaming). Solid understanding of stream processing concepts (e.g., event time, state, backpressure). Understanding of software engineering best ...

will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We … version control.* **Data Engineering Stack** Experience building pipelines using streaming frameworks, distributed processing engines and relational or analytical storage technologies. Examples include Kafka, Spark, PostgreSQL.* **Orchestration** Experience defining and running data workflows using modern orchestration frameworks. Examples include Airflow, Dagster or Prefect.* **Data Quality and Lineage** Familiarity with tools ...

with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema design, metadata management, and governance. Familiarity with ...

stakeholder/data consumer management. Strong programming skills in languages such as Python, familiarity with large scale, distributed data processing engines such as Apache Spark, with a solid understanding of data structures and algorithms. Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL) or NoSQL databases ...

Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse architectures and dimensional modelling. Experience delivering scalable, high-quality data solutions to support analytics ...

modern cloud engineering practices. Where you land, you’ll be working with some of the biggest datasets in the UK – using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. Cloud-native AI engineering and large-scale data platforms, leveraging Google Cloud technologies and modern ...

modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK — using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. Cloud-native AI engineering and large-scale data platforms, leveraging Google Cloud technologies and modern ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

into business processes and decision-making. Technical Skills: Hands-on experience in one or more of the following areas: Programming & Analytics – Python, SQL, R, Spark, machine learning and advanced analytics Cloud & Data Platforms – Microsoft Fabric, Databricks, Snowflake, Azure, AWS and GCP AI Technologies – GenAI frameworks, Agentic AI, vector databases ...

environment Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Experience performing tasks such as writing scripts, extracting data using APIs, writing SQL queries etc. Experience in processing large amounts of structured ...

Building and maintaining data pipelines. Data modelling and warehousing fundamentals. dbt or similar modern transformation frameworks Experience with AWS data services. Exposure to Kafka, Spark, streaming data, or modern BI tools. Interest in AI, analytics engineering, and data products THE BENEFITS You will receive a salary ...

Building and maintaining data pipelines. Data modelling and warehousing fundamentals. dbt or similar modern transformation frameworks Experience with AWS data services. Exposure to Kafka, Spark, streaming data, or modern BI tools. Interest in AI, analytics engineering, and data products THE BENEFITS You will receive a salary ...

Building and maintaining data pipelines. Data modelling and warehousing fundamentals. dbt or similar modern transformation frameworks Experience with AWS data services. Exposure to Kafka, Spark, streaming data, or modern BI tools. Interest in AI, analytics engineering, and data products THE BENEFITS You will receive a salary ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

Scala. Strong knowledge of SQL and experience with data warehousing solutions. Familiarity with cloud platforms (AWS, Azure, Google Cloud) and big data technologies (Hadoop, Spark). Excellent problem-solving skills and a keen eye for detail. A collaborative spirit and a genuine passion for data! Our Client's Benefits ...

Roku in Manchester is looking for a Senior Data Engineer to join the Viewer Product Device & Themed Experiences team. You will design data models and build scalable pipelines across Roku devices, apps, and web/ ...

stakeholders. Architecture design informed through the use of principles, patterns, technical radars, practices and standards. AWS Solution expertise and relevant certification - EMR/Spark (Hadoop)/AWS/Data Modelling-Mapping/GitHub/GitLab Technologies Azure AWS Power BI #J-18808-Ljbffr ...

depth actually gets used. AWS across the board, including S3, Glue, Redshift, Lake Formation and Kinesis Databricks and Snowflake running on top of AWS Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kinesis or Kafka You're expected to pick the right tools ...

role you should have the following skills: Experience implementingdata processes for Data Warehouse and production systems. Strong experience withBig Data technologies(e.g.,Hadoop/Spark). Programming in modern languages (e.g.,ScalaandPython). Strong delivery engineering tooling:Git/GitHub,Jenkins,Ansible, andJIRAfor requirement/backlog management. Strong understanding ...

maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions Building robust data … would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group locations will ...

maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions Building robust data … would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group locations will ...

Python is required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation ...