26 to 50 of 701 Apache Spark Jobs in the UK

related disciplines within enterprise environments 4+ years of hands‐on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines … solutions on the Databricks Lakehouse Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business ...

related disciplines within enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines … solutions on the Databricks Lakehouse Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business ...

related disciplines within enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines … solutions on the Databricks Lakehouse Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business ...

related disciplines within enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines … solutions on the Databricks Lakehouse Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business ...

Generative AI initiatives by developing scalable, secure and reliable data platforms. Key Responsibilities Design, build and maintain high-throughput data pipelines using Apache Spark and cloud-native technologies. Develop scalable ETL/ELT workflows for structured and unstructured datasets. Build and optimise data-processing solutions within AWS environments. … standards. Troubleshoot complex data-processing and distributed-system issues within production environments. Essential Experience 5+ years of commercial Data Engineering experience. Strong hands-on Apache Spark development experience. Strong programming experience with Java and/or Scala. Strong AWS experience, ideally including technologies such as S3, EMR, Glue ...

data strategy Helping build and mentor the future data engineering team as the company scales Python, PostgreSQL, MongoDB, vector databases (Qdrant, Milvus or pgvector), Apache Spark, Apache Airflow, Kafka, Elasticsearch/OpenSearch, Pandas, Polars. What they're looking for: 7+ years of experience in data engineering ...

frameworks, tools and applications that make that data available to other teams and systems. Your primary toolset in your work will be Java microservices, Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling.**What You'll Do:*** Write highly fault-tolerant Java code within Apache Spark to produce platform products used by our customers to query our event pipelines/ingestion for insight into active threat trends and related analytics* Design, develop, and maintain ultra-high-scale data platforms that process petabytes of data* Participate in technical reviews of our products and help ...

frameworks, tools and applications that make that data available to other teams and systems.Your primary toolset in your work will be Java microservices, Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling.What You'll Do: Write highly fault-tolerant Java code within Apache Spark … combined between backend/cloud development and data platform engineering roles5+ years of experience building data platform product(s) or features with (one of) Apache Spark, Flink or Iceberg, or with comparable tools in GCP5+ years of experience programming with Java, Scala or Kotlin.Proven experience utilizing AI technologies ...

control (Git) and related software lifecycle tooling.Experience with tooling for monitoring, logging and alerting e.g. Splunk, New Relic or Grafana.Experience of technologies such as Apache Spark, Databricks, Snowflake, or BigQuery to transform big-data optimised for Tesco scaleExperience building solutions that run in the cloud, ideally Azure.Experience with … control (Git) and related software lifecycle tooling.Experience with tooling for monitoring, logging and alerting e.g. Splunk, New Relic or Grafana.Experience of technologies such as Apache Spark, Databricks, Snowflake, or BigQuery to transform big-data optimised for Tesco scaleExperience building solutions that run in the cloud, ideally Azure.Experience with ...

knowledge of optimisation techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

platforms.Solid Cloud and DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible.Experience building and supporting reactive, event-sourced systems.Hands-on experience with Apache Spark and Apache Flink.Strong understanding of Apache HTTP, Envoy, and REST API design.Experience working with Protobuf, Avro, and Parquet data formats.Knowledge ...

DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible. Experience building and supporting reactive, event-sourced systems. Hands-on experience with Apache Spark and Apache Flink. Strong understanding of Apache HTTP, Envoy, and REST API design. Experience working with Protobuf, Avro, and Parquet data ...

Modern data platforms: Databricks, Snowflake and cloud‐native data services Data architecture: Lakehouse, medallion, data mesh and domain‐oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise‐scale ETL/ELT, batch … streaming patterns Transformation and quality: dbt, Great Expectations and equivalent tooling Orchestration: Apache Airflow and cloud‐native orchestration services Event streaming: Kafka and cloud‐native event‐streaming technologies Data modelling: Dimensional, analytical and domain‐oriented modelling Data governance: Catalogue, metadata, lineage, quality, security, privacy and access controls Data products ...

data processing solutions using BigQuery, Pub/Sub, Dataflow and Dataproc Experience developing end‐to‐end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam. Expert understanding of when to use a range of data storage technologies including relational/non‐relational, document ...

Modern data platforms: Databricks, Snowflake and cloud-native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch … streaming patterns Transformation and quality: dbt, Great Expectations and equivalent tooling Orchestration: Apache Airflow and cloud-native orchestration services Event streaming: Kafka and cloud-native event-streaming technologies Data modelling: Dimensional, analytical and domain-oriented modelling Data governance: Catalogue, metadata, lineage, quality, security, privacy and access controls Data products ...

Modern data platforms: Databricks, Snowflake and cloud-native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch … streaming patterns Transformation and quality: dbt, Great Expectations and equivalent tooling Orchestration: Apache Airflow and cloud-native orchestration services Event streaming: Kafka and cloud-native event-streaming technologies Data modelling: Dimensional, analytical and domain-oriented modelling Data governance: Catalogue, metadata, lineage, quality, security, privacy and access controls Data products ...

application containerization.Experience building applications on public cloud platforms like AWS, GCP, and Azure, with a preference for AWS.Proficiency in real-time streaming applications using Apache Spark Streaming, Apache Kafka, Amazon Kinesis, and emerging cloud serverless managed services.J.P. Morgan is a global leader in financial services, providing strategic ...

containerization. Experience building applications on public cloud platforms like AWS, GCP, and Azure, with a preference for AWS.Proficiency in real-time streaming applications using Apache Spark Streaming, Apache Kafka, Amazon Kinesis, and emerging cloud serverless managed services.J.P. Morgan is a global leader in financial services, providing strategic ...

ready data processing solutions using BigQuery, Pub/Sub, Dataflow and DataprocExperience developing end-to-end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam.Expert understanding of when to use a range of data storage technologies including relational/non-relational, document ...

Document stores).Hands-on experience building, scaling, and shipping large-scale data platforms in production.Deep practical experience with distributed data processing frameworks (e.g., Apache Spark, Flink, Hadoop).Strong expertise in message brokers and event streaming platforms (e.g., Apache Kafka, Kinesis).End-to-end exposure to data pipeline … lifecycle development, including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster).Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg).Advanced SQL skills and proficiency in Python. Strong background in modern software development practices ...

Solid knowledge of Linux/Unix, OpenShift/Kubernetes, and Ansible. Distributed Systems: Experience building and supporting reactive, event-sourced systems, with hands-on Apache Spark and Apache Flink. APIs & Data Formats: Strong understanding of Apache HTTP, Envoy, and REST API design, plus experience with Protobuf ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

ingestion, incremental loading, and change data capture. Design and implement layered data models and transformation logic (raw, conformed, aggregated) using tools such as dbt, Spark, and cloud‐native processing services, ensuring data is clean, consistent, and ready for downstream consumption. Develop modular, reusable data transformation components and functions that … tooling, or cloud cost management across at least one major cloud provider (Azure, AWS, or GCP). Technical Skills Data engineering: strong proficiency with Apache Spark, dbt, Airflow/Azure Data Factory or equivalent orchestration, Data Lake/Apache Iceberg, and advanced SQL for large‐scale transformations. ...

wider stakeholders. Technical Environment Experience with some of the following technologies would be beneficial: Python SQL Cloud platforms – AWS, Azure or GCP Databricks Apache Spark/PySpark Kafka/event-driven architectures Airflow dbt Git/GitHub CI/CD Infrastructure as Code Modern data platforms and distributed ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...