51 to 75 of 96 Apache Spark Jobs in England

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...

Lakehouse platform in Azure. Owning Databricks engineering, administration and platform optimisation. Designing end-to-end data platform solutions Leading implementation using Databricks, PySpark, Spark SQL and Azure Data Factory. Creating and maintaining CI/CD pipelines and automated testing frameworks. Developing APIs to support integration Implementing monitoring, logging ...

Experience with cloud platforms (AWS) and containerisation (such as Docker, Podman, Kubernetes) Experience working in secure or regulated environments Exposure to big data technologies (Spark, Hadoop) and Apache tools Familiarity with Agile and DevOps practices Industry certifications (e.g., TensorFlow Developer, AWS Machine Learning Specialty) Knowledge of NLP, computer ...

Gen2). Optimize data storage formats (Delta, Parquet, Avro, ORC) for performance and cost. Maintain and optimize Synapse SQL Pools, Dedicated SQL Pools, and Spark Pools. Implement and monitor data quality rules, validation frameworks, and error handling. Ensure compliance with data governance, security, RBAC, and privacy standards. Work with … years of professional experience in data Engineer or a related field, with a proven track record of delivering impactful solutions. SQL Pools, serverless SQL, Spark Pools Strong SQL, performance tuning, query optimization Data modeling & warehouse concepts (Kimball/Inmon) Pipelines, triggers, linked services, integration runtime Data flows & orchestration ...

influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing … Engineer or Solutions Architect certification (desirable). Databricks Certified Data Engineer or Machine Learning Associate (desirable). Experience with streaming solutions (Kafka, Event Hubs, Spark Streaming) (desirable). Knowledge of machine learning and AI on Databricks (desirable). ...

Kubernetes Strong Git-based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities … React frontend Build and maintain Dockerised microservices deployed to AWS (EKS) Troubleshoot and optimise performance across the stack Support data ingestion and transformation pipelines (Spark-based) Work cross-functionally with product and data specialists Help shape new features and influence engineering standards This role is 100% hands-on engineering ...

Kubernetes Strong Git-based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities … React frontend Build and maintain Dockerised microservices deployed to AWS (EKS) Troubleshoot and optimise performance across the stack Support data ingestion and transformation pipelines (Spark-based) Work cross-functionally with product and data specialists Help shape new features and influence engineering standards This role is 100% hands-on engineering ...

cloud data platforms Required Technical Expertise - Data Migration & Data Synchronisation Architecture - SQL/Python/Scala - Data Engineering & Data Pipelines - Relational & NoSQL Databases - Apache Spark - Databricks - Data Warehousing/Data Lakes - Cloud & On-Prem Data Integration Additional Requirements - Freelance contractor only - Must be eligible for security clearance - Availability ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

team Exposure to sectors such as defence, public sector and consumer/retail Familiarity with modern data and AI tooling such as Python, Spark, Databricks, Azure cloud, and Generative AI platforms (experience with specific tools is helpful but not essential) A mix of technical, product and operational experience - comfortable ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

innovative thinking process and encourage it in others. Working within the agile framework at Chambers Skills and Experience Strong proficiency in SQL , including Spark SQL and MS SQL Server , for querying, data manipulation, and performance optimization. Hands-on experience with Python and PySpark for data processing, transformation, and automation … design, build, and maintain scalable data pipelines for batch and streaming data using modern frameworks. Experience working with Databricks for big data processing, Spark-based transformations, and collaborative analytics workflows. Familiarity with Azure Data Factory (ADF) for data orchestration and integration across cloud environments. Skilled in version control using ...

like ER/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record … with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability to appropriately prioritise ...

brightest minds, and build systems that matter, we want to hear from you. Skills Python (Programming Language) Java Software Development Computer Science C++ SQL Apache Spark Mathematics FastAPI Flask If this sounds like you apply today ...

engineering with stakeholder engagement and solution design. Key Responsibilities Build and optimise data pipelines, ETL processes, and data platforms Develop solutions using Python, Spark, Kafka, Hadoop or similar Wo click apply for full job details ...

will be working across different client setups. Typically, that includes: AWS, Azure and GCP depending on the project Tools like Databricks, Snowflake and Synapse Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kafka or Kinesis You will use your expertise to choose the right ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

power, and deep knowledge of just about every industry. We believe a mix of data, analytics, automation, and responsible AI can do almost anything—spark digital metamorphoses, widen the range of what humans can do, and breathe life into smart products and services. Want to join our crew ...

/ELT) pulling from core platforms, cloud storage, databases and APIs Turn raw, messy, disparate data into curated, analysis-ready datasets Work with Python, Spark and SQL every day Develop models and schemas using a medallion architecture approach Help shape our data architecture with the Data Architect Implement governance … solve a problem you'll fit right in. What you'll bring You'll likely have: Hands-on expertise with Databricks Confidence in Python, Spark and SQL Experience building pipelines in Azure A solid understanding of data modelling and scalable architectures But just as important: You're inclusive ...

data sets can be effectively combined and analysed. They can also utilise programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful - R, Apache Spark. Very useful - Python, SQL, JSON, LakeFS. Reasonable ...

data sets can be effectively combined and analysed. They can also utilise programming languages such as Python, Java or SQL and tools such as Apache Spark or in this instance LakeFS, for data processing and management. Useful - R, Apache Spark. Very useful - Python, SQL, JSON, LakeFS. Reasonable ...

models for performance, scalability, and accuracy. Qualifications: Deep knowledge of neural networks (CNNs, RNNs, LSTMs, Transformers). Strong experience with data tools (Pandas, NumPy, Apache Spark). Solid understanding of NLP algorithms. Experience integrating ML models via RESTful APIs. Familiarity with CI/CD pipelines and deployment automation. ...

experience building APIs at scale ⚙️ You’ll need at least 2 of the below: ☁️ AWS (3+ years) – S3, Lambda, DynamoDB 📊 Big Data (3+ years) – Spark, Snowflake, Kafka 📦 Containers (3+ years) – Kubernetes/OpenShift 💳 Payments experience (2–3 years) – card payments/fintech/transaction systems 👉 We’re being strict ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks ...