1 to 25 of 161 Apache Jobs in the UK excluding London

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop, maintain ...

however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR ...

delivering large-scale data integration, engineering, and analytics projects Strong expertise in cloud platforms (e.g. Azure, MS Fabric, AWS) and modern data tools (e.g. Apache Spark, Hadoop) Proficiency in programming languages (e.g. Python, SQL, Java) Extensive experience in stakeholder management and collaboration across health and local government to drive ...

PostgreSQL, MySQL) Strong knowledge of data warehouse, data lake, and lakehouse architectures Experience with ETL/ELT frameworks and orchestration tools (e.g. Informatica, Talend, Apache Airflow) Proven experience establishing and maintaining data governance frameworks Strong understanding of data security, access controls, and compliance requirements Excellent problem-solving, communication ...

Oracle, SQL Server, PostgreSQL, MySQL). Knowledge of data warehouse and lakehouse architectures. Familiarity with ETL/ELT and orchestration tools (e.g., Informatica, Talend, Apache Airflow). Experience establishing and maintaining data governance frameworks. Experience complying with data security requirements. Experience designing data models. Excellent problem-solving, communication ...

Charity Volunteer Days Cycle to work scheme And More.. DATA ENGINEER – ESSTENTIAL SKILLS Proven experience building data pipelines using Databricks . Strong understanding of Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

individuals’ desires and capabilities. Essential Skills Proven experience as a Data Architect (or similar/related role) Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience with Big Data Experience with Enterprise level Data sets Experience of using programming languages such ...

from applicants. To be successful as a Data Engineer you should have experience with: Demonstrated experience developing, optimizing, and troubleshooting data processing applications using Apache Spark. Proficiency in writing efficient SQL queries and implementing data transformation pipelines at scale. Strong command of Scala with emphasis on functional programming paradigms. ...

data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems and distributed systems Hands-on with Lakehouse technologies (Apache Iceberg, S3 Tables, StarRocks) Strong grasp of data governance, schema design, and quality frameworks Comfortable leading infrastructure decisions and collaborating across distributed teams This ...

using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format Schema evolution, partitioning, compaction & metadata management Query performance tuning with Athena & Redshift Spectrum Redshift Expertise: Data modeling, distribution styles, sort keys ...

development using SCRUM. Experience with Oracle or other relational databases and/or MongoDB. Experience with GitLab CI/CD pipelines. Experience working with Apache NiFi. Familiarity with Atlassian tools including Bitbucket, Jira, and Confluence. Experience with Elasticsearch and Kibana. Experience with Hibernate. Experience with AWS services such ...

Desired Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar) Building data pipelines (real-time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt Familiarity with recommender systems, transformers, or graph neural networks Exposure to model optimisation techniques (quantisation, distillation, efficient inference) What ...

Agile Development using SCRUM. Experience of Oracle/Relational Databases and/or Mongo. Experience of GitLab CI/CD Pipelines. Experience of Apache NiFi. Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence. Experience of Elasticsearch and Kibana. Experience of Hibernate. Experience of AWS Services such ...

serving platforms such as BentoML, MLflow, or SageMaker . Hands-on work with streaming and batch data pipelines using tools like Spark, Airflow, or Apache Beam . Background in front-end or mobile development , enabling end-to-end solution thinking. Prior experience in healthcare, life sciences, or regulated industries ...

development lifecycle from requirements to deployment Tech Stack Includes: Java, Python, Linux, Git, JUnit, GitLab CI/CD, Oracle, MongoDB, JavaScript/TypeScript, React, Apache NiFi, Elasticsearch, Kibana, AWS, Hibernate, Atlassian Suite What’s on Offer: Hybrid working and flexible schedules (4xFlex) Ongoing training and career development Exciting projects ...

development lifecycle from requirements to deployment Tech Stack Includes: Java, Python, Linux, Git, JUnit, GitLab CI/CD, Oracle, MongoDB, JavaScript/TypeScript, React, Apache NiFi, Elasticsearch, Kibana, AWS, Hibernate, Atlassian Suite What s on Offer: Hybrid working and flexible schedules (4xFlex) Ongoing training and career development Exciting projects ...

solutions.• Pipeline Development: Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools like Apache Spark to ensure seamless data flow.• Monitoring and Failure Recovery: Implement monitoring procedures to detect failures or unusual data profiles and establish recovery processes ...

concepts, Data Factory, Synapse and/or Fabric AWS experience with Redshift, Lambda, and SQL-based analytics services Strong Python skills and experience using Apache Spark Hands-on experience with Databricks Experience designing and maintaining ETL/ELT pipelines Solid understanding of data modelling techniques Experience working in cross ...

architecture, and network stack optimization. Data Infrastructure: Familiarity with the modern data stack, including time-series databases, object stores, and streaming frameworks like Apache Flink or Spark Streaming. Engineering Rigor: You write clean, testable, and reliable code. You understand that in this environment, a system failure can cost millions ...

Agile Development using SCRUM. Advanced knowledge of Oracle/Relational Databases and/or Mongo. Experience of GitLab CI/CD Pipelines. Experience of Apache NiFi. Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence. Experience of Elasticsearch and Kibana. Advanced knowledge of Hibernate. Advanced knowledge ...

Oracle, SQL Server, PostgreSQL, MySQL). Knowledge of data warehouse and lakehouse architectures Familiarity with ETL/ELT and orchestration tools (e.g., Informatica, Talend, Apache Airflow). Experience in establishing and maintaining data governance frameworks Experience complying with data security requirements. Experience designing data models. Excellent problem-solving, communication ...

organisation Supporting a move towards a product-aligned, hub-based operating model Technology Stack Cloud-first environment with Microsoft Azure Fabric Python, PyTorch SQL Apache Spark Azure Data Factory (ADF) Git What We’re Looking For 3–4 years’ experience in AI/ML engineering, data engineering, or applied ...