1 to 25 of 78 Permanent Apache Jobs in the UK

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop, maintain ...

driving innovation in data engineering. Desirable Criteria Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives. Familiarity with Apache Airflow, data modelling, and metadata management Experience influencing enterprise data strategy and contributing to architectural governance. To apply for this position please submit your ...

with experience in CI/CD pipelines, monitoring, and observability tools (Grafana or equivalent). Exposure to analytics, reporting, and BI tools such as Apache Superset, Lightdash or OpenSearch Willingness to work across the stack by contributing to API development and, at times, UI components (Vue.js, Zoho, or similar ...

implement lakehouse architecture standards (medallion model: bronze, silver, gold), including governance, lineage, and data quality controls Design and manage secure data ingestion frameworks (e.g. Apache NiFi, APIs, SFTP/FTPS) for internal and external data sources Architect and maintain secure AWS-based data infrastructure (S3, IAM, KMS, Glue, Lake ...

critical data at national scale. You must already hold active SC clearance to be considered. Responsibilities Design, build and maintain Databricks data pipelines using Apache Spark (PySpark & Spark SQL) Implement Delta Lake/Lakehouse architectures (Bronze, Silver, Gold) Develop scalable ELT/ETL workflows for batch and streaming data ...

stream data processing. Cloud Pub/Sub: For real-time messaging and event ingestion. Cloud Storage: For scalable and durable object storage. Cloud Composer (Apache Airflow): For workflow orchestration and pipeline management. Cloud Dataproc: For managed Hadoop and Spark clusters (good to have). Cloud SQL/Cloud Spanner ...

clear, structured data engineering framework Build, enhance, and maintain data pipelines within Snowflake Develop transformations and data models using DBT Orchestrate workflows using Apache Airflow Write clean, modular Python code for data engineering solutions Improve pipeline reliability, performance, and observability Introduce and embed CI/CD, testing, and deployment ...

documented data products Nice to Have Financial markets data - market data feeds (Bloomberg, Refinitiv), tick data, trade history, reference data, or instrument master management Apache Spark or Flink for large-scale stream and batch processing beyond the Databricks ecosystem dbt or equivalent SQL transformation layer; experience building and maintaining ...

systems. • Ability to obtain UK Security Clearance (SC). We'd love it if you also have: • Experience with data visualisation tools such as Apache Superset, Jupytr or Power BI • Familiarity with visualisation libraries like D3.js, Chart.js or Plotly for building interactive dashboards. • Exposure to DevOps practices including ...

techniques. Desirable: Experience with cloud platforms (AWS) and containerisation (such as Docker, Podman, Kubernetes). Exposure to big data technologies (Spark, Hadoop) and Apache tools. Knowledge of NLP, computer vision, and deep learning architectures. Familiarity with Agile and DevOps practices. STEM degree or equivalent experience in AI, Data Science ...

computer science, IT or a related field, or in bioinformatics with a demonstrated IT expertise Experience in big data technologies and frameworks, such as Apache Spark, Hadoop or similar platforms Hands-on experience with CI/CD (GitLab CI/GitHub Actions) Familiarity with Java Familiarity with Google Cloud ...

experience working with data warehouses. Solid understanding of ETL/ELT design and data pipeline optimisation. Experience with big data frameworks such as Apache Spark, and exposure to streaming technologies like Kafka. Strong problem-solving skills and the ability to collaborate effectively across teams. What's on offer: Opportunity ...

systems. • Ability to obtain UK Security Clearance (SC). We’d love it if you also have: • Experience with data visualisation tools such as Apache Superset, Jupytr or Power BI • Familiarity with visualisation libraries like D3.js, Chart.js or Plotly for building interactive dashboards. • Exposure to DevOps practices including ...

architectural strategies, blueprints for hybrid and cloud-native solutions ELT/ETL Frameworks & Pipelines Essential Develop robust ELT/ETL pipelines using tools like Apache Airflow, DBT, AWS Glue, Azure Data Factory, or Kafka Connect. Desirable Optimize data transformations for performance, reusability, and modular design (e.g., using SQL/ ...

Airflow or similar orchestration frameworks, with solid grounding in software design principles Strong Python proficiency; familiarity with streaming technologies such as Kafka or Apache Flink is a strong plus Hands‐on experience translating complex business requirements into robust data sourcing and modeling solutions Exposure to data mesh or data ...

delivering customer-facing Enterprise SaaS products at scale. Strong hands-on or architectural experience with modern data platform tooling, including but not limited to Apache Airflow, dbt, Kafka, AWS Glue, AWS Redshift, AWS Sagemaker, AWS Dynamo, AWS Quick Sight. Success leading a data engineering team through complex challenges that ...

have strong hands-on experience of building scalable data pipelines in cloud based environments using tools such as DBT, AWS Glue, AWS Lake Formation, Apache Spark and Amazon Redshift You have a good knowledge of data modelling, ELT design patterns, data governance and security best practices You're collaborative ...

strong focus on scalability, resilience, and performance. Develop and optimise scalable data pipelines , supporting both batch and streaming workloads, using technologies such as Apache Spark . Work extensively with data technologies , leveraging Python and SQL to deliver high-quality analytical and data-driven solutions. Lead the design and delivery ...

Agile Development using SCRUM. Experience of Oracle/Relational Databases and/or Mongo. Experience of GitLab CI/CD Pipelines. Experience of Apache NiFi. Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence. Experience of Elasticsearch and Kibana. Experience of Hibernate. Experience of AWS Services such ...

Agile Development using SCRUM. Experience of Oracle/Relational Databases and/or Mongo. Experience of GitLab CI/CD Pipelines. Experience of Apache NiFi. Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence. Experience of Elasticsearch and Kibana. Experience of Hibernate. Experience of AWS Services such ...

collaboration skills with a positive, adaptable mindset Eligibility to obtain UK Security Clearance (SC) Experience with data visualisation tools such as Power BI, Tableau, Apache Superset, or Jupyter Desirable Experience Exposure to cloud platforms, simulation environments, or defence systems Experience optimising systems for performance, scalability, and reliability Familiarity with ...

Agile Development using SCRUM. Advanced knowledge of Oracle/Relational Databases and/or Mongo. Experience of GitLab CI/CD Pipelines. Experience of Apache NiFi. Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence. Experience of Elasticsearch and Kibana. Advanced knowledge of Hibernate. Advanced knowledge ...

Agile Development using SCRUM. Advanced knowledge of Oracle/Relational Databases and/or Mongo. Experience of GitLab CI/CD Pipelines. Experience of Apache NiFi. Experience in the use of Atlassian Suite - Bitbucket, Jira, Confluence. Experience of Elasticsearch and Kibana. Advanced knowledge of Hibernate. Advanced knowledge ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...