226 to 250 of 351 Apache Jobs

Data Analytics & Business Intelligence (BI) Experience: 5-10 years Location: United Kingdom - Glasgow, London Skills: Apache Airflow, AWS, Data Lake/Lakehouse, Databricks Certified (Data Engineer, ML), DBT, Python, SQL We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. … PySpark, governed cataloging). Modern Data Stack: Strong proficiency in dbt (Core or Cloud) for data transformation and understanding of modern table formats, specifically Apache Iceberg. Orchestration: Experience building and maintaining data pipelines using Apache Airflow (Astronomer experience is a strong plus). Languages: Expert‐level SQL (ability ...

Data Analytics & Business Intelligence (BI) Experience 5-10 years Location United Kingdom - Glasgow, London Skills Apache Airflow, AWS, Data Lake/Lakehouse, Databricks Certified (Data Engineer, ML), DBT, Python, SQL We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. … PySpark, governed cataloging). Modern Data Stack: Strong proficiency in dbt (Core or Cloud) for data transformation and understanding of modern table formats, specifically Apache Iceberg. Orchestration: Experience building and maintaining data pipelines using Apache Airflow (Astronomer experience is a strong plus). Languages: Expert-level SQL (ability ...

reconciliation and root-cause analysis.Experience building or supporting production data pipelines in a collaborative engineering environment.Experience working with distributed data processing frameworks such as Apache Spark.Working knowledge of common data formats such as JSON, Avro and Parquet.Technology EnvironmentThe role will involve working with a modern and evolving data stack. … should bring relevant experience and the ability to work across comparable technologies.Examples of technologies in scope include:Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/CD tooling, containerised ...

root-cause analysis. Experience building or supporting production data pipelines in a collaborative engineering environment. Experience working with distributed data processing frameworks such as Apache Spark. Working knowledge of common data formats such as JSON, Avro and Parquet. Technology EnvironmentThe role will involve working with a modern and evolving … should bring relevant experience and the ability to work across comparable technologies. Examples of technologies in scope include: Data processing and logic: ANSI SQL, Apache Spark, KafkaData formats: JSON, Avro, ParquetPlatforms and storage: Snowflake, Apache Iceberg, Databricks, Hadoop ecosystem technologies, Sybase IQEngineering and deployment: CI/CD tooling ...

Learning, or AI workloads. Experience with MLOps tooling such as MLflow, Kubeflow, Vertex AI, or similar technologies. Knowledge of workflow orchestration platforms such as Apache Airflow. Experience with data processing technologies such as Pandas, Apache Beam, Spark, or similar frameworks. Strong SQL skills and experience with analytical databases ...

focusing on building and delivering high-quality, data-intensive solutions.Proven experience with the internals of streaming or data integration frameworks, with strong preference for Apache Kafka (or similar pub/sub systems), Kafka Connect, Apache Flink or Apache Beam.Experience developing or extending connectors, sinks, or sources … least one streaming processing framework such as Apache Flink, Beam, or Kafka Connect, with preference for experience building or maintaining production-grade streaming connectors.Hands-on experience with Apache Kafka or similar distributed messaging systems (e.g., Pulsar, Kinesis), including topic design, consumer groups, and performance tuning.Strong understanding of database ...

building and delivering high-quality, data-intensive solutions. Proven experience with the internals of streaming or data integration frameworks, with strong preference for Apache Kafka (or similar pub/sub systems), Kafka Connect, Apache Flink or Apache Beam. Experience developing or extending connectors, sinks, or sources … least one streaming processing framework such as Apache Flink, Beam, or Kafka Connect, with preference for experience building or maintaining production-grade streaming connectors. Hands-on experience with Apache Kafka or similar distributed messaging systems (e.g., Pulsar, Kinesis), including topic design, consumer groups, and performance tuning. Strong understanding ...

coding skills (any version, but we use Oracle) Experience coding in at least one dynamic scripting language (e.g., Python, Node JS, PowerShell or Apache Groovy) Business reporting intelligence experience (using tools like BIRT, Tableau or Power BI) Knowledge of REST APIs and standard HTTP methods and data migration; either ...

automation, and data architects. The role focuses on building scalable data pipelines, migrating hundreds of Teradata scripts to dbt models, and orchestrating workflows with Apache Airflow (Astronomer). #J-18808-Ljbffr ...

ideal candidate will have more than 10 years of experience in backend development and data platform engineering, with strong expertise in Java and Apache Spark. The role offers competitive compensation and benefits, reflecting the company's commitment to employee well-being. #J-18808-Ljbffr ...

similar). Experience with source control systems (e.g., Git). Comfortable working in Visual Studio Code or similar IDEs. Knowledge of web servers (Apache or Nginx). If this is you and you want to know more, please send me your CV and we can arrange a suitable time ...

similar). Experience with source control systems (e.g., Git). Comfortable working in Visual Studio Code or similar IDEs. Knowledge of web servers (Apache or Nginx). If this is you and you want to know more, please send me your CV and we can arrange a suitable time ...

strong skills in several of the following Modern JavaScript GitLab CI/CD Observability Technologies Web API Integration GitLab Version Control Database Technologies OpenShift Apache NiFi Message Broker Technologies Java Web Auth Distributed Architecture Kubernetes SecDevOps Cloud Native Technologies The Interview Process We believe in keeping the interview process ...

some of the following technologies on both Windows and Linux: C++ Javascript Python XML JSON HTML CSS Web backend servers (Angular, Django, NodeJS, React, Apache or similar) Web Sockets Docker TCP-IP/UDP SQL or other databases Dev tools (Wireshark, system log interrogation etc..) Databases Cyber Security Desirable ...

field. Experience with LLM agents, autonomous multi-step reasoning systems, meta-learning, or self-improving ML pipelines. Experience with large-scale data pipelines (e.g., Apache Beam) or foundation model training and fine-tuning at scale. Experience working in research environments or track record of published research in relevant ...

field. Experience with LLM agents, autonomous multi-step reasoning systems, meta-learning, or self-improving ML pipelines. Experience with large-scale data pipelines (e.g., Apache Beam) or foundation model training and fine-tuning at scale. Experience working in research environments or track record of published research in relevant ...

specific area or have broad knowledge across the systems engineering lifecycle, your skills matter. Your Impact Protecting RAF aircraft like Typhoon and Apache Advancing next-gen air combat with Team Tempest and GCAP Developing future electronic warfare systems Harnessing AI/ML to unlock intelligence from vast data What ...

field. Experience with LLM agents, autonomous multi-step reasoning systems, meta-learning, or self-improving ML pipelines. Experience with large-scale data pipelines (e.g., Apache Beam) or foundation model training and fine-tuning at scale. Experience working in research environments or track record of published research in relevant ...

other genome browser Experience working in a multi-person team delivering production services Experience with large-scale biological datasets Experience working with the Apache Parquet file format Experience implementing AI/ML approaches for quality control and/or genome annotation Behaviours we value in our team: You bring ...

engineers should be adept at Extract, Transform, Load (ETL) processes. This includes using frameworks such as the Medallion Architecture, as well as tools like Apache Airflow, Fivetran, Azure Data Factory, and Amazon Glue to automate data workflows and ensure data quality. Familiarity with cloud computing services, such … Cloud Platform, is increasingly important. Data engineers often leverage these platforms for scalable data storage and processing capabilities. Knowledge of big data frameworks like Apache Spark, Beam is crucial for handling large datasets and real-time data processing. Understanding data modelling techniques helps data engineers design efficient data structures ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team where … progression within a growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. ...

complex legacy data pipelines into production‐grade PySpark Building scalable, reusable data movement frameworks across multiple engineering teams Driving best practice around AWS Glue, Apache Spark and Apache Airflow Designing high‐quality testing frameworks covering performance, data quality and reliability Leveraging agentic AI development tools to improve engineering … data is governed, managed and consumed Qualifications Staff, Principal or Lead Data Engineering roles Expertise in Python and PySpark Experience with AWS Glue, Apache Spark and Apache Airflow Cloud‐native data engineering architectures Large‐scale data pipeline modernisation Building reusable engineering patterns within enterprise environments Data testing, validation ...

platforms.Solid Cloud and DevOps knowledge, including Linux/Unix, OpenShift/Kubernetes, and Ansible.Experience building and supporting reactive, event-sourced systems.Hands-on experience with Apache Spark and Apache Flink.Strong understanding of Apache HTTP, Envoy, and REST API design.Experience working with Protobuf, Avro, and Parquet data formats.Knowledge ...

working as a Data Architect within the Defence space. Active DV Clearance and sole UK nationality Strong AWS data lake, platform and security expertise Apache NiFi architecture, governance and pipeline delivery Secure data integration experience across multiple security domains API, file-based, legacy and bespoke integration design and delivery ...

working as a Data Architect within the Defence space. Active DV Clearance and sole UK nationality Strong AWS data lake, platform and security expertise Apache NiFi architecture, governance and pipeline delivery Secure data integration experience across multiple security domains API, file-based, legacy and bespoke integration design and delivery ...