1 to 25 of 218 Apache Jobs

environments. Strong understanding of system design, scalability, and clean coding practices. Familiarity with application monitoring solutions Data Engineering 5+ years building scalable and reliable data pipelines using Databricks and Apache Spark. We need both. Experience with cloud data warehouse/data lake architectures and tooling. Experience building solutions using cloud providers, preferably Azure (AWS/GCP also valued). More ❯

in software engineering practices (e.g., version control, CI/CD pipelines, containerization). Experience building and optimizing data pipelines, ETL processes, and real-time streaming solutions using tools like Apache Airflow, Kafka, Spark, or equivalent. Knowledge of applied AI principles, particularly in implementing AI systems for operational decision support and analyzing unstructured data (e.g., text, imagery). Ability to More ❯

data applications using the latest open-source technologies. Desired working in offshore model and Managed outcome Develop logical and physical data models for big data platforms. Automate workflows using Apache Airflow. Create data pipelines using Apache Hive, Apache Spark, Apache Kafka. Provide ongoing maintenance and enhancements to existing systems and participate in rotational on-call support. More ❯

. AWS Platform Build: Demonstrable experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data processing, scripting, and automation). More ❯

. AWS Platform Build: Demonstrable experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data processing, scripting, and automation). More ❯

orchestration. Familiarity with data cataloging and governance tools (AWS Glue Catalog, Lake Formation). Knowledge of data warehouse design patterns and best practices. Experience with data orchestration tools (e.g., Apache Airflow, Step Functions). Working knowledge of Java is a plus. Education B.S. in Computer Science, MIS or related degree and a minimum of five (5) years of related More ❯

Azure, or GCP, with hands-on experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR, ISO27001). Hands-on More ❯

the delivery of robust, cloud-native solutions that support analytics, automation, and digital transformation. Data Engineering, Full Stack & Platform Development Design and implement scalable data pipelines using tools like Apache Spark, Airflow, or dbt.Build and maintain data lakes, warehouses, and real-time streaming solutions. Develop APIs and microservices to expose data securely and efficiently. Ensure data quality, governance, and More ❯

London office. About you : You have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI, Django, Airflow, Kafka, ETL, CI More ❯

Nottingham office. About you : You have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI, Django, Airflow, Kafka, ETL, CI More ❯

services (e.g., AWS Glue, S3, Lambda, Snowflake). Advanced knowledge of SQL and experience with modern data warehousing and database performance tuning. Familiarity with distributed data processing technologies (e.g., Apache Spark, Hadoop). More ❯

Development Opportunities Enhanced Maternity & Paternity Charity Volunteer Days Cycle to work scheme And More.. DATA ENGINEER - ESSTENTIAL SKILLS Proven experience building data pipelines using Databricks . Strong understanding of Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake/Delta Live Tables and More ❯

AWS (S3, Lambda, Glue, Redshift) and/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA, MES, and Historian systems. Information Modeling:Experience in defining semantic More ❯

understanding of data modelling, warehousing, and performance optimisation. Proven experience with cloud platforms (AWS, Azure, or GCP) and their data services. Hands-on experience with big data frameworks (e.g. Apache Spark, Hadoop). Strong knowledge of data governance, security, and compliance. Ability to lead technical projects and mentor junior engineers. Excellent problem-solving skills and experience in agile environments. More ❯

Framework principles (security, cost, performance, reliabilityData Engineering & Pipelines Build and optimize data pipelines in AWS using Glue, Lambda, Step Functions, EMR, Athena, and S3. Implement DAG-based orchestration using Apache Airflow, AWS Managed Workflows (MWAA), or Glue Workflows. Ensure data quality, reliability, lineage, and observability across all pipelines.Machine Learning Pipeline Enablement Collaborate with ML teams to productionize models, including More ❯

a Senior Principal Engineer or equivalent in enterprise data architecture, data engineering, or data platform strategy. • Deep expertise in building and modernizing distributed data platforms using technologies such as Apache Spark, Kafka, Flink, NiFi, and Cloudera/Hadoop. • Hands-on proficiency with modern data platforms and tools including Databricks, Snowflake, Delta Lake, and Redshift. • Experienced in designing and operating More ❯

delivery oriented, and applies critical thinking to create innovative functions and solve technical issues. In this role, you will: Utilize Microsoft Azure services including Azure Data Factory, Synapse Pipelines, Apache Spark Notebooks, Python, SQL, stored procedures to develop high performing data pipelines. Continuously improve and optimize the automation toolset for reliability, scalability, and adaptability. Research and implement cutting-edge More ❯

e.g., PL/SQL, T-SQL, etc.) Proficiency in programming/scripting languages (e.g., Python, Java, PowerShell, etc.) Proficiency in data engineering technologies and tools (e.g., Azure Data Factory, Apache Spark, Azure Synapse Analytics, Python, Airflow, etc.) Strong attention to detail and organization skills Strong customer service skills and phone and email etiquette USG Core Values The University System More ❯

Google Cloud, Databricks) are a strong plus Technical Skills: • Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL) • Familiarity with data pipeline and workflow management tools (e.g., Apache Airflow) • Experience with programming languages such as Python, Java, or Scala. Python is highly preferred • Basic understanding of cloud platforms and services (e.g., AWS, Azure, Google Cloud) • Knowledge of More ❯

and managing cloud infrastructure as code Proficiency in programming languages such as Python, Spark, SQL Strong experience with SQL databases Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF) Experience with cloud platforms (Azure preferred) and related data services Excellent problem-solving skills and attention to detail Inclusive and curious, continuously seeks to build knowledge and More ❯

Google Cloud Platform (GCP) - especially BigQuery, Cloud Storage, Dataproc, Pub/Sub, and Composer (Airflow). Expertise in Python and SQL for ETL, data transformation, and orchestration. Experience with Apache Airflow DAG development and scheduling best practices. Strong understanding of data warehouse and data lake concepts. Knowledge of Spark or PySpark for big data processing. Familiarity with CI/ More ❯

frameworks Confident using Git and working within Agile/SCRUM teams Experience mentoring junior developers Knowledge of Oracle/relational databases, MongoDB, and GitLab CI/CD Familiarity with Apache NiFi, JavaScript/TypeScript, and React Experience with Elasticsearch, Kibana, Hibernate, and the Atlassian suite (Bitbucket, Jira, Confluence) Desirable; Experience with JSF (PrimeFaces) Knowledge of AWS and cloud-ready More ❯

to-end. • Comfortable engaging with stakeholders, understanding business requirements, and translating them into technical design. • Strong Experience in Databricks development/migration. (this is for Databricks engineer) • Experience with Apache Spark, Databricks Delta Lake, Unity Catalog, and MLflow.( this is for Databricks engineer) Accountabilities • Build and maintenance of data pipelines that enable the transfer and processing of durable, complete More ❯

to-end. • Comfortable engaging with stakeholders, understanding business requirements, and translating them into technical design. • Strong Experience in Databricks development/migration. (this is for Databricks engineer) • Experience with Apache Spark, Databricks Delta Lake, Unity Catalog, and MLflow.( this is for Databricks engineer) Accountabilities • Build and maintenance of data pipelines that enable the transfer and processing of durable, complete More ❯

to-end. • Comfortable engaging with stakeholders, understanding business requirements, and translating them into technical design. • Strong Experience in Databricks development/migration. (this is for Databricks engineer) • Experience with Apache Spark, Databricks Delta Lake, Unity Catalog, and MLflow.( this is for Databricks engineer) Accountabilities • Build and maintenance of data pipelines that enable the transfer and processing of durable, complete More ❯