1 to 25 of 47 Remote/Hybrid Apache Jobs

maintain scalable and robust data pipelines using SQL and Python, and leveraging any of the following: Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow and Pyspark. Lead the integration of complex data systems and ensure consistency and accuracy of data across multiple platforms. Implement continuous integration and continuous ...

role We are seeking a high‐skilled Senior Software Engineer with deep expertise in API Development using Spring Boot while also possessing knowledge of Apache Spark‐based Big Data pipelines using Airflow. This hybrid position bridges software engineering and data engineering, requiring the ability to design, build, and maintain … teams to architect and manage robust, production‐grade data products that power critical analytics and measurement capabilities. You will work with technologies such as Apache Spark, Apache Airflow, Trino, Druid, Spring Boot, StarRocks, and Looker to deliver reliable, high‐performance solutions. The ideal candidate is a proactive, self ...

data estate. Deep expertise in Microsoft Azure cloud services — ADF, ADLS, Synapse, Purview. Databricks — Delta Lake architecture, optimisation and advanced data processing. Apache Spark for large‐scale distributed computing and performance tuning. Microsoft Fabric — OneLake and DirectLake integration. Azure Synapse Analytics for enterprise‐scale data warehousing. Infrastructure‐as‐Code ...

data estate. Deep expertise in Microsoft Azure cloud services — ADF, ADLS, Synapse, Purview. Databricks — Delta Lake architecture, optimisation and advanced data processing. Apache Spark for large‐scale distributed computing and performance tuning. Microsoft Fabric — OneLake and DirectLake integration. Azure Synapse Analytics for enterprise‐scale data warehousing. Infrastructure‐as‐Code ...

role Strong hands-on experience with Snowflake data platform Proficiency in SQL and data modeling techniques Experience with ETL/ELT tools (e.g., dbt, Apache Airflow, Talend, Informatica) Experience with cloud platforms (AWS, Azure, or GCP) Familiarity with data warehousing concepts and best practices Understanding of data governance ...

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

understand where traditional patterns remain appropriate - and where modern approaches add measurable value. Data Engineering & Platform Understanding: Strong knowledge of distributed processing (e.g. Apache Spark). Familiarity with Hadoop ecosystem components. Working knowledge of SQL and NoSQL technologies. Understanding of ETL/ELT design trade-offs. Awareness of DevOps ...

with a backend or software engineering background Desirable: Experience with Snowflake Orchestration tools such as Dagster or Airflow dbt for data transformation and modelling Apache Iceberg or modern table formats Background in high-growth, consumer-facing products SALARY AND BENEFITS Base salary up to £100,000 Competitive benefits package ...

Deliver changes through controlled environments in line with enterprise change and release processes Required Technical Experience Core Technologies Strong hands‐on development experience with Apache Spark Proven experience building Kafka‐based event streaming systems Deep familiarity with Hadoop ecosystems , including Hive Advanced SQL for complex, large‐scale datasets Engineering ...

scaling data platforms, with a strong focus on Azure Cloud technologies (e.g., Azure Data Factory, Azure Synapse, Azure Data Lake, Azure SQL Database, Databricks, Apache Spark). Proven experience leading the implementation of data platform strategies and managing large‐scale data infrastructure initiatives using Azure. Demonstrated experience implementing modern ...

Manchester, UK, to develop scalable APIs and manage big data solutions. Candidates should have 8+ years in software or data engineering, with skills in Apache Spark, Airflow, and a strong foundation in Java, Python, or Scala. The role requires collaboration with cross-functional teams to deliver high-quality data ...

technical trade-offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with (or strong advocacy for) AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. Proficiency ...

lead quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. This role combines hands-on technical leadership with test strategy ownership . … reconciliation, and performance under load Oversee PySpark-based data validation and interrogation activities on AWS EMR Support testing and monitoring of workflows orchestrated through Apache Airflow Drive root cause analysis for production data issues and platform defects Risk Management & Reporting Identify, assess, and communicate quality risks across releases ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

with Python-based development workflows and tooling Technology Stack: Azure Cloud Services C# (.NET 10) Python React.js and modern JavaScript (ES6+) Snowflake (Data Warehouse) Apache Airflow Terraform dbt SQL Server ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … days a week in office Lean principles and Agile development practices Continuous deployment across all microservices Big Data ecosystem and SQL Databases (Apache Airflow, Iceberg, Spark/Thrift, PostgreSQL, Azure Hyperscale) Python as a primary backend language High-level frameworks (Flask, SQLAlchemy, Alembic, Pytest, Socket.IO) Amazon Web Services ...

professionals with strong Python development expertise, cloud engineering experience, and exposure to modern DevOps and data engineering practices. Must Have: AWS services; Terraform; Apache Spark; Python; Apache Airflow; Docker; GitLab; Security scanning agent; Jupyter; Notebooks; Key Responsibilities Design, develop, and maintain scalable Python applications and services Build … optimize cloud-native data processing pipelines Develop and manage workflows using orchestration tools such as Airflow Work with large-scale data processing frameworks including Apache Spark Deploy and maintain containerized applications using Docker Implement infrastructure automation using Terraform Integrate solutions with enterprise data and secure file transfer systems Collaborate ...

Proven track record with data governance, quality, lineage, and security practices.* Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub).* Experience in, or knowledge of, DevSecOps Tooling and Processes.* Self-starter with the ability ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Work closely with Product, Data Science, and senior stakeholders Tech Stack Cloud & DevOps: GCP, GKE, Terraform, GitHub, Cloud Build Data & Processing: GCS, Apache Spark, PostgreSQL, Redis Frontend & Analytics: React, Power BI Languages: Python, R Profile Experience as a Technical Architect/Lead/Principal Engineer Strong background in cloud ...

skills and collaborative approach Degree in a relevant subject (or equivalent experience) Nice to have Exposure to Linux/Unix environments Familiarity with MySQL, Apache, or similar technologies Experience with Python (Flask) or backend integration work Exposure to AWS (Lambda, EC2, API Gateway, etc.) Understanding of microservices or serverless ...

strong desire to learn Python* Familiarity with modern web standards, including HTML5 and CSS3* Experience with Linux or UNIX-like systems* Experience with Apache, PostgreSQL, or ReactJS is a plus* Experience with Python unit testing and full-stack web testing is a plus**Benefits and Perks of working ...

environment and manage multiple projects simultaneously. Hands-on experience working with at least one OLAP-specific engine or analytical data warehouse technology (such as Apache Druid, ClickHouse, Amazon Redshift, Google BigQuery or Snowflake) to build scalable reporting solutions. Confidence can sometimes hold us back from applying for a job. ...