1 to 25 of 125 Apache Jobs

Principal Data Platform Engineer

Hiring Organisation
Anaplan
Location
Manchester, England, United Kingdom
build and manage scalable data pipelines and data analytic solutions that support strategic, ad hoc, and AI-driven data analytics using technologies such as Apache Spark, Apache Flink, Unity Catalog, Apache Hive, Databricks, Apache Iceberg, and Delta Tables. Innovation and Research: Drive innovation by researching … building high-throughput, low-latency systems to support short- and long-running processes at scale. Hands-on experience with event-driven platforms like Apache Kafka, RabbitMQ, Apache Pulsar, and other messaging systems for system integrations and real-time data streaming. Strong background in processing large batch and streaming ...

Senior Software Engineer

Hiring Organisation
TD Bank
Location
Dublin, Ireland
Employment Type
Permanent
Salary
EUR 70,000 - 90,000 Annual
design, scalability, and clean coding practices. Familiarity with application monitoring solutions Data Engineering 5+ years building scalable and reliable data pipelines using Databricks and Apache Spark. We need both. Experience with cloud data warehouse/data lake architectures and tooling. Experience building solutions using cloud providers, preferably Azure ...

Lead Data Engineer

Hiring Organisation
ARC IT Recruitment Ltd
Location
South West London, London, United Kingdom
Employment Type
Permanent, Work From Home
experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

Lead Data Engineer

Hiring Organisation
Arc IT Recruitment
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£90,000 - £100,000 per annum
experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

Data Pipeline Engineer On prem to AWS migration

Hiring Organisation
Pontoon
Location
Bristol, Avon, England, United Kingdom
Employment Type
Contractor
Contract Rate
£700 - £750 per day
experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR ...

Data Software Engineer Python Spark SaaS

Hiring Organisation
Client Server
Location
Nottingham, England, United Kingdom
have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI ...

Sr.Data Engineer

Hiring Organisation
Cognizant
Location
London, United Kingdom
Employment Type
Permanent
/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA, MES, and Historian systems. Information Modeling ...

Data Engineer Manager

Hiring Organisation
Young's Employment Services Ltd
Location
West London, London, England, United Kingdom
Employment Type
Full-Time
Salary
£85,000 - £95,000 per annum, Negotiable
Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Data Engineer Manager

Hiring Organisation
Youngs Employment Services
Location
W12, Shepherds Bush, Greater London, Brook Green, United Kingdom
Employment Type
Permanent
Salary
£85000 - £95000/annum circ £85,000 - £95,000 + Bonus
Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Senior Data Engineer

Hiring Organisation
hackajob
Location
London, England, United Kingdom
Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

Senior Software Engineer (Python) - FinCrime

Hiring Organisation
Zepz
Location
United Kingdom
Java and Spring or are willing to learn on the job. Have strong familiarity with SQL Experience with the following tools and frameworks: Apache Airflow and or data pipelines. Celery FastAPI and/or Flask Postgres (AWS Aurora Ideally) DynamoDB Have experience working in the FinTech industry, particularly ...

Senior DevOps Engineer

Hiring Organisation
BI:PROCSI
Location
United Kingdom
automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. Proficiency ...

Junior Data Engineer

Hiring Organisation
Sagacity
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
Competitive salary
strong plus Technical Skills: • Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL) • Familiarity with data pipeline and workflow management tools (e.g., Apache Airflow) • Experience with programming languages such as Python, Java, or Scala. Python is highly preferred • Basic understanding of cloud platforms and services (e.g. ...

Senior Data Engineer

Hiring Organisation
Burns Sheehan
Location
Essex, England, United Kingdom
equivalent experience) ⭐ Desired Qualifications Cloud certifications (GCP, Azure, or AWS) Experience in a regulated or financial services environment Programming skills in Python, Java, or Apache Beam Familiarity with IaC tools (Terraform, CloudFormation, ARM) Experience guiding or mentoring other engineers Knowledge of data catalogue or governance tools (e.g., DataPlex, Informatica ...

Senior Data Platform Engineer

Hiring Organisation
Humanoid
Location
London Area, United Kingdom
related field. 5+ years of experience in data engineering and data quality. Strong proficiency in Python/Java, SQL, and data processing frameworks including Apache Spark. Knowledge of machine learning and its data requirements. Attention to detail and a strong commitment to data integrity. Excellent problem-solving skills ...

Senior Databricks Engineer

Hiring Organisation
Bibby Financial Services
Location
Banbury, Oxfordshire, South East, United Kingdom
Employment Type
Permanent
Salary
£80,000
code Proficiency in programming languages such as Python, Spark, SQL Strong experience with SQL databases Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF) Experience with cloud platforms (Azure preferred) and related data services Excellent problem-solving skills and attention to detail Inclusive and curious, continuously ...

Data Engineer

Hiring Organisation
RED Global
Location
Belfast, Northern Ireland, United Kingdom
Oracle databases. Extensive experience coding in Python. Understanding of ETL processes and data pipeline design. Familiar with data lakes and their architecture. Experience with Apache Airflow (MWAA) for workflow management. Experience with AWS Athena/PySpark (Glue) for data querying and processing. Proficient in SQL with experience in both ...

Senior Software Engineer

Hiring Organisation
YT Technologies
Location
Gloucester, Gloucestershire, South West, United Kingdom
Employment Type
Permanent
Salary
£75,000
working within Agile/SCRUM teams Experience mentoring junior developers Knowledge of Oracle/relational databases, MongoDB, and GitLab CI/CD Familiarity with Apache NiFi, JavaScript/TypeScript, and React Experience with Elasticsearch, Kibana, Hibernate, and the Atlassian suite (Bitbucket, Jira, Confluence) Desirable; Experience with JSF (PrimeFaces) Knowledge ...

AWS Data Engineer

Hiring Organisation
Digital Minds Technologies Inc
Location
Boston, Massachusetts, United States
Employment Type
Any
Salary
USD Annual
ensuring data quality and security. This role often requires proficiency in programming languages like Python and skills with other technologies such as SQL, Apache Spark, and serverless architectures. Key responsibilities:AWS data engineers are responsible for designing and building data pipelines and developing ETL/ELT processes using tools ...

Senior Data Engineer

Hiring Organisation
Fruition Group
Location
Leeds, West Yorkshire, Yorkshire, United Kingdom
Employment Type
Permanent, Work From Home
Salary
£65,000
compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse and ETL/ ...

Senior Software Engineer

Hiring Organisation
Mane Contract Services
Location
Gloucester, England, United Kingdom
testing frameworks such as JUnit 5 • Git and Agile/Scrum experience • Experience with relational or document databases (Oracle, MongoDB) • GitLab CI/CD, Apache NiFi, Bitbucket/Jira/Confluence • Elasticsearch, Kibana and Hibernate • AWS experience (EC2, EKS, Fargate, IAM, S3, Lambda) • Cloud-focused CI/CD pipelines ...

Data Engineering Lead – Elite HFT Firm - Trading Systems - WFH - London - Up to £600k TC

Hiring Organisation
Mondrian Alpha
Location
London Area, United Kingdom
FICC options trading. Experience with Linux-based, concurrent, low-latency systems. Familiarity with Airflow/Dagster (pipeline orchestration), Kafka (streaming), Delta Lake/Apache Iceberg (data lakes), and relational databases. Exceptional communication skills, comfortable working directly with both engineers and business stakeholders. Degree in Computer Science, Mathematics, Engineering ...

Senior Data Engineer

Hiring Organisation
Made Tech
Location
London Area, United Kingdom
environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

Data Science and AI Industrial Placement Scheme

Hiring Organisation
Lloyds Banking Group
Location
Manchester, North West, United Kingdom
Employment Type
Contract, Work From Home
Contract Rate
£24,000
detection, credit risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

Principal Data Engineer

Hiring Organisation
Made Tech
Location
Manchester, England, United Kingdom
data lake houses and data mesh Strong understanding of best practice DataOps and MLOps Up-to-date understanding of various data engineering technologies including Apache Spark, Databricks and Hadoop Strong understanding of agile ways of working Up-to-date understanding of various programming languages including Python, Scala ...