1 to 25 of 108 Apache Jobs in the UK excluding London

supporting Spark‐based platforms (e.g. EMR, Glue, Databricks) (Preferred)Proficient with DevOps tools and practices (Git, Jenkins, Docker, Kubernetes) as well as ETL frameworks (Apache Spark, Airflow, AWS IoT Analytics/Device Management/Events). (Preferred)Programming expertise in Python, Java, Scala, SQL, Spark, Unix shell scripting ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs. Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

development and automation capabilities, to improve the value realized by automationRequired qualifications, skills and capabilitiesExperience with building data pipelines on Databricks and/or Apache Spark in production.Strong coding skills in Python (PySpark) and SQL (Scala a plus).Hands-on experience with Delta Lake (MERGE/UPSERT patterns, schema ...

technologies as our stack evolves.Nice to HaveExperience with stream processing frameworks such as Flink or Spark Streaming.Experience with task orchestration tools such as Apache Airflow or similar systems.Familiarity with additional technologies: GraphQL, React, HTML5, JavaScript, CSS, Postgres, Gradle, BERT.Experience working with and designing infrastructures for large-scale data processing ...

stack evolves. Nice to Have Experience with stream processing frameworks such as Flink or Spark Streaming. Experience with task orchestration tools such as Apache Airflow or similar systems. Experience working with and designing infrastructures for large‐scale data processing (Hive, Snowflake, NoSQL databases). Experience with data governance practices ...

automation capabilities, to improve the value realized by automation Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT ...

understanding of Quantexa components such as: Entity Resolution (ER) Scoring Framework Contextual Network Generation UI Configuration Strong background in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable ...

understanding of Quantexa components such as: Entity Resolution (ER), Scoring Framework, Contextual Network Generation, UI Configuration. Strong background in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch. Experience with data modelling and design in the context of large-scale analytics platforms. Strong stakeholder management and communication skills – capable ...

large-scale data processing.Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning.Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar).Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.).Strong understanding of data quality, security, and lineage best ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern data ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Python skills * Hands-on experience designing and developing ETL pipelines and data workflows * Experience designing and maintaining data pipelines using technologies such as Apache Airflow or Spark * Experience working with cloud platforms including AWS, Azure or GCP * Strong understanding of data modelling, integration and governance principles * Experience working with ...

standards across the team Essential: 7 years designing and building robust, scalable, distributed data systems and pipelines Strong experience with orchestration tools such as Apache Airflow or Dagster Experience with dbt, Spark, SQL, Athena/Trino, Redshift, Snowflake, PostgreSQL/MySQL Experience with public cloud (AWS, GCP or Azure ...

range of projects that include internal and external suppliers. Essential Skills Proven experience as a Data Engineer Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience with Microsoft Fabric Experience of using programming languages such as Python, Scala and SQL. Demonstrable knowledge ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with AI evaluation and observability practices ...

secure handling of inputs/outputs, and adherence to resiliency and security expectationsPreferred qualifications, capabilities, and skillsExperience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworksFamiliarity with AWS-native data services such as Glue, Lake Formation, Redshift, or AthenaExposure to data governance practices including ...

SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with emerging technologies in data engineeringAbility ...

Redshift).Semantic Layer: Design and implementation for governed reusable data views.Data Governance Tools: Collibra, Alation, Talend or equivalent.Expertise in ETL tools and processes (e.g., Apache NiFi, Talend, Informatica)Security: Data Privacy, Data Protection & Encryption frameworks & governanceEnterprise Architecture Frameworks: TOGAF, Zachman; familiarity with ITIL processes.Additional Helpful Skills:Cloud Platform Awareness ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

Have: Relevant certifications such as AWS Certified Data Analytics or Google Professional Data Engineer Knowledge of data governance tools such as Collibra or Apache Atlas Familiarity with encryption standards and key management, such as AES256, TLS, KMS or Vault Why Apply? Competitive salary and a strong benefits package Hybrid ...

build out technology roadmapsCodebase ownershipTechnical Application ownershipQualificationsStrong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , HarnessStrong experience of developing on Apache Spark/HadoopStrong experience with AWS cloud services including AWS Glue, S3, StepStrong experience building data pipelines and data lakeExperience of the full development ...

ownership Technical Application ownership Qualifications Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , Harness Strong experience of developing on Apache Spark/Hadoop Strong experience with AWS cloud services including AWS Glue, S3, Step Strong experience building data pipelines and data lake Experience ...

reviews and continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices ...

innovation-led working environment Key Experience Proven experience as a Data Architect, Databricks Architect, Lead Data Engineer or Solutions Architect Strong expertise in Databricks, Apache Spark, PySpark and SQL Experience designing and implementing modern data lake or lakehouse architectures Hands-on experience with Kafka Strong understanding of AWS data ...