1 to 25 of 69 Apache Jobs in England

supporting Spark‐based platforms (e.g. EMR, Glue, Databricks) (Preferred)Proficient with DevOps tools and practices (Git, Jenkins, Docker, Kubernetes) as well as ETL frameworks (Apache Spark, Airflow, AWS IoT Analytics/Device Management/Events). (Preferred)Programming expertise in Python, Java, Scala, SQL, Spark, Unix shell scripting ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs. Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills: - Proficient in Python and SQL, with experience in Apache Spark. - Strong experience building backend services and APIs. Database Expertise: - Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

design data architectures that facilitate model training and inference.Data Governance Tools: Familiarity with data governance or data cataloguing tools (such as Collibra, Alation, or Apache Atlas) and lineage-tracking systems. Hands-on experience setting up or maintaining a data catalogue, documenting data definitions, or automating data lineage capture ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern data ...

platform and lakehouse transformation journeys What We're Looking For Strong commercial experience in Data Engineering Hands-on experience with Databricks and/or Apache Spark Strong PySpark and SQL development skills Experience designing, building and supporting data pipelines Experience working with Azure or AWS Good understanding of modern ...

market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

Python skills * Hands-on experience designing and developing ETL pipelines and data workflows * Experience designing and maintaining data pipelines using technologies such as Apache Airflow or Spark * Experience working with cloud platforms including AWS, Azure or GCP * Strong understanding of data modelling, integration and governance principles * Experience working with ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

with dbt or Airbyte is preferred. Experience developing production Spark or Hadoop applications Gained experience with a workload scheduling and orchestration tool such as Apache Airflow/Cloud Composer Familiarity with modern DataOps (e.g. quality eng, testing, pipeline deployments etc) practices and IaC (e.g. Terraform) Familiarity with the constellation ...

ownership Technical Application ownership Qualifications Strong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , Harness Strong experience of developing on Apache Spark/Hadoop Strong experience with AWS cloud services including AWS Glue, S3, Step Strong experience building data pipelines and data lake Experience ...

reviews and continuous improvement Skills & Experience * Commercial experience in Data Engineering * Strong Python or Java development skills * Experience with AWS cloud services * Knowledge of Apache Spark, Hadoop, NiFi or similar big data technologies * Experience building ETL/ELT pipelines and processing large datasets * Understanding of software engineering best practices ...

innovation-led working environment Key Experience Proven experience as a Data Architect, Databricks Architect, Lead Data Engineer or Solutions Architect Strong expertise in Databricks, Apache Spark, PySpark and SQL Experience designing and implementing modern data lake or lakehouse architectures Hands-on experience with Kafka Strong understanding of AWS data ...

Understanding of Azure data services and API-based integration. Next.js 14, FastAPI Python, Pipeline orchestration, Data transformation, Data quality, Casbin RBAC, Kuzu knowledge graph, Apache Iceberg, MCP AI server integration and ODCS v2 data contracts layer. Experience in establishing the infrastructure, pipelines, processes, and governance required for production environment ...

security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement Why Apply ...

Essential Skills of the Analytical Platform Engineer Python Kubernetes Docker CI/CD pipelines (GitLab or similar) Desirable skills MLOps and data engineering concepts Apache Spark Scala S3-compatible object storage Graph database technologies If you are an driven and experienced Analytical platform engineer looking for their next exciting ...

well as present complex findings clearly to both technical and non-technical audiencesProficient in using Tech Stack such as Amazon S3, Lambda function, Kafka, Apache Airflow or Bloomberg Teck Stack in production environmentExceptional problem-solving skills, numerical proficiency and high attention to detailAbility to work independently as well ...

Databricks-based Lakehouse that drives real-time data, analytics, and reporting across the Tote.You’ll build streaming and batch data pipelines on AWS using Apache Spark Structured Streaming, Delta Live Tables (DLT), and Kafka (MSK), ensuring our business teams across Liquidity, Product, Marketing, Finance, and Compliance have fast, trusted … looking for:We’re looking for an experienced Data Engineer with proven expertise in building pipelines in Databricks and a strong grasp of Apache Spark, whether in PySpark or Scala, including Structured Streaming. You should have experience with Kafka (MSK) and real-time data ingestion, as well ...

configuration management and an interest in migrating to Ansible and cloud native tooling. Experience managing Kubernetes platforms and deploying containerised applications. Knowledge of GitLab, Apache NiFi, HashiCorp Vault, Trend Micro Deep Security Manager, Ivanti device control, and Ansible Tower/AWX. Windows Infrastructure Experience with Windows Server administration, deployment ...

configuration management and an interest in migrating to Ansible and cloud native tooling. Experience managing Kubernetes platforms and deploying containerised applications. Knowledge of GitLab, Apache NiFi, HashiCorp Vault, Trend Micro Deep Security Manager, Ivanti device control, and Ansible Tower/AWX. Windows Infrastructure Experience with Windows Server administration, deployment ...

Databases: PostgreSQL, Graph Databases ( Neo4j , Kuzu ), and data mesh/metadata management platforms. Modern Tech Stack: Next.js 14, FastAPI (Python), Casbin RBAC, Pipeline Orchestration, Apache Iceberg, MCP AI Server Integration, and ODCS v2 Data Contracts layer. DevOps & Integration: Azure DevOps, CI/CD, Data Transformation/Quality frameworks, ITSM ...

Months contract 2 Days a month onsite - Expenses to Newcastle depends on their hub. Start 3 from offer Experience required AWS Services Terraform Apache Spark Python Airflow Docker Nice to haves Gitlab Security Scanning Agents (eg Trivy, Trend Micro, Wiz etc) Jupyter Notebooks Integration with DWP Integration Team Products ...

Expopsure to data governance frameworks, metadata management, and regulatory complianc It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders ...

preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...