51 to 75 of 304 Apache Jobs in England

knowledge of optimisation techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

Skills MUST HAVE: Strong hands‐on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud ...

skills; experience with Oracle databases . Experience designing and supporting ETL/ELT pipelines and data quality frameworks . Working knowledge of Autosys& Apache Airflow Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift) Familiarity with AWS S3 for large-scale data storage. Experience ...

decisions, documentation and continuous improvement initiatives. Key Skills & Experience Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. ...

skills Desirable Experience Experience with any of the following would be advantageous: * Angular, React or Vue * Chart.js, D3.js or other data visualisation tools * Databricks, Apache Spark or distributed data processing * Docker, Kubernetes or Cloud Run * Terraform or other infrastructure-as-code technologies * GPU or accelerated computing technologies such ...

Software Engineer on large software applications. Proficient in technologies including Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, and Apache Spark. Experience working with one or more of the following database systems: DynamoDB, DocumentDB, MongoDB. Demonstrated expertise in unit testing and tools such as JUnit ...

TypeScript), Bash Frontend: Vue CI/CD & Automation: Jenkins, GitLab CI, Ansible, Cucumber Cloud & Platforms: AWS, OpenShift, Linux Infrastructure as Code: Terraform Data & Streaming: Apache NiFi Monitoring & Logging: Grafana, ELK Stack, SonarQube Version Control: GitLab Automation & Infrastructure as Code (IaC) Automate infrastructure provisioning, configuration, and CI/CD pipelines ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and "infrastructure as a code" tools ...

project stakeholders; Strong problem-solving skills; Ability to refine problem statements and propose solutions considering effort, impact, and scalability trade-offs; Nice to have: Apache Spark, Iceberg, Kafka, dbt, Scikit-Learn, XGBoost, PyTorch, MLFlow, GraphFrames, Ray, AWS S3, EMR, SageMaker, Lakeformation, Terraform, Docker, GitHub CI/CD, Knowledge Graphs ...

critical data ecosystem while supporting the delivery of high-value data products. Key Responsibilities Design, build and maintain scalable data pipelines using Databricks and Apache Spark. Develop and optimise batch and streaming data workflows, ensuring reliability, scalability and performance. Contribute to the design and implementation of modern Lakehouse data ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

root‐cause analysis Experience building or supporting production data pipelines in a collaborative engineering environment Experience with distributed data processing frameworks such as Apache Spark Working knowledge of JSON, Avro and Parquet Nice to have: technical design ownership across multiple datasets or pipeline domains, experience guiding implementation standards ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open‐source frameworks Familiarity with AWS‐native data services such as Glue, Lake Formation, Redshift, or Athena Exposure to data governance practices ...

written fluency in English is required — we're a distributed team and written communication is important. Preferred Qualifications Airflow for orchestration. Experience with Apache Hudi. Experience with financial, transactional, or payment data. What We Offer at Yuno Competitive Compensation. Remote Work – You can work from everywhere! Home Office Bonus ...

validate transformations and trace data across platforms. Tooling/Delivery Methods Working knowledge of scheduling/orchestration tools such as Autosys and/or Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift ...

into a coherent operational story. Data engineering & processing systems: You have built, operated, or supported production data pipelines using distributed processing systems such as Apache Spark or Databricks, and understand common failure modes, performance tradeoffs, and operational challenges in batch and hybrid pipelines. Data platforms & analytics systems: You have ...

tooling, or cloud cost management across at least one major cloud provider (Azure, AWS, or GCP). Technical Skills Data engineering: strong proficiency with Apache Spark, dbt, Airflow/Azure Data Factory or equivalent orchestration, Data Lake/Apache Iceberg, and advanced SQL for large‐scale transformations. Microservices ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools ...

Azure SQL Database, or Azure Data Factory Experience with Python or another object-oriented programming language Experience with Azure Databricks and/or Apache Spark Experience with reporting or visualisation tools such as Power BI, MicroStrategy, or similar platforms Experience with CI/CD workflows, including automated build, test ...

tooling, automated testing and reliable deployment processes. Startup Mindset: Proactive, resourceful and comfortable taking responsibility in a small, fast‐moving engineering team. Experience with Apache Airflow, Kubernetes, real‐time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial‐market systems would be useful ...