76 to 100 of 299 Apache Jobs in England

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP) Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink) Experience leading technical customer conversations - discovery, whiteboarding, architecture reviews Familiarity with one or more: data engineering ...

Security In-depth knowledge of the financial services industry and their IT systems Practical cloud native experience Preferred qualifications, capabilities, and skills Advanced Apache Spark experience(PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing patterns. Deep lakehouse ...

asubject, preferred knowledge of AI infrastructure. Proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

data Experience with data visualization tools to create dashboards and reports Understanding of data engineering concepts (ETL, data pipelines) and automation tools such as Apache Airflow Familiarity with cloud based computing resources such as AWS Interviewing with Veeva We value your time and believe in a transparent hiring process. ...

taking customers from EDW to the full capabilities of the Data Intelligence Platform. Demonstrated competence in the Lakehouse architecture including hands-on experience with Apache Spark, Python and SQL Excellent communication skills; both written and verbal Public Cloud Architecting: Hands-on experience designing and implementing big data architectures within ...

platforms:AWS,Azureand GCP Modern data platforms:Databricks, Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation … quality:dbt, Great Expectations and equivalent tooling Orchestration:Apache Airflow and cloud-native orchestration services Event streaming:Kafka and cloud-native event-streaming technologies Data modelling:Dimensional,analyticaland domain-oriented modelling Data governance:Catalogue, metadata, lineage, quality, security,privacyand access controls Data products:Reusable, discoverable and well-governed dataproductsand ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure data ...

owned infrastructure. You’ll work as part of a technical team building and supporting data pipelines using Python, Azure and Kubernetes, alongside technologies including Apache Airflow and Apache Spark. Key Responsibilities Build and maintain new and existing data pipelines Develop data solutions using Python, Azure and Kubernetes Work … with Apache Airflow and Apache Spark Support the ongoing development and improvement of the data platform Ensure data remains securely hosted on MoD-owned infrastructure Collaborate with wider technical and delivery teams Troubleshoot and resolve technical issues across existing data pipelines Required Experience Strong commercial experience ...

cloud development. Exposure in hosting and setting up container-based applications. Experience is Api Gateway Management services Exposure to Caching Framework like (Redis, Apache Ignite etc) Exposure to Log Monitoring Tools like (Elastic Search & Splunk etc ) The ability to understand and support, modify, and maintain systems and code developed ...

design, build, and support scalable data pipelines powering Moody’s next-generation digital content platform. You will develop data workflows using Databricks, Snowflake, Apache Airflow, dbt (SQL), and Python within AWS to turn diverse data sources into reliable analytics. You will collaborate with cross-functional teams and mentor peers ...

security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement Why Apply ...

modern data engineering best practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence ...

Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python ...

Monitor and maintain - Monitor data systems for performance issues and make any necessary updates. What You’ll Bring ETL processing languages such as Python Apache Spark/NiFi/Kafka Relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra) AWS and data-related services Palantir Foundry ...

/CD paradigms and systems such as: Ansible, Terraform, Jenkins, Bamboo, Concourse etc. Observability – SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Excellent knowledge of YAML or similar languages The following Technical Skills & Experience would be desirable: Jupyter Hub Awareness RabbitMQ ...

databases Experience with one of Azure/AWS/GCP ETL tools like Informatica, Alteryx and Business Glossary tools like Collibra, Azure Purview or Apache Atlas. TDD/BDD, Experience working in an Agile development environment Identity management and synchronisation will be an add on for the resources Experience ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache Spark and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

stakeholder engagement, and a passion for national security. Ideal candidates will have knowledge in some of the following: ETL processing languages such as Python Apache Spark/NiFi/Kafka Relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra) AWS and data-related services Palantir Foundry ...

Experience of working with MOD Accreditors; and Knowledge of Onsite/project tech stack includes but is not limited to Azure, AWS, Docker, Kubernetes, Apache (NiFi, Kafka), NodeJS, Typescript, MongoDB, AI, Machine Learning etc. Just as important are curiosity, strong problem solving ability, and the motivation to continuously improve ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

preferred knowledge of AI infrastructure. Good proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts across financial domains ...

integration, and production support. Good understanding of ETL/ELT, data warehousing, data lakes, dimensional modelling, batch processing, and stream processing. Experience with Apache Beam, Kafka, REST APIs, Docker, Kubernetes, or GKE. Experience with data formats such as JSON, CSV, Avro, and Parquet. Working knowledge of Git, automated testing … observability, and release management. Ability to design solutions for scalability, resilience, security, performance, operability, and cost efficiency. Preferred Skills Experience with Cloud Composer or Apache Airflow, Dataproc or Spark, Dataform or dbt, and metadata-driven pipeline frameworks. Exposure to Terraform or another infrastructure-as-code tool. Knowledge of Dataplex ...

Linux operating systems, including command-line administration and troubleshooting. Solid experience with SQL or data pipeline technologies, Experience with data processing frameworks such as Apache Airflow, Apache Flink or Google Dataflow, with a practical understanding of ETL pipeline design and the ability to diagnose issues across data transformation …/IP networking principles, including the ability to diagnose connectivity issues and interpret network traffic. Familiarity with event streaming platforms such as Apache Kafka, including an understanding of producer/consumer patterns and how message queues are used to move data reliably between systems at scale. Familiarity with infrastructure ...

Spring Boot, REST APIs and microservices Experience with a relational database like MS SQL Server, PostgreSQL, Sybase ASE Experience with Streaming technologies like Kafka, Apache Flink Experience with cloud technologies, such as Microsoft Azure Experience with Agile process Experience with DevOps and tools like Azure DevOps Our benefits ...