1 to 25 of 46 Apache Jobs in the South East

senior stakeholders Required Experience 10+ years' experience across Data Engineering or closely related data disciplines Strong hands-on Python development experience Strong PySpark/Apache Spark engineering experience Extensive hands-on experience with Databricks , including notebooks, jobs/workflows and Delta Lake Advanced SQL and relational database experience Strong ...

with the Lead Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement ...

Data Engineering. Strong commercial experience with Python. Advanced hands-on knowledge of Databricks, including notebooks, workflows, jobs, and Delta Lake. Strong experience with Apache Spark/PySpark. Proven track record building and maintaining scalable ETL pipelines. Strong SQL and data modelling expertise. Experience working within AWS, Azure ...

Excellent Python/PySpark and SQL data engineering skills Strong experience designing and developing ETL/ELT and large-scale data pipelines Experience with Apache Spark and Databricks Lakehouse architectures Strong knowledge of Azure Data Lake Storage (ADLS) Experience integrating Databricks with wider Azure data and analytics services Understanding ...

warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering workflows - code generation, code review, pipeline debugging, or documentation Strong coding experience ...

knowledge of optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

knowledge of optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

knowledge of optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and “infrastructure as a code” tools ...

critical data ecosystem while supporting the delivery of high-value data products. Key Responsibilities Design, build and maintain scalable data pipelines using Databricks and Apache Spark. Develop and optimise batch and streaming data workflows, ensuring reliability, scalability and performance. Contribute to the design and implementation of modern Lakehouse data ...

production problems and taking responsibility for critical infrastructure. Startup Mindset: Proactive, resourceful and comfortable taking ownership in a fast-moving engineering environment. Bonus : with Apache Airflow, real-time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial-market systems. Benefits An Ambitious, Collaborative Culture: Join ...

platforms, ideally Microsoft Azure. Experience using technologies such as Spark, Databricks and data lake architectures. Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake. Experience with containerisation and orchestration technologies, including Docker and Kubernetes. Experience using Git/GitHub and working within collaborative development ...

. Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows and pipelines Understanding of common data transformation and storage formats, e.g. Apache Parquet. Awareness of data standards such as GA4GH ( ) and FAIR (). Experience with Spark, Databricks, data lakes. Follow best practices like code reviews, clean ...

. Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows and pipelines Understanding of common data transformation and storage formats, e.g. Apache Parquet. Awareness of data standards such as GA4GH ( ) and FAIR (). Experience with Spark, Databricks, data lakes. Follow best practices like code reviews, clean ...

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will be joining … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure data ...

Monitor and maintain - Monitor data systems for performance issues and make any necessary updates. What You’ll Bring ETL processing languages such as Python Apache Spark/NiFi/Kafka Relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra) AWS and data-related services Palantir Foundry ...

Experience of working with MOD Accreditors; and Knowledge of Onsite/project tech stack includes but is not limited to Azure, AWS, Docker, Kubernetes, Apache (NiFi, Kafka), NodeJS, Typescript, MongoDB, AI, Machine Learning etc. Just as important are curiosity, strong problem solving ability, and the motivation to continuously improve ...

help design and deliver cutting-edge data solutions for customers operating in some of the UK's most complex and challenging environments. Using Databricks, Apache Spark and cloud-native technologies, you’ll develop scalable Lakehouse architectures and production-grade data pipelines that enable organisations to trust, share and exploit … opportunity to grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data ...

MySQL, PostgreSQL, MongoDB) Experience with software delivery and CI/CD Experience with software service metrics and monitoring tools such as Grafana, Plausible and Apache Superset. Familiarity with Linux server administration and networking Experience directly interacting with end users, replying professionally to queries and questions Enthusiasm for growing ...

MySQL, PostgreSQL, MongoDB) Experience with software delivery and CI/CD Experience with software service metrics and monitoring tools such as Grafana, Plausible and Apache Superset. Familiarity with Linux server administration and networking Experience directly interacting with end users, replying professionally to queries and questions Enthusiasm for growing ...

Feature engineering. Exposure to common data analysis and ML technologies such as on scikit-learn, pandas, NumPy, XGBoost, LightGBM. Exposure to tools like Apache Oozie, Apache Airflow. £70,000 - £105,000 a year Why you’ll love it here: We want to make Prima a happy and empowering ...

Feature engineering. Exposure to common data analysis and ML technologies such as on scikit-learn, pandas, NumPy, XGBoost, LightGBM. Exposure to tools like Apache Oozie, Apache Airflow. £70,000 - £105,000 a year Why you’ll love it here: We want to make Prima a happy and empowering ...

Infrastructure-as-Code using Terraform Work with containers, orchestration and troubleshooting Support Linux infrastructure and PostgreSQL Manage load balancers including AWS ALB, NGINX and Apache Troubleshoot infrastructure issues, including memory faults and server crashes Manage Active Directory, SSL certificates and secure communications Support networking across AWS, including VPCs, IGWs ...

cloud cost management. What we're looking for: Strong commercial experience as a Data Engineer working with Databricks in production. Excellent knowledge of Apache Spark, Delta Lake and Databricks Workflows. Strong Python and SQL skills. Understanding of Lakehouse architecture, ETL/ELT and distributed data processing. Databricks certification highly ...

experience, conversion, and operational efficiency. What we are looking for Essential: · Demonstrable experience as a Full-Stack PHP Developer , with strong expertise across PHP, Apache/Ngnix , SQL, and Linux environments. · Proven experience developing and integrating RESTful APIs and third-party services, alongside deploying and managing applications within Cloud ...