6 of 6 Apache Airflow Jobs in the Thames Valley

in Python with libraries like TensorFlow, PyTorch, or Scikit-learn for ML, and Pandas, PySpark, or similar for data processing. Experience designing and orchestrating data pipelines with tools like Apache Airflow, Spark, or Kafka. Strong understanding of SQL, NoSQL, and data modeling. Familiarity with cloud platforms (AWS, Azure, GCP) for deploying ML and data solutions. Knowledge of MLOps More ❯

members, fostering collaboration, and driving best practices in data engineering . What you bring: Extensive experience in data engineering, including building and optimising data pipelines and distributed systems using Airflow, Dagster, or Kedro . Expertise in cloud platforms such as AWS, GCP, or Azure , along with modern data technologies like Spark, Kafka, or Hadoop . Proficiency in programming languages More ❯

data engineering or a related field. Proficiency in programming languages such as Python, Spark, SQL. Strong experience with SQL databases. Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF). Experience with cloud platforms (Azure preferred) and related data services. There's no place quite like BFS and we're proud of that. And it More ❯

technologies – Azure, AWS, GCP, Snowflake, Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis/MQ/Event Hubs, Kafka … Dataflow/Airflow/ADF etc. Excellent consulting experience and ability to design and build solutions, actively contribute to RfP response. Ability to be a SPOC for all technical discussions across industry groups. Excellent design experience, with entrepreneurship skills to own and lead solutions for clients Excellent ETL skills, Data Modeling Skills Excellent communication skills Ability to define the … skills. A minimum of 5 years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF Desirable Skills Designing Databricks based More ❯

across infrastructure design, deployment, and optimisation. Key Tech Environment Cloud: AWS/Azure/GCP (2+ preferred) Data Platforms: Databricks, Snowflake IaC: Terraform, Pulumi, CI/CD pipelines Orchestration: Airflow, dbt, data ingestion frameworks DevOps: GitHub Actions, Jenkins, Docker, Kubernetes What We're Looking For Strong platform/infrastructure engineering experience within data environments Experience deploying cloud-native data More ❯

Lumi Space is empowering the future prosperity of earth - making space scalable and sustainable using ground-based laser systems. We work with global companies and institutions to build products and services to precisely track satellites and remove the dangers of More ❯