201 to 225 of 458 Apache Airflow Jobs in England

data scientists in the "PyData" ecosystem (Numpy, Pandas) is desirable. Familiarity with some of the other tools in our stack is desirable: Redis Postgres Airflow Spark This is a hybrid role based in London, UK. About Arrive Arrive, including brands like EasyPark, Flowbird, RingGo, ParkMobile and Parkopedia ...

auction systemsExcellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams.Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructureExperience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones.What we can offerBonusHybrid working ...

data modelling Databricks as our datalake platform Kubernetes for data services Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data ...

systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

MLflow) Experience with cloud‐based ML tools (e.g., SageMaker), data and model versioning (e.g., DVC), CI/CD (e.g., GitHub Actions), workflow orchestration (e.g., Airflow/Dagster) and containerised solutions (e.g., Docker, ECS) nice to have Experience in code testing (unit, integration, end‐to‐end tests) Strong data engineering ...

business environments is a plusWhile we advocate using the right tech for the right task, we often leverage: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureFamiliarity with Generative ...

these things but think you fit the main criteria, we’d still love to hear you.Creating data pipelines and scheduling regular batch jobs (in Airflow or equivalent).Working with cloud environments in an engineering capacityMarketing orchestration/ad tech tools/stacks (Salesforce/Braze/Adobe/DSPs ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift Why else ...

data challengesWhile we advocate using the right tech for the right task, we often leverage the following technologies: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such as AWS, GCP, and AzureExperience with Generative ...

stakeholders to the right data solutions Familiarity with using AI tools to accelerate your workflow Nice to Have Experience with orchestration tools like Dagster, Airflow, or similar Experience building real-time or near-real-time data pipelines Familiarity with machine learning workflows and feature engineering Experience embedded in teams ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows.Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systemsWhat ...

expertise , particularly in complex analytical and warehouse queries Proficiency in Python and/or Java Experience with Data Warehouses, such as Snowflake or Apache Iceberg Practical knowledge deploying and operating containerised workloads using Docker and Kubernetes Knowledge of streaming tooling (Kafka, Kafka Connect, Flink,...) Experience with ETL/… tooling (dbt, Airflow, Airbyte, dlt,..) Hands-on experience provisioning and managing cloud infrastructure with Terraform Experience with CI/CD pipelines, automated testing and Git-based development workflows Familiarity with observability practices, including logging, metrics, alerting and production troubleshooting. Strong grasp of software engineering principles and best practices ...

using dbt Core, with a strong grasp of top-down/bottom-up data modelling methodologies (Kimball, OBT, or equivalent). Build and maintain Airflow-orchestrated ELT workflows across the retail media data platform, including attribution pipelines supporting campaign performance and incrementality measurement. Contribute to Retail Media event tracking … design: batch vs. streaming, ETL/ELT, schema design, partitioning, and efficient storage patterns. Hands-on experience with dbt Core (or dbt Cloud) and Airflow for orchestration and transformation at scale. Solid grounding in data warehouse and data lake architecture, including dimensional modelling and layered architectures (Gold/Silver ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Preferred: Experience in one or more relevant ...

Familiarity with PostgreSQL and Snowflake Preferred: Familiarity with Web Frameworks such as Django, Flask or FastAPI Preferred: Familiarity with event streaming platforms such as Apache Kafka Preferred: Familiarity with data pipeline platforms such as Apache Airflow Preferred: Familiarity with Java Benefits and Perks We value the total ...

well as present complex findings clearly to both technical and non-technical audiencesProficient in using Tech Stack such as Amazon S3, Lambda function, Kafka, Apache Airflow or Bloomberg Teck Stack in production environmentExceptional problem-solving skills, numerical proficiency and high attention to detailAbility to work independently as well ...

Skills and knowledge Demonstrable experience designing, deploying, and operating big data platforms at scale. Hands‐on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies). Knowledge of cloud platforms and traditional on‐premise hosting; hands‐on experience with Azure, AWS and/ ...

asubject, preferred knowledge of AI infrastructure. Proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Proven experience ...

preferred knowledge of AI infrastructure. Good proficiencyin programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Significant experience ...

/silver/gold layers, and partner with analytics, ML, and trading teams to deliver trusted data assets. You will work with dbt, Spark, Airflow, Kafka, and streaming technologies to ensure data quality, governance, and efficient storage. #J-18808-Ljbffr ...

team mentorship. Key Skills: Strong experience in a Lead/Senior Data Engineering role Expertise with GCP (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow) Advanced Python and SQL Experience building scalable batch and real-time data pipelines Strong data modelling, CI/CD, Git, and Terraform knowledge Experience ...

developing production-grade data pipelines Strong understanding of REST APIs, webhooks, OAuth, and event-driven architectures Familiarity with orchestration tools such as Airflow, Prefect, or Dagster Cloud platform experience with AWS, GCP, or Azure Hands-on experience with containerisation technologies like Docker and Kubernetes Knowledge of data integration patterns ...

Role You will: Own and develop the infrastructure supporting AI and ML services. Build and maintain orchestration pipelines using tools such as Dagster, Airflow or Prefect. Deploy, manage and optimise workloads in Kubernetes environments. Improve monitoring, observability and platform reliability. Implement infrastructure-as-code and CI/CD best ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

equivalent. Experience with big data sets and big data warehouses Experience with proxies, anti-bot measures and scraper monitoring. Experience with Dagster, Airflow, Prefect or another data orchestrator. Comfort working with TypeScript/Node.js. A pragmatic, hands-on approach and strong problem-solving skills. Desirable: Geospatial data experience (PostGIS ...