1 to 25 of 135 Remote/Hybrid Apache Jobs in England

computations are represented, how query plans are optimised, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers operate … learn how modern data systems work from the inside and apply that knowledge to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability ...

computations are represented, how query plans are optimized, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers operate … want to advance the capabilities of modern data systems and apply that work to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability ...

replay, backfill, recoverability and schema evolution Define and enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using:Apache Kafka Apache NiFi Apache Spark/PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment … interested in engineers who can demonstrate strong expertise across several of the following areas: Core Technical Skills Python development in enterprise environments Apache Spark/PySpark Apache Kafka Apache NiFi Hadoop and Hive Advanced SQL REST API integration and development Data modelling and data lineage Distributed systems ...

platform at scale, ideally in a consumer or marketplace environment. Deep understanding of distributed systems and modern data ecosystems — including extensive experience with Databricks, Apache Spark, Apache Kafka, Apache Flink, Apache Beam and DBT. Demonstrated success in designing, building, and operating data platforms at scale, encompassing ...

experience as a Data Engineer or Senior Data Engineer Expert-level Python development skills Extensive hands-on experience with Databricks in enterprise environments Strong Apache Spark and PySpark expertise Advanced SQL development and optimisation skills Experience designing and building large-scale ETL and data integration solutions Strong understanding ...

Dataflow, Dataproc, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL. Proficiency in Python and SQL, with experience in Spark or Apache Beam. Experience designing data warehouse and data lake architectures, including data modeling. Working experience with Terraform and CI/CD tooling such as Jenkins ...

solving and communication skills. Able to work independently and remotely, and to travel to Newcastle once a week. Desirable skills and experience Experience with Apache Spark, Databricks or Kafka. Familiarity with orchestration tools such as Airflow. Experience with CI/CD for data pipelines and Infrastructure as Code. Knowledge ...

pipelines, deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. ...

pipelines, deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. ...

will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations.We are looking ...

engineering Site Reliability Engineering principles Data & Analytics Experience with technologies such as: Databricks Microsoft Fabric Azure Data Factory Synapse Analytics Snowflake BigQuery Redshift Kafka Apache Spark Knowledge of: Data modelling Data governance Data integration Data migration Data warehousing Master Data Management Applications & Integration Experience designing solutions using ...

production pipelines Strong SQL and Python, plus hands-on Git Production experience with at least one of Databricks, Snowflake, Azure Data Factory, AWS Glue, Apache Airflow or PySpark Cloud-native orchestration and distributed data processing Infrastructure as code (e.g. Terraform) CI/CD and DevOps or DevSecOps practice ...

possess hands‐on experience with one or more of the following technological areas (or their equivalent cloud‐native alternatives): Analytics & Platform Frameworks like Apache Spark or similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON ...

possess hands‐on experience with one or more of the following technological areas (or their equivalent cloud‐native alternatives): Analytics & Platform Frameworks like Apache Spark or similar cloud-specific services. Transformation & Orchestration tools Database Technologies Libraries & Schema Management Storage Formats: Familiarity managing diverse data types through formats like JSON ...

SQL. Well-versed in software engineering and systems design principles. Hands-on experience with Azure cloud data platforms is essential. Familiarity with Spark and Apache Airflow is required. Proficient in Linux/Unix/*nix environments. Strong knowledge of ETL tools and data modelling techniques. Candidates will need ...

schemas for both structured and unstructured data with a snowflake environment Enhance cloud data capabilities, working with AWS S3, Azure Data Lake, and Apache Spark Collaborate with cross-functional teams to ensure data availability for advanced analytics and reporting Automate workflows, implement CI/CD pipelines, and maintain data ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data ...

gateways for AI applications Infrastructure as Code experience with Terraform, Ansible, or other IaC tools Experience building and managing data pipelines using tools like Apache Spark AI safety and governance experience with model evaluation, LLM judges, bias detection, and responsible AI practices Familiarity with statistical fundamentals and running ...

Hands‐on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases. Expert‐level proficiency in Python, SQL, and Apache Spark for data pipeline development. Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks. Solid understanding of data modeling, dimensional design ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with ...

similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

engineering or business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source systems. Knowledge ...

agent frameworks. Inference and edge: TensorRT, ONNX Runtime, vLLM, Nvidia Triton Inference Server, KServe, Nvidia Jetson (e.g. Jetson Thor). Data and platforms: Databricks, Apache Spark, MLflow, SQL, AWS, GCP, Docker. Working at Rowden We are committed to building a flexible, inclusive, and enabling company. ...