1 to 25 of 360 Apache Jobs

Software Engineer – Query Engines

Location
Greater London, England, United Kingdom
computations are represented, how query plans are optimized, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers operate … want to advance the capabilities of modern data systems and apply that work to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability ...

Software Engineer, New Grad - Query Engines

Location
Greater London, England, United Kingdom
computations are represented, how query plans are optimised, how operators execute, and how data is read and written efficiently. We build on and extend Apache Spark, Apache DataFusion, and Apache Iceberg, bringing advances in open-source engines and table formats into the demanding environments our customers operate … learn how modern data systems work from the inside and apply that knowledge to consequential problems. Technologies We Use Java, Scala, Rust, and Python Apache Spark, Apache DataFusion, Apache Comet, and Velox for data processing and query execution Apache Iceberg for table management and catalog interoperability ...

Contract Software Data Engineer - Chester - Inside IR35 - Investment Banking

Location
Chester, England, United Kingdom
replay, backfill, recoverability and schema evolution Define and enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using:Apache Kafka Apache NiFi Apache Spark/PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment … interested in engineers who can demonstrate strong expertise across several of the following areas: Core Technical Skills Python development in enterprise environments Apache Spark/PySpark Apache Kafka Apache NiFi Hadoop and Hive Advanced SQL REST API integration and development Data modelling and data lineage Distributed systems ...

Technical Application Consultant (Oracle Database, Apache Tomcat, httpd)

Location
United Kingdom
Technical Application Consultant (Oracle Database, Apache Tomcat, httpd) Full-time Come join us and make a difference in the world! Please note, role can be remote or hybrid, subject to location.This role requires weekend overtime approximately once per month to facilitate production system upgrades. We are looking … customers. Principal duties and responsibilities Apply our software updates to customer non-production and production systems. Perform related software technology upgrades including Oracle Database, Apache Tomcat, Apache Web Server, and Jasper Reports Server. Perform installations of new software developed by our company. Troubleshoot issues relating to software installation ...

Senior Data Engineer

Location
Greater London, England, United Kingdom
experience as a Data Engineer or Senior Data Engineer Expert-level Python development skills Extensive hands-on experience with Databricks in enterprise environments Strong Apache Spark and PySpark expertise Advanced SQL development and optimisation skills Experience designing and building large-scale ETL and data integration solutions Strong understanding ...

Senior Data Platform Engineer (Hybrid)

Location
Greater London, England, United Kingdom
Dataflow, Dataproc, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL. Proficiency in Python and SQL, with experience in Spark or Apache Beam. Experience designing data warehouse and data lake architectures, including data modeling. Working experience with Terraform and CI/CD tooling such as Jenkins ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
Austin, Texas, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
Lowell, Massachusetts, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
Tucson, Arizona, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
Rochester, Minnesota, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
Seattle, Washington, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
San Francisco, California, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Confluent Staff Software Engineer - Kafka

Hiring Organisation
IBM
Location
Durham, North Carolina, United States
Employment Type
Permanent
Salary
USD Annual
guide stakeholders at all levels. Preferred Skills What Gives You an Edge: - Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink, Apache Fluss - Experience with query engines or distributed SQL systems (e.g. Presto/Trino, Spark ...

Data Engineer (SC Eligible/ SC Cleared)

Location
Newcastle upon Tyne, England, United Kingdom
solving and communication skills. Able to work independently and remotely, and to travel to Newcastle once a week. Desirable skills and experience Experience with Apache Spark, Databricks or Kafka. Familiarity with orchestration tools such as Airflow. Experience with CI/CD for data pipelines and Infrastructure as Code. Knowledge ...

Senior Data Engineer London

Location
Greater London, England, United Kingdom
software engineering experience using Python and/or other modern programming languages. Experience developing and orchestrating data pipelines using technologies such as DBT, Apache Spark, Airflow, Dagster, Kafka, Flink, or equivalent tooling. Solid understanding of data lakehouse, warehouse, and medallion architecture patterns. Experience applying DevOps and DataOps best practices ...

Staff Data Engineer

Hiring Organisation
Visa
Location
Austin, Texas, United States
Employment Type
Permanent
Salary
USD Annual
with a PhD • Demonstrated leadership in delivering high-quality, large-scale, enterprise-class applications. • Solid experience in big data engineering, with knowledge of Hadoop, Apache Spark, Python, and SQL. • Experience in Docker, Kubernetes for containerization • Proficiency in creating and managing large-scale data pipelines and machine learning models. • Experience ...

Data Engineer

Location
Glasgow, Scotland, United Kingdom
vector databases Streaming technologies (Kafka, Kinesis, Flink, etc.) Containers and related services (Docker, Kubernetes, container Registries, etc) Data modelling approaches (Kimball, Imnon) Orchestration tools - Apache Airflow, Prefect or cloud-native tools Backend software development (Java, APIs, Scalability, Logging and Monitoring etc.) MLFlow and other MLOps/Machine Learning Engineering ...

Data Engineer - Security & Intelligence

Location
Cheltenham, England, United Kingdom
skills, with experience of Java or Scala beneficial Experience building large-scale ETL/ELT and data processing pipelines Hands‐on experience with Apache Spark and distributed data processing technologies Experience designing and operating cloud-native data platforms, particularly within AWS Familiarity with Docker, Kubernetes and CI/ ...

Software Engineer - Connectors

Location
Greater London, England, United Kingdom
analysis and returning the results to various business systems. We build on a vast variety of technological stacks, from building products on top of Apache Kafka, Apache Spark, and Apache Beam to integrating Neo4j into Google Dataflow Templates and Microsoft Fabric. We also build backend components … ideally have professional experience developing software systems, either in cloud or on-premises, involving either of the following; Distributed event streaming platforms, such as Apache Kafka or similar products, Data processing and analytics platforms, such as Apache Spark, Apache Beam, or their derivatives, Data warehouses ...

AWS Data Engineer – Power BI

Location
Lancashire, England, United Kingdom
Automated deployments across environments. Monitoring and logging. Release and configuration management. Production support and incident troubleshooting. Python or another scripting language. AWS certification. Terraform. Apache Airflow/MWAA. Databricks or Spark. Kafka/Kinesis. dbt. Experience working in Agile/Scrum environments. Experience 4+ years of experience in Data ...

Data Engineer

Location
Greater London, England, United Kingdom
pipelines, deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. ...

Senior Data Engineer

Location
City Of London, England, United Kingdom
pipelines, deployment automation and infrastructure-as-code practices. What We're Looking For Essential Strong commercial experience building production-grade data pipelines using Apache Spark (PySpark preferred). Experience delivering modern data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. ...

Data Operations Engineer

Location
Manchester, England, United Kingdom
established standards. Strong communication and collaboration skills. Desirable Experience Experience across some of the following would be beneficial: AWS Redshift, Athena, EMR or Kinesis. Apache Spark or PySpark for distributed data processing. AWS Step Functions or other workflow orchestration technologies. Terraform, CloudFormation or other Infrastructure as Code tooling. GitHub ...

Data Operations Engineer

Location
Greater London, England, United Kingdom
established standards. Strong communication and collaboration skills. Desirable Experience Experience across some of the following would be beneficial: AWS Redshift, Athena, EMR or Kinesis. Apache Spark or PySpark for distributed data processing. AWS Step Functions or other workflow orchestration technologies. Terraform, CloudFormation or other Infrastructure as Code tooling. GitHub ...

Senior Data Engineer - Security Cleared

Hiring Organisation
Hackajob Ltd
Location
South West London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...