1 to 25 of 180 Apache Jobs

design, scalability, and clean coding practices. Familiarity with application monitoring solutions Data Engineering 5+ years building scalable and reliable data pipelines using Databricks and Apache Spark. We need both. Experience with cloud data warehouse/data lake architectures and tooling. Experience building solutions using cloud providers, preferably Azure ...

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop, maintain ...

data engineering roles Strong experience working with Snowflake and Oracle Solid engineering background in Java and/or Python Hands-on experience with Apache Spark (Java or Scala) Experience with AWS , including EMR and other core services Exposure to Docker and Kubernetes (EKS) Proven experience building scalable ETL pipelines ...

however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

Solid understanding of data warehousing, lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Excellent Python and SQL skills Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI ...

have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI ...

coding skills in Java (Spring Boot) for data engineering and services. Proven background working within financial services environments. Hands-on experience building and managing Apache Airflow/Spark and Hadoop clusters. Deep experience in big data engineering, including Hadoop, Kafka, and Spark-based data pipelines. Strong SQL/RDBMS ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

understanding of data governance and regulatory compliance frameworks such asPCI-DSS, GDPR, SOX, and CPRA. Bonus: Experience with Lakehouse technologies such as Delta Lake, Apache Iceberg, or Hudi. Daily Duties: Design, develop, and maintain scalable data pipelines that support enterprise reporting, regulatory compliance, and analytics. Implement data validation frameworks ...

Charity Volunteer Days Cycle to work scheme And More.. DATA ENGINEER – ESSTENTIAL SKILLS Proven experience building data pipelines using Databricks . Strong understanding of Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

solutions. Required Skills: 1) Strong hands-on Python programming 2) 3+ years of experience in ETL development 3) Experience with data platforms like Informatica, Apache Airflow, or similar 4) Strong SQL skills complex joins, aggregations, indexing, and tuning 5) Prior experience in Banking or Financial Services (Retail Banking, Payments ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...