1 to 25 of 281 Apache Jobs

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop, maintain ...

design, scalability, and clean coding practices. Familiarity with application monitoring solutions Data Engineering 5+ years building scalable and reliable data pipelines using Databricks and Apache Spark. We need both. Experience with cloud data warehouse/data lake architectures and tooling. Experience building solutions using cloud providers, preferably Azure ...

data engineering roles Strong experience working with Snowflake and Oracle Solid engineering background in Java and/or Python Hands-on experience with Apache Spark (Java or Scala) Experience with AWS , including EMR and other core services Exposure to Docker and Kubernetes (EKS) Proven experience building scalable ETL pipelines ...

however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Solid understanding of data warehousing, lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Experience working with Apache Spark and large-scale data processing For a full consultation on this exciting new role, please get in touch with ARC IT Recruitment today. ...

coding skills in Java (Spring Boot) for data engineering and services. Proven background working within financial services environments. Hands-on experience building and managing Apache Airflow/Spark and Hadoop clusters. Deep experience in big data engineering, including Hadoop, Kafka, and Spark-based data pipelines. Strong SQL/RDBMS ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

platform, features, and best practices with Strong understanding of data modeling principles and techniques A solid understanding of big data technologies such as Apache Spark, and knowledge of Hadoop ecosystems. Knowledge of programming languages such as Python, R, or Java is beneficial. Knowledge of ETL/ELT processes (such ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

platform, features, and best practices with Strong understanding of data modeling principles and techniques A solid understanding of big data technologies such as Apache Spark, and knowledge of Hadoop ecosystems. Knowledge of programming languages such as Python, R, or Java is beneficial. Knowledge of ETL/ELT processes (such ...

delivering large-scale data integration, engineering, and analytics projects Strong expertise in cloud platforms (e.g. Azure, MS Fabric, AWS) and modern data tools (e.g. Apache Spark, Hadoop) Proficiency in programming languages (e.g. Python, SQL, Java) Extensive experience in stakeholder management and collaboration across health and local government to drive ...

solutions. Required Skills: 1) Strong hands-on Python programming 2) 3+ years of experience in ETL development 3) Experience with data platforms like Informatica, Apache Airflow, or similar 4) Strong SQL skills complex joins, aggregations, indexing, and tuning 5) Prior experience in Banking or Financial Services (Retail Banking, Payments ...

Knowledge of machine learning frameworks (e.g., PySpark, PyTorch) and model deployment tools (e.g., MLflow, TensorFlow Serving). Strong experience with data processing frameworks (e.g., Apache Spark, Flink). Expertise in SQL and NoSQL databases (e.g., MySQL, PostgreSQL, MongoDB, Cassandra). Hands-on experience with cloud platforms (e.g. ...

governance, and compliance in AWS Required Skills & Experience Strong experience in Python for data engineering and pipeline development Hands-on experience with PySpark/Apache Spark Advanced proficiency in SQL (query optimization, window functions, CTEs) Experience with AWS data warehousing tools, including: o Amazon Redshift & RDS o AWS Glue ...

Charity Volunteer Days Cycle to work scheme And More.. DATA ENGINEER – ESSTENTIAL SKILLS Proven experience building data pipelines using Databricks . Strong understanding of Apache Spark (PySpark or Scala) and Structured Streaming . Experience working with Kafka (MSK) and handling real-time data . Good knowledge of Delta Lake ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...

Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as Databricks ...