1 to 25 of 284 Apache Jobs

design, scalability, and clean coding practices. Familiarity with application monitoring solutions Data Engineering 5+ years building scalable and reliable data pipelines using Databricks and Apache Spark. We need both. Experience with cloud data warehouse/data lake architectures and tooling. Experience building solutions using cloud providers, preferably Azure ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

Azure, or another cloud services 1+ years of experience using AWS Glue or Lambda 1+ years of experience SQL querying in Snowflake or Apache Spark 3+ years of experience in open source frameworks 2+ years of experience in Agile practices At this time, Capital One will not sponsor ...

open-source technologies. Desired working in offshore model and Managed outcome Develop logical and physical data models for big data platforms. Automate workflows using Apache Airflow. Create data pipelines using Apache Hive, Apache Spark, Apache Kafka. Provide ongoing maintenance and enhancements to existing systems and participate ...

environments. Kubernetes — Experience deploying and managing distributed applications using container orchestration. Modern data-lake technologies — Familiarity with columnar formats (e.g. Parquet), table formats (e.g. Apache Iceberg, Hudi, Delta Lake), or data-lake management tools (e.g. AWS Glue, Google BigLake). Not sure you meet every requirement? Studies show that ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR ...

software development experience with Python and experience with modern software development and release engineering practices (e.g. TDD, CI/CD). Experience with Apache Spark or any other distributed data programming frameworks. Comfortable writing efficient SQL and debugging on cloud warehouses like Databricks SQL or Snowflake. Experience with cloud ...

JSON files, complex joins, complex calculations using CTEs etc) and performance optimisation of queries. Experience with or knowledge of data pipeline orchestration tools (e.g., Apache Airflow, Dagster, Prefect) as well as monitoring and observability tools such as Grafana is desirable. Ideally hands-on experience using infrastructure-as-code tools ...

JSON files, complex joins, complex calculations using CTEs etc) and performance optimisation of queries. Experience with or knowledge of data pipeline orchestration tools (e.g., Apache Airflow, Dagster, Prefect) as well as monitoring and observability tools such as Grafana is desirable. Ideally hands-on experience using infrastructure-as-code tools ...

native solutions that support analytics, automation, and digital transformation. Data Engineering, Full Stack & Platform Development Design and implement scalable data pipelines using tools like Apache Spark, Airflow, or dbt.Build and maintain data lakes, warehouses, and real-time streaming solutions. Develop APIs and microservices to expose data securely and efficiently. ...

have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI ...

have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology in the stack includes: FastAPI ...

/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA, MES, and Historian systems. Information Modeling ...

proficiency in Python. You should be able to demonstrate: Hands-on experience with data pipelines (ETL/ELT) and workflow orchestration tools such as Apache Airflow, Solid knowledge of SQL/NoSQL databases, data modeling, and schema design, Familiarity with streaming technologies (e.g., Kafka), containerization (Docker, Kubernetes ...

reliabilityData Engineering & Pipelines Build and optimize data pipelines in AWS using Glue, Lambda, Step Functions, EMR, Athena, and S3. Implement DAG-based orchestration using Apache Airflow, AWS Managed Workflows (MWAA), or Glue Workflows. Ensure data quality, reliability, lineage, and observability across all pipelines.Machine Learning Pipeline Enablement Collaborate with ...

learning solutions. With published authors and award-winning data scientists who contribute some of the major machine learning and distributed data technologies, such as Apache Spark, we are a friendly, passionate group of engineers making a career out of building great software for our customers. The Adthena data team ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. Proficiency ...