1 to 25 of 122 Apache Jobs

design, scalability, and clean coding practices. Familiarity with application monitoring solutions Data Engineering 5+ years building scalable and reliable data pipelines using Databricks and Apache Spark. We need both. Experience with cloud data warehouse/data lake architectures and tooling. Experience building solutions using cloud providers, preferably Azure ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience in cloud-based data services. Proficiency in SQL and Python for data manipulation and transformation. Experience with modern data engineering tools, including Apache Spark, Kafka, and Airflow. Strong understanding of data modelling, schema design, and data warehousing concepts. Familiarity with data governance, privacy, and compliance frameworks (e.g., GDPR ...

using both ETL and ELT methodologies to support scalable data architecture in high-demand environments. Excellent understanding of distributed data processing, ideally working with Apache Spark Extensive experience (5+ years) with advanced SQL techniques and database management, including proficiency with database systems like PostgreSQL, MySQL, and Amazon Redshift, optimizing ...

/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema originating from OSIsoft PI, SCADA, MES, and Historian systems. Information Modeling ...

along with fluency in English and proficiency in Python. Hands-on experience with data pipelines (ETL/ELT) and workflow orchestration tools such as Apache Airflow, Solid knowledge of SQL/NoSQL databases, data modeling, and schema design, Familiarity with streaming technologies (e.g., Kafka), containerization (Docker, Kubernetes ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

Lead, Data Engineering Manager etc. Proven success with modern data infrastructure: distributed systems, batch and streaming pipelines Hands-on knowledge of tools such as Apache Spark, Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes ...

proficiency in Python. You should be able to demonstrate: Hands-on experience with data pipelines (ETL/ELT) and workflow orchestration tools such as Apache Airflow, Solid knowledge of SQL/NoSQL databases, data modeling, and schema design, Familiarity with streaming technologies (e.g., Kafka), containerization (Docker, Kubernetes ...

proficiency in Python. You should be able to demonstrate: Hands-on experience with data pipelines (ETL/ELT) and workflow orchestration tools such as Apache Airflow, Solid knowledge of SQL/NoSQL databases, data modeling, and schema design, Familiarity with streaming technologies (e.g., Kafka), containerization (Docker, Kubernetes ...

proven track record of developing high-performance applications in production settings. Insightful experience with big data technologies like Hadoop, real-time processing frameworks (e.g., Apache Spark), and advanced knowledge of SQL and data architecture. Thinks in systems: their edge cases, failure modes, and life cycles Uses a metrics driven ...

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client projects … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. Proficiency ...

strong plus Technical Skills: • Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL) • Familiarity with data pipeline and workflow management tools (e.g., Apache Airflow) • Experience with programming languages such as Python, Java, or Scala. Python is highly preferred • Basic understanding of cloud platforms and services (e.g. ...

operating large scale data processing systems. Has successfully led data platform initiatives. A good understanding of data processing technologies and tools such as Apache Spark, Data Lake, Data Warehousing and SQL Databases. Proficiency in programming languages such as Python and CICD techniques to efficiently deliver change in a consistent ...

code Proficiency in programming languages such as Python, Spark, SQL Strong experience with SQL databases Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF) Experience with cloud platforms (Azure preferred) and related data services Excellent problem-solving skills and attention to detail Inclusive and curious, continuously ...

Oracle databases. Extensive experience coding in Python. Understanding of ETL processes and data pipeline design. Familiar with data lakes and their architecture. Experience with Apache Airflow (MWAA) for workflow management. Experience with AWS Athena/PySpark (Glue) for data querying and processing. Proficient in SQL with experience in both ...

working within Agile/SCRUM teams Experience mentoring junior developers Knowledge of Oracle/relational databases, MongoDB, and GitLab CI/CD Familiarity with Apache NiFi, JavaScript/TypeScript, and React Experience with Elasticsearch, Kibana, Hibernate, and the Atlassian suite (Bitbucket, Jira, Confluence) Desirable; Experience with JSF (PrimeFaces) Knowledge ...

compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse and ETL/ ...

management principles, including modelling, integration, and governance. Familiarity with modern data architectures (cloud-based, distributed systems). Proficiency in SQL, Python, and pipeline tools (Apache Airflow, Spark). Experience with cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness and evidence of data analytics and visualisation. Essentials Sole ...

distributed computing. Experience with CI/CD workflows, version control (Git), and testing frameworks (pytest, ScalaTest). Knowledge of open table formats (Delta, Iceberg, Apache Hudi). Familiarity with Infrastructure as Code tools (Terraform or Bicep). Proven implementation experience in data governance principles, including data catalogs, lineage, security ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

environments Owning the cloud infrastructure underpinning data systems through a DevOps approach Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of the possible architectures involved in modern data system design (e.g. Data Warehouse, Data Lakes and Data Meshes ...

detection, credit risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...