51 to 75 of 399 Apache Jobs

pipelines.QualificationsHands-on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases.Expert-level proficiency in Python, SQL, and Apache Spark for data pipeline development.Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks.Solid understanding of data modeling, dimensional design, and schema normalization ...

comfort working through ambiguity. Desirable MLOps excellence: reproducible pipelines, model versioning, CI/CD, observability, and automated evaluation. Data engineering: proficiency with Databricks, Apache Spark, Delta Lake, MLflow, and SQL; experience integrating datasets and maintaining data quality. Model training and optimisation: experience with pre‐training, fine‐tuning, distributed training ...

Terraform and Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery.Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines.Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

Excellent Python/PySpark and SQL data engineering skills Strong experience designing and developing ETL/ELT and large-scale data pipelines Experience with Apache Spark and Databricks Lakehouse architectures Strong knowledge of Azure Data Lake Storage (ADLS) Experience integrating Databricks with wider Azure data and analytics services Understanding ...

experience in data engineering or a similar role. Proficiency in programming languages such as Python, Java, or Scala. Experience with data processing frameworks like Apache Spark or Hadoop. Strong knowledge of SQL and database systems. Familiarity with cloud platforms such as AWS, Azure, or Google Cloud. Excellent problem-solving ...

pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of scheduling/orchestration tools such as Autosys and/or Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift or equivalent ...

Oracle databases. Experience designing and supporting ETL/ELT pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of Autosys and Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift ...

similar data-focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands-on experience with Apache Spark, Databricks, dbt and cloud-based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with ...

Computer Science or equivalent practical experience Experience developing finance applications is a plus Working experience with Terraform is a plus Experience creating workflows for Apache Airflow is a plus #J-18808-Ljbffr ...

with data transformation tools such as dbt, or Dataflow. Understanding of data modeling techniques. Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset. Understanding of software engineering best practices and their application to data processing systems. Knowledge of Python, Java or Flink is a plus ...

similar data‐focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands‐on experience with Apache Spark, Databricks, dbt and cloud‐based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with ...

mesh concepts and their practical implementation within large organisations Experience integrating Databricks with AWS services including S3, IAM, VPC, and AWS Glue Familiarity with Apache Spark, dbt, data pipelines, and modern analytics architectures Understanding of access management, audit frameworks, and enterprise governance controls Delivery Experience Experience leading data product ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python ...

dimensional modelling, star/snowflake schemas, data vault, etc.). Experience designing and operating ETL/ELT pipelines and modern workflow orchestration tools (e.g., Apache Airflow, Azure Data Factory, Azure Functions). Professional experience in delivering cloud-based data engineering solutions at scale, with strong client-facing, stakeholder management ...

Computer Science or equivalent practical experience Experience developing finance applications is a plus Working experience with Terraform is a plus Experience creating workflows for Apache Airflow is a plus What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally work ...

knowledge of optimisation techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

scripts, with a particular focus on building and optimizing multi-threaded, multi-process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI-powered solutions using Large Language Models (LLMs), retrieval-augmented ...

skills; experience with Oracle databases . Experience designing and supporting ETL/ELT pipelines and data quality frameworks . Working knowledge of Autosys& Apache Airflow Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift) Familiarity with AWS S3 for large-scale data storage. Experience ...

skills Desirable Experience Experience with any of the following would be advantageous: Angular, React or Vue Chart.js, D3.js or other data visualisation tools Databricks, Apache Spark or distributed data processing Docker, Kubernetes or Cloud Run Terraform or other infrastructure-as-code technologies GPU or accelerated computing technologies such ...

skills Desirable Experience Experience with any of the following would be advantageous: * Angular, React or Vue * Chart.js, D3.js or other data visualisation tools * Databricks, Apache Spark or distributed data processing * Docker, Kubernetes or Cloud Run * Terraform or other infrastructure-as-code technologies * GPU or accelerated computing technologies such ...

with monitoring/logging tools (Prometheus, Grafana, ELK) and root cause analysis Knowledge of storage (NFS/EFS), databases (PostgreSQL), and web servers (Nginx, Apache, Tomcat) Experience with SSO and security (SAML, OIDC, Azure AD, Okta, Keycloak) Scripting skills in Python and/or Bash Profile Client-focused with ...

build tools (Maven/Gradle). Desirable skills: Scala: Hands-on experience or a strong functional programming background using Scala. Big Data: Exposure to Apache Spark, Kafka, or Hadoop ecosystems. Cloud: Experience with AWS, Azure, or Google Cloud Platform. Experience in banking or financial services industry. Soft Skills Strong ...

scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure deployment and scaling decisionsCI/CD fluency ...