26 to 50 of 335 Apache Jobs

Learning (ML):• Deep understanding of machine learning principles, algorithms, and techniques.• Experience with popular ML frameworks and libraries like TensorFlow, PyTorch, scikit-learn, or Apache Spark.• Proficiency in data preprocessing, feature engineering, and model evaluation.• Knowledge of ML model deployment and serving strategies, including containerization and microservices.• Familiarity with ...

platformsAdvanced Python and SQL skills, with experience in modern engineering practices such as CI/CD and TDDExperience with big data frameworks such as Apache Spark or similar technologiesSolid understanding of data modelling, data warehousing, OLTP and data lake architecturesExperience working with cloud ecosystems such as AWS, Azure ...

pipelines.QualificationsHands-on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases.Expert-level proficiency in Python, SQL, and Apache Spark for data pipeline development.Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks.Solid understanding of data modeling, dimensional design, and schema normalization ...

Terraform and Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery.Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines.Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

expertise in data pipeline development, ETL/ELT processes, and data warehousing solutions.Proficient with cloud platforms (AWS, GCP, Azure) and big data tools like Apache Spark, Python, etc.Experience with relational and NoSQL databases.Solid understanding of data modeling, data architecture, engineering frameworksProven ability to lead and scale engineering teams ...

Redshift.Experience with data transformation tools such as dbt, or Dataflow.Understanding of data modeling techniques.Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset.Understanding of software engineering best practices and their application to data processing systems.Knowledge of Python, Java or Flink is a plus, but not a necessity.Strong ...

self‐motivated, inquisitive mindset and a methodical, agile approach to problem‐solving Solid Linux administration and networking skills, plus Git‐based workflows Familiarity with Apache Airflow, Dask and JupyterHub Experience with modern build systems (especially Bazel) and CI/CD pipelines. We love hearing from anyone inspired to build ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Experience with Infrastructure-as-Code tools such as Terraform Knowledge of DevOps, CI/CD, platform automation, and cloud‐native operational practices Familiarity with Apache Spark, dbt, and modern data integration technologies Delivery Experience Experience leading architecture for enterprise-wide Databricks implementations and cloud data modernization initiatives Proven success ...

Auto ML, Tensorflow, Natural Language, Vision AIPA’s common technology components:Google CloudCloud RunCloud FunctionsKubernetes Engine, Compute Engine, VPCsCloud Spanner, Cloud SQL and BigQueryDataflow, Apache BeamVertex AI, DialogFlow, Auto MLArchitectureMicroservices patternsEvent-driven architectures and message queuesRelational and non-relational databasesLanguages and frameworksJava 17+, JavaScript, Python, SQLCommon frameworks and APIs ...

market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

datasets— hands-on familiarity with vessel-tracking data, its structure, quirks, and limitations.• SQL— strong proficiency, including query optimization on large datasets.Nice-to-Haves• Apache Airflow (or equivalent orchestration) for scheduling and managing pipelines.• AWS experience (Azure or GCP equally welcome).• Machine Learning experience, particularly applied to geospatial ...

Python skills * Hands-on experience designing and developing ETL pipelines and data workflows * Experience designing and maintaining data pipelines using technologies such as Apache Airflow or Spark * Experience working with cloud platforms including AWS, Azure or GCP * Strong understanding of data modelling, integration and governance principles * Experience working with ...

strategy. What We're Looking For Essential 5+ years' experience in Data Engineering or a similar role. Strong experience with Python, SQL, Apache Spark, and Cloud platforms (AWS, Azure or GCP) Experience designing and operating production-scale data platforms. Desirable Experience working with AWS. Experience supporting machine learning ...

strategy. What We're Looking For Essential 5+ years' experience in Data Engineering or a similar role. Strong experience with Python, SQL, Apache Spark, and Cloud platforms (AWS, Azure or GCP) Experience designing and operating production-scale data platforms. Desirable Experience working with AWS. Experience supporting machine learning ...

range of projects that include internal and external suppliers. Essential Skills Proven experience as a Data Engineer Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience with Microsoft Fabric Experience of using programming languages such as Python, Scala and SQL. Demonstrable knowledge ...

ability to help refine problem statements and propose solutions taking effort-impact-scalability tradeoff into account.Some skills that will make you stand out: Apache Spark, Iceberg, Kafka, dbtScikit-Learn, XGBoost, PyTorch, MLFlow,, GraphFrames, RayAWS (S3, EMR, SageMaker, Lakeformation), Terraform, Docker, GitHub CI/CDKnowledge Graphs (+ RAG), graph ...

query optimisation. Redis - caching, pub/sub, distributed locks and idempotency patterns. Serverless Framework v3, Docker (container-based Lambdas), Terraform, Jenkins CI/CD. Apache Airflow DAG authoring and AWS Glue with PySpark for batch ETL. pytest and moto (offline AWS service mocking), Black formatter, ruff linter. Datadog ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with AI evaluation and observability practices ...

scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure deployment and scaling decisionsCI/CD fluency ...

SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI)Familiarity with emerging technologies in data engineeringAbility ...

Redshift).Semantic Layer: Design and implementation for governed reusable data views.Data Governance Tools: Collibra, Alation, Talend or equivalent.Expertise in ETL tools and processes (e.g., Apache NiFi, Talend, Informatica)Security: Data Privacy, Data Protection & Encryption frameworks & governanceEnterprise Architecture Frameworks: TOGAF, Zachman; familiarity with ITIL processes.Additional Helpful Skills:Cloud Platform Awareness ...

/CD pipelines and deployment automation. • Familiarity with monitoring, logging, and observability tools. • Awareness of application platforms and middleware such as Java, Tomcat, Apache, and NodeJS. • Experience using Git and collaborative development workflows. • Basic scripting skills using Shell or Python. • Understanding of containerisation concepts using Docker. • Strong problem-solving ...

warehouse solutions.Proficiency in at least one programming language (e.g., Python) for data processing and pipeline automation.Familiarity with ETL tools and workflow orchestration frameworks (e.g., Apache Airflow or similar).Experience implementing data quality checks and working with large-scale datasets.Good problem-solving abilities, plus strong communication and teamwork skills ...