26 to 50 of 332 Permanent Apache Jobs

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands ...

within either Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.Data acquisition and development of data sets and improving data qualityPreparing data for predictive and prescriptive modellingHands on coding experience, such ...

/CD, DevOps, and Infrastructure as Code practices for data platforms. Metadata management, lineage tooling, and data observability/monitoring experience. Familiarity with dbt, Apache Airflow, Azure Data Factory, Kafka, Event Hubs, or comparable orchestration/integration tools. Familiarity with enterprise AI platforms — Azure AI, AWS SageMaker, Google Vertex ...

with the Lead Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security ...

Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Reporting tools ...

large-scale data processing.Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning.Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar).Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.).Strong understanding of data quality, security, and lineage best ...

with Snowflake and Databricks native solutions. Experience on Snowflake Cortex is must. Proficiency in programming skills in Python. Experience with data processing frameworks like Apache Spark or Hadoop. Knowledge of database systems (SQL and NoSQL). Familiarity with cloud platforms (AWS, Azure) and their data services. Understanding of data ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar).Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.).Strong understanding of data quality, security, and lineage best ...

Data Engineering experience with Snowflake and Databricks native solutions.Experience on Snowflake Cortex is must.Proficiency in programming skills in Python.Experience with data processing frameworks like Apache Spark or Hadoop.Knowledge of database systems (SQL and NoSQL).Familiarity with cloud platforms (AWS, Azure) and their data services.Understanding of data modeling and data ...

design data architectures that facilitate model training and inference.Data Governance Tools: Familiarity with data governance or data cataloguing tools (such as Collibra, Alation, or Apache Atlas) and lineage-tracking systems. Hands-on experience setting up or maintaining a data catalogue, documenting data definitions, or automating data lineage capture ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern data ...

technology partners. Knowledge of enterprise data platforms, data engineering, data governance, and analytics ecosystems. Experience supporting large-scale transformation programmes. Understanding of Power BI, Apache Superset, Grafana, and enterprise reporting solutions. Strong knowledge of CI/CD pipelines and automation technologies. Experience with Terraform, Ansible, PowerShell, Python, or similar ...

/ML Familiarity with Microsoft Fabric (Data Engineering, Data Science, Real-Time Analytics) and Azure ML, including MLflow Proficiency with tools such as Apache Spark, Azure Data Factory, GIT, Python and SQL Skilled in developing, training and deploying machine learning models with frameworks like TensorFlow, PyTorch or Scikit-learn ...

platformsAdvanced Python and SQL skills, with experience in modern engineering practices such as CI/CD and TDDExperience with big data frameworks such as Apache Spark or similar technologiesSolid understanding of data modelling, data warehousing, OLTP and data lake architecturesExperience working with cloud ecosystems such as AWS, Azure ...

Terraform and Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery.Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines.Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

Ansible for repeatable, automated provisioning of GPU clusters and AI platform environments; GitOps (ArgoCD) for continuous, declarative platform delivery. Pipeline orchestration: Kubeflow Pipelines, Apache Airflow, or Argo Workflows to orchestrate multi-stage training, fine-tuning, and inference pipelines. Cluster & workload scheduling: Slurm, Run:ai, and NVIDIA Base Command Manager ...

expertise in data pipeline development, ETL/ELT processes, and data warehousing solutions.Proficient with cloud platforms (AWS, GCP, Azure) and big data tools like Apache Spark, Python, etc.Experience with relational and NoSQL databases.Solid understanding of data modeling, data architecture, engineering frameworksProven ability to lead and scale engineering teams ...

Redshift.Experience with data transformation tools such as dbt, or Dataflow.Understanding of data modeling techniques.Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset.Understanding of software engineering best practices and their application to data processing systems.Knowledge of Python, Java or Flink is a plus, but not a necessity.Strong ...

with data transformation tools such as dbt, or Dataflow. Understanding of data modeling techniques. Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset. Understanding of software engineering best practices and their application to data processing systems. Knowledge of Python, Java or Flink is a plus ...

with data transformation tools such as dbt, or Dataflow. Understanding of data modeling techniques. Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset. Understanding of software engineering best practices and their application to data processing systems. Knowledge of Python, Java or Flink is a plus ...

using Databricks REST APIs, Prometheus, or Azure Monitor.• Maintain best practices for data engineering performance, testing, and documentation.Technical Skills Required• Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake.• Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies.• Experience tuning job configurations (e.g. ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Auto ML, Tensorflow, Natural Language, Vision AIPA’s common technology components:Google CloudCloud RunCloud FunctionsKubernetes Engine, Compute Engine, VPCsCloud Spanner, Cloud SQL and BigQueryDataflow, Apache BeamVertex AI, DialogFlow, Auto MLArchitectureMicroservices patternsEvent-driven architectures and message queuesRelational and non-relational databasesLanguages and frameworksJava 17+, JavaScript, Python, SQLCommon frameworks and APIs ...