26 to 50 of 83 Apache Spark Jobs in London

10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. ·Good proficiency in Python and Spark (PySpark) or Scala. ·Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. ·Extensive experience with Azure data services, including Azure ...

plus. Preferred Qualifications Certifications in Pentaho, Big Data, or Cloud Platforms (AWS/GCP/Azure). Experience with Big Data technologies (Hadoop, Spark) and cloud data services. ...

Lake Azure Synapse Azure SQL and Azure Storage Azure Data Factory ADF Expertise in building and managing complex data pipelines Databricks Handson experience with Spark based data processing notebooks and ML workflows Data Modeling Proficiency in conceptual logical and physical data modeling SQL Python Advanced skills for data manipulation ...

Optimise data workflows for performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence to data governance ...

Optimise data workflows for performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence to data governance ...

Optimise data workflows for performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence to data governance ...

data quality. Strong proficiency in one the programming languages, such as Python, Java, Scala, C++, etc. Experience with distributed computing frameworks such as Ray, Spark, and MapReduce. Attention to detail and a strong commitment to data integrity. Excellent problem-solving skills and ability to work in a fast-paced ...

data quality. Strong proficiency in one the programming languages, such as Python, Java, Scala, C++, etc. Experience with distributed computing frameworks such as Ray, Spark, and MapReduce. Attention to detail and a strong commitment to data integrity. Excellent problem-solving skills and ability to work in a fast-paced ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout … data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages such as SQL, Python, R, YAML ...

performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules within data pipelines. Data Transformation & Processing: Implement complex data transformations using Spark (PySpark or Scala) and other relevant technologies. Develop and maintain data processing logic for cleaning, enriching, and aggregating data. Ensure data consistency and accuracy throughout … data lifecycle. Azure Databricks Implementation: Work extensively with Azure Databricks Unity Catalog, including Delta Lake, Spark SQL, and other relevant services. Implement best practices for Databricks development and deployment. Optimise Databricks workloads for performance and cost. Need to program using the languages such as SQL, Python, R, YAML ...

Nginx, Redis, Event driven programming, Message queues, RDBS, NoSQL Database, Data Streams, real time distributed computing. 15. Experience in DevOps and release pipelines 16. Apache Camel 17. Postgresql Good to Have: 1. OBIEE Knowledge 2. TOGAF Certified 3. Splunk 4. Hadoop/EMR/Apache Spark Experience ...

Nginx, Redis, Event driven programming, Message queues, RDBS, NoSQL Database, Data Streams, real time distributed computing. 15. Experience in DevOps and release pipelines 16. Apache Camel 17. Postgresql Good to Have: 1. OBIEE Knowledge 2. TOGAF Certified 3. Splunk 4. Hadoop/EMR/Apache Spark Experience ...

MLOps: experiment tracking (eg, MLflow), model registry, CI/CD for ML. Serving and orchestration: AWS SageMaker, EKS, batch & Real Time inference. Data pipelines: Spark/SQL, orchestration (Airflow), monitoring for drift/perf. Nice to Have Cost/performance optimisation on AWS; GPU scheduling; quantisation/distillation. Experience ...

MLOps: experiment tracking (e.g., MLflow), model registry, CI/CD for ML. Serving and orchestration: AWS SageMaker, EKS, batch & real-time inference. Data pipelines: Spark/SQL, orchestration (Airflow), monitoring for drift/perf. Nice to Have Cost/performance optimisation on AWS; GPU scheduling; quantisation/distillation. Experience ...

Messaging Technologies with a good understanding of Kubernetes, Dockers and preferably some exposure to GCP (Google Cloud Platform), ReactJS, Python, Big Data Technologies like Spark, Beam Excellent personal organisation, completer/finisher, consensus builder with excellent communication skills and able to lead global development teams both directly ...

private/on-prem AI platforms (e.g. NVIDIA-based stacks, private AI platforms). Solid understanding of data engineering concepts (ETL, data lakes, Spark or similar). Experience designing scalable, distributed systems. Familiarity with model lifecycle management and MLOps practices. Proven ability to align AI initiatives to business outcomes ...

private/on-prem AI platforms (e.g. NVIDIA-based stacks, private AI platforms). • Solid understanding of data engineering concepts (ETL, data lakes, Spark or similar). • Experience designing scalable, distributed systems. • Familiarity with model lifecycle management and MLOps practices. • Proven ability to align AI initiatives to business outcomes ...

private/on-prem AI platforms (e.g. NVIDIA-based stacks, private AI platforms). • Solid understanding of data engineering concepts (ETL, data lakes, Spark or similar). • Experience designing scalable, distributed systems. • Familiarity with model lifecycle management and MLOps practices. • Proven ability to align AI initiatives to business outcomes ...

private/on-prem AI platforms (e.g. NVIDIA-based stacks, private AI platforms). • Solid understanding of data engineering concepts (ETL, data lakes, Spark or similar). • Experience designing scalable, distributed systems. • Familiarity with model lifecycle management and MLOps practices. • Proven ability to align AI initiatives to business outcomes ...

setting. MSc in Computer Science, Machine Learning, or a related field. Experience building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). Strong foundational knowledge of machine learning and deep learning algorithms, including deep neural networks, supervised/unsupervised learning ...

setting. MSc in Computer Science, Machine Learning, or a related field. Experience building data pipelines (realtime or batch) & data quality using modern toolchain (e.g., Apache Spark, Kafka, Airflow, dbt). Strong foundational knowledge of machine learning and deep learning algorithms, including deep neural networks, supervised/unsupervised learning ...

data extraction, transformation, and pipeline development. Experience with dashboarding and visualization tools (Tableau, Qlik, or similar). Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes. Useful experience; Python or R for advanced analytics, automation, or experimentation support. Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development. Experience with dashboarding and visualization tools (Tableau, Qlik, or similar). Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes. Useful experience; Python or R for advanced analytics, automation, or experimentation support. Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development. Experience with dashboarding and visualization tools (Tableau, Qlik, or similar). Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes. Useful experience; Python or R for advanced analytics, automation, or experimentation support. Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development. Experience with dashboarding and visualization tools (Tableau, Qlik, or similar). Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes. Useful experience; Python or R for advanced analytics, automation, or experimentation support. Knowledge of statistical methods and experimentation (A/ ...