51 to 75 of 138 Apache Spark Jobs in England

experience with the SQL Server platform and T-SQL querying. Familiarity with cloud data platforms (e.g., AWS, Azure) and big data technologies (e.g., Kafka, Spark, Databricks, Snowflake). Expert-level knowledge of building ETL processes in Python. Expert-level knowledge of performance tuning and optimization in a Python/ ...

technology sectors. You'll bring: Significant experience leading Data Engineering teams and large-scale programmes Strong expertise in Python and SQL Deep experience with Spark and/or Databricks Strong cloud expertise across AWS, Azure and/or GCP Experience with orchestration technologies such as Airflow A track record ...

experience productionising machine learning models Are an expert in at least one of: predictive modelling, classification, regression, optimisation or recommendation systems Have experience with Spark Have knowledge of DevOps technologies such as Docker and Terraform and ML Ops practices and platforms like ML Flow Have experience with agile delivery ...

through influence with strong collaborator engagement skills. Preferred Experience Digital Twin, asset virtualization, or industrial data platforms Graph technologies (e.g., Neo4j), big data ecosystems (Spark, Hive, Hadoop) Databricks, Python, and modern analytics platforms Delivery of data products at enterprise scale Key Competencies Strategic product thinking with strong execution bias ...

Senior Data Engineer (Scala/Spark) London | Hybrid Working | Initial 12 month contract | £450-550pd PAYE + holiday pay We are partnering with a world-leading technology organisation seeking an experienced Senior Data Engineer to join a high-performing engineering team responsible for building and operating large-scale … machine learning workflows at scale. As a Senior Data Engineer, you will design, build and operate large-scale batch data pipelines using Scala and Apache Spark. Working alongside Machine Learning Engineers, Researchers and Platform Engineers, you will help develop the infrastructure and tooling required to process vast datasets ...

Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar). Building data pipelines (real‐time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt. Familiarity with recommender systems, transformers, or graph neural networks. Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

promote DevOps. Support internal and external stakeholders, including customers and regulatory authorities. Tools, Systems & Continuous Improvement Manage and optimise the use of tools (e.g. Apache Spark, AWS Glue, Databricks or equivalent). Support system enhancements of data processing and transformation, migrations, and process improvements. Contribute to the development ...

scaling data platforms, with a strong focus on Azure Cloud technologies (e.g., Azure Data Factory, Azure Synapse, Azure Data Lake, Azure SQL Database, Databricks, Apache Spark). Proven experience leading the implementation of data platform strategies and managing large‐scale data infrastructure initiatives using Azure. Demonstrated experience implementing ...

/ELT design and implementation) Strong SQL & Data Modeling (query optimization, schema design) Programming Skills (Python/Scala for data processing) Big Data Processing (Spark/Databricks) Cloud Data Platforms (Azure/AWS/GCP - especially data services) Secondary skills API-based Data Integration & Microservices exposure Banking/Financial ...

bring: Strong SQL + ETL/ELT experience Knowledge of Python/T-SQL Exposure to cloud platforms (Azure preferred) Nice to have: Databricks, Spark, CI/CD, big data This is a great opportunity to join a high-impact data team in a global organisation and shape ...

tools, and emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated ...

Prometheus, ElasticSearch , Grafana, OpenTelemetry ) for metrics, logs and traces Platform Technologies & BSS Vendor Platforms - Open source workflow engines (Temporal) - Stream/batch processing (Flink, Spark) - Test automation: Robot Framework and BDD - can review BDD robot files - BSS vendor platforms: Salesforce, Amdocs, Ciena - BSS/OSS proficiency: awareness of Product ...

Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda is a plus. Direct experience in building and applying large-scale (100M+ users) machine learning ...

/ELT design and implementation) Strong SQL & Data Modeling (query optimization, schema design) Programming Skills (Python/Scala for data processing) Big Data Processing (Spark/Databricks) Cloud Data Platforms (Azure/AWS/GCP - especially data services ...

/ELT design and implementation) Strong SQL & Data Modeling (query optimization, schema design) Programming Skills (Python/Scala for data processing) Big Data Processing (Spark/Databricks) Cloud Data Platforms (Azure/AWS/GCP - especially data services) Secondary s click apply for full job details ...

design and implementation) Strong SQL & Data Modeling (query optimization, schema design) Programming Skills (Python/Scala for data processing) Big Data xxuwjjq Processing (Spark/Databricks) Cloud Data Platforms (Azure/AWS/GCP - especially data services ...

teams to drive outcomes Core skills (indicative) Production pipeline design and deployment experience Strong engineering capability with Python, SQL, plus big data tooling (e.g., Spark, and Java/Scala where relevant AWS, Azure, GCP Working pattern Hybrid working, with the team on client site or in the office ...

spanning model development through to data engineering Comfortable working across major cloud platforms (Azure, AWS or GCP) with hands-on exposure to tools like Spark, Hive or Redshift A track record of leading cross-functional teams and communicating technical concepts to non-technical audiences Experience with MLOps practices ...

learning frameworks (TensorFlow, PyTorch, Keras) Knowledge of prompt engineering, RAG (Retrieval Augmented Generation), and LLM fine‐tuning techniques Familiarity with distributed computing frameworks (eg Spark) Knowledge of graph neural networks, reinforcement learning, or causal inference Experience with AI governance, model risk management, and regulatory compliance Experience using Pro code ...

experience in software engineering, MLOps, and deploying machine learning models at scale.* Proficiency in SQL, Python, and familiarity with big data technologies (e.g., Kafka, Spark) and/or cloud platforms (e.g., GCP, AWS, or Azure).* Industry knowledge: Experience in gaming or digital entertainment is a strong plus. ...

experience in software engineering, MLOps, and deploying machine learning models at scale. Proficiency in SQL, Python, and familiarity with big data technologies (e.g., Kafka, Spark) and/or cloud platforms (e.g., GCP, AWS, or Azure). Industry knowledge: Experience in gaming or digital entertainment is a strong plus. ...

environments Strong experience designing large-scale, cloud-based data platforms Deep expertise in AWS services, including EC2, S3, RDS, Lambda, Redshift Proven experience with Apache Flink and Real Time streaming architectures Strong background in: Data modelling ETL/ELT processes Data warehousing and distributed systems Experience leading, mentoring …/security-focused environments Strong understanding of data governance, standards, and architecture frameworks Desirable Skills Experience with Azure or GCP Knowledge of Scala and Apache Spark Familiarity with DevOps practices and cloud-native delivery models Experience working alongside cyber security teams on secure systems Security Requirements Must currently ...

support data pipelines, ensuring they are production-ready and meet data quality standards. You will work with technologies such as Python, SQL, and Apache Spark, helping drive analytics and operational decision-making. Ideal candidates possess strong programming skills, a relevant degree, and a collaborative spirit, as you will ...

Work closely with Product, Data Science, and senior stakeholders Tech Stack Cloud & DevOps: GCP, GKE, Terraform, GitHub, Cloud Build Data & Processing: GCS, Apache Spark, PostgreSQL, Redis Frontend & Analytics: React, Power BI Languages: Python, R Profile Experience as a Technical Architect/Lead/Principal Engineer Strong background ...