301 to 325 of 572 Apache Spark Jobs in the UK

forecasting and predictive modelling. Advanced Python skills and experience with common ML libraries. Strong SQL skills. Experience working with Databricks and/or Spark environments. Ability to translate business requirements into practical AI and ML solutions. Experience communicating technical concepts to business stakeholders. Strong problem-solving skills with ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing Experience with containerisation (Docker) and scalable data systems (e.g. Spark, Kafka) Nice-to-Have Experience with AWS SageMaker or similar managed ML platforms Background in safety-critical or regulated industries (aerospace, aviation, or similar ...

work effectively with technical teams including Data Scientists and MLOps Engineers. Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/vector ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing Experience with containerisation (Docker) and scalable data systems (e.g. Spark, Kafka) Nice-to-Have Experience with AWS SageMaker or similar managed ML platforms Background in safety-critical or regulated industries (aerospace, aviation, or similar ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

hybrid-cloud operating model Evaluate proposed solution architectures and provide recommendations Provision, configure, upgrade, and monitor technologies such as Kubernetes (EKS and on-premise), Spark, Kafka, and other large-scale distributed databases Automate infrastructure management and development pipelines with tools like Terraform, Ansible, Jenkins and GitLab Explore new technologies ...

such as Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. ...

collaboratively Strong communication and interpersonal skills Preferred qualifications: Experience in the gaming or entertainment industry Experience working with tools and technologies such as Spark, Snowflake, Qwak, MLFlow, Airflow, etc Contributions to open‐source AI/ML projects or participation in AI/ML competitions (e.g., Kaggle) Experience with training ...

collaboratively Strong communication and interpersonal skills Preferred qualifications: Experience in the gaming or entertainment industry Experience working with tools and technologies such as Spark, Snowflake, Qwak, MLFlow, Airflow, etc Contributions to open‐source AI/ML projects or participation in AI/ML competitions (e.g., Kaggle) Experience with training ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing. Experience with containerization (Docker) and scalable data systems (e.g., Spark, Kafka). Strong leadership presence, excellent communication, strategic thinking, and empathy with a hands-on execution mindset. Desirable Requirements: Experience with AWS SageMaker ...

Candidate Requirements: Extensive experience building and deploying machine learning models in production, with strong hands-on Python capability. Experience with distributed computing frameworks (e.g., Spark or similar) and scalable data processing environments. Exposure to cloud-based platforms (e.g., AWS, Azure, GCP or Databricks) and modern ML/MLOps practices. ...

manipulation and analysis of data. Manage and integrate data across dissimilar data sets. Analyze large-scale structured and unstructured data. Use frameworks such as Spark and Hadoop to conduct large-scale data processing. Perform statistical modeling and create data visualizations using products like Tableau, Microsoft Power ...

data libraries such as Pandas, NumPy and Scikit-learnA solid grounding in statistical methodologies, along with data extraction, manipulation and feature engineering techniquesExperience with Spark, agile delivery methods and CI/CD practicesFamiliarity with DevOps and MLOps tools and practices, such as Docker, Terraform and MLFlowConfidence influencing and communicating ...

such as Pandas, NumPy and Scikit-learn A solid grounding in statistical methodologies, along with data extraction, manipulation and feature engineering techniques Experience with Spark, agile delivery methods and CI/CD practices Familiarity with DevOps and MLOps tools and practices, such as Docker, Terraform and MLFlow Confidence influencing ...

technologies that our Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing. Discover more about ...

engines. Optimize processing engines for scalability and latency Expand capabilities to support new databases and cloud platforms Tech stack: Java, Python, Angular, Kubernetes, Spark, AWS, Azure, GCP AI Governance: Build features for AI compliance, connecting disparate systems and simplifying governance processes. Develop customizable platforms for managing AI compliance ...

agentic systems. Experience designing external‐facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications, merit ...

agentic systems. Experience designing external-facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications, merit ...

tools e.g. Microsoft Power BI, Spotfire, TableauExperience of briefing and communication analytical products to non-technical staffExperience of using big data ecosystems e.g. Hadoop, Spark and/or MapReduce. Experience of working in NATO or similar multi-national defence/security sector. Why work with us? By joining ...

related field, or equivalent practical experience. Experience with AWS or GCP, Kubernetes, SQL warehouse and batch-processing technologies such as Trino, Presto or Spark, Kafka or other streaming systems, and Go or Python would be useful. Node.js experience is a plus. We value transferable evidence over an exact match ...

develop homogenous data repositories to deliver data led decision making. Ingest data from various sources, including on premise SQL databases, REST API’s, Apache Kafka streams, etc. Define and apply cleansing rules to the data to ensure meets quality expectations. Define and model data for its appropriate use including … etc.) Other languages, such as C#, PowerShell IaC, e.g. Terraform, ARM, Bicep, etc. Test driven development (TDD) Streaming technologies, such as Azure Stream Analytics, Spark Structured Streaming, etc. Power BI Engineering experience Certified SCRUM Developer (CSD) Data Governance tool experience e.g. Purview Machine learning and Artificial intelligence ...

Responsibilities Azure Platform Support & Monitoring Support and maintain Azure-based data solutions, including Azure Data Factory pipelines, datasets, linked services, triggers; Azure Databricks (Spark jobs, notebooks, clusters); Azure Machine Learning models and endpoints; Power BI dashboards and dataset refreshes. Monitor and troubleshoot failures in pipelines, jobs, and ML workflows … detailed SOPs, runbooks, knowledge‐base articles, and client documentation. Required Skills and Qualifications Azure Services – Azure Data Factory (pipelines, triggers, parameterization, monitoring); Azure Databricks (Spark, notebooks, job orchestration); Azure Machine Learning (pipelines, model deployment, monitoring); Power BI Service (dataset refreshes, access control, report diagnostics). DevOps & Automation – CI/ ...

tasks. Key Responsibilities: Azure Platform Support & MonitoringSupport and maintain Azure-based data solutions, including: Azure Data Factory (ADF) pipelines, dataset, linked service, triggerAzure Databricks (Spark jobs, notebooks, clusters)Azure Machine Learning models and endpointsPower BI dashboards and dataset refreshesMonitor and troubleshoot failures in pipelines, jobs, and ML workflows using … Maintain detailed SOPs, runbooks, knowledge base articles, and client documentation. Required Skills and Qualifications: Azure ServicesAzure Data Factory (ADF): pipelines, triggers, parameterization, monitoringAzure Databricks: Spark, notebooks, job orchestrationAzure Machine Learning: pipelines, model deployment, monitoringPower BI Service: dataset refreshes, access control, report diagnosticsDevOps & AutomationCI/CD: Azure DevOps, GitHub Actions ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes Big Data Analytics platforms Cloud-based solutions experience AWS preferred Software Engineering Expertise with: RDBMS … Knowledge of: MLOps Feature Engineering Exposure to data analysis and ML tools such as: scikit-learn pandas NumPy XGBoost LightGBM Familiarity with orchestration tools: Apache Oozie Apache Airflow Why you’ll love it here We want to make Prima a happy and empowering place to work. ...