151 to 175 of 297 Apache Spark Jobs in England

/CD paradigms and systems such as: Ansible, Terraform, Jenkins, Bamboo, Concourse etc. Observability – SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Excellent knowledge of YAML or similar languages The following Technical Skills & Experience would be desirable: Jupyter Hub Awareness RabbitMQ ...

practices. Proven experience in designing, implementing and maintaining Software solutions. Hands-on experience with any cloud, AWS is preferred. Advanced proficiency in Python and Apache Spark, with a strong focus on ETL data processing and scalable data engineering workflows. Our Package 10% on target bonus A BT Pension ...

machine learning and data science frameworks. Hands-on experience with large language models, generative AI, prompt engineering, and RAG architectures. Experience with Databricks, MLflow, Spark, and cloud-based AI platforms (Azure preferred). Strong understanding of MLOps, software engineering best practices, and production deployment pipelines. Experience leading technical teams ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

including Feast or AWS Feature Store. Docker, Kubernetes, or Amazon ECS. Infrastructure as Code using Terraform or CloudFormation. Big data processing technologies such as Spark or Dask. Experience working within regulated or highly secure environments. What's On Offer Work on meaningful AI programmes supporting critical national security missions. ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing Experience with containerisation (Docker) and scalable data systems (e.g. Spark, Kafka) Nice-to-Have Experience with AWS SageMaker or similar managed ML platforms Background in safety-critical or regulated industries (aerospace, aviation, or similar ...

such as Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. ...

platforms (AWS, Azure, or GCP).Distributed Systems Mastery: Deep technical capability in structuring data platforms, data pipelines, and high-throughput distributed systems (like Spark or Kafka) optimized for performance and scalability. Cross-Functional Technical Leadership: Exceptional ability to bridge the gap between deep technical engineering and strategic business needs ...

manipulation and analysis of data. Manage and integrate data across dissimilar data sets. Analyze large-scale structured and unstructured data. Use frameworks such as Spark and Hadoop to conduct large-scale data processing. Perform statistical modeling and create data visualizations using products like Tableau, Microsoft Power ...

data libraries such as Pandas, NumPy and Scikit-learnA solid grounding in statistical methodologies, along with data extraction, manipulation and feature engineering techniquesExperience with Spark, agile delivery methods and CI/CD practicesFamiliarity with DevOps and MLOps tools and practices, such as Docker, Terraform and MLFlowConfidence influencing and communicating ...

such as Pandas, NumPy and Scikit-learn A solid grounding in statistical methodologies, along with data extraction, manipulation and feature engineering techniques Experience with Spark, agile delivery methods and CI/CD practices Familiarity with DevOps and MLOps tools and practices, such as Docker, Terraform and MLFlow Confidence influencing ...

both OnPrem and Cloud* Experience with processing large datasets as well as streaming data and unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java ...

agentic systems. Experience designing external‐facing APIs, including versioning, backwards compatibility, and OpenAPI/REST standards. Familiarity with data infrastructure such as Airflow, Spark, Iceberg, or Kafka. Experience with energy markets. Modo Energy is an equal opportunity employer. Employment decisions are made on the basis of qualifications, merit ...

develop homogenous data repositories to deliver data led decision making. Ingest data from various sources, including on premise SQL databases, REST API’s, Apache Kafka streams, etc. Define and apply cleansing rules to the data to ensure meets quality expectations. Define and model data for its appropriate use including … etc.) Other languages, such as C#, PowerShell IaC, e.g. Terraform, ARM, Bicep, etc. Test driven development (TDD) Streaming technologies, such as Azure Stream Analytics, Spark Structured Streaming, etc. Power BI Engineering experience Certified SCRUM Developer (CSD) Data Governance tool experience e.g. Purview Machine learning and Artificial intelligence ...

tasks. Key Responsibilities: Azure Platform Support & MonitoringSupport and maintain Azure-based data solutions, including: Azure Data Factory (ADF) pipelines, dataset, linked service, triggerAzure Databricks (Spark jobs, notebooks, clusters)Azure Machine Learning models and endpointsPower BI dashboards and dataset refreshesMonitor and troubleshoot failures in pipelines, jobs, and ML workflows using … Maintain detailed SOPs, runbooks, knowledge base articles, and client documentation. Required Skills and Qualifications: Azure ServicesAzure Data Factory (ADF): pipelines, triggers, parameterization, monitoringAzure Databricks: Spark, notebooks, job orchestrationAzure Machine Learning: pipelines, model deployment, monitoringPower BI Service: dataset refreshes, access control, report diagnosticsDevOps & AutomationCI/CD: Azure DevOps, GitHub Actions ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes Big Data Analytics platforms Cloud-based solutions experience AWS preferred Software Engineering Expertise with: RDBMS … Knowledge of: MLOps Feature Engineering Exposure to data analysis and ML tools such as: scikit-learn pandas NumPy XGBoost LightGBM Familiarity with orchestration tools: Apache Oozie Apache Airflow Why you’ll love it here We want to make Prima a happy and empowering place to work. ...

unify data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits Databricks strives to provide comprehensive benefits and perks. For specific details in your region, refer ...

technical and technical audiences alikePassion for collaboration, life-long learning, and driving business value through AI[Preferred] Experience using the Databricks Intelligence Platform and Apache Spark to process large-scale distributed datasetsWilling to travel once every 4-8 weeks to see customers (as needed)About DatabricksDatabricks ...

develop pipelines for analytics, ML and business intelligence in a security-conscious environment. The role requires 3+ years in data engineering, strong experience with Spark, Python, Kafka, SQL and Airflow, and a willingness to work hybrid in London or Cheltenham. You will contribute to client engagements and growth ...

analytics and ML teams across multiple brands. Ideal candidates have 5+ years in data engineering, strong SQL and data modelling, and experience with dbt, Spark, Airflow, AWS, Kafka or Flink. #J-18808-Ljbffr ...

Databricks. Familiarity with cloud‐based application development (AWS & Azure). Familiarity with one or more ML frameworks and technologies: scikit‐learn, xgboost, TensorFlow, PyTorch, Spark, SageMaker, Vertex AI, Kubeflow, Seldon, Triton. Strong communication skills, able to express ideas clearly and collaborate across teams. #J-18808-Ljbffr ...

Python and MLflow Strong AWS/cloud-native experience ( ideally with SageMaker exposure) React frontend proficiency Docker, Kafka, and large-scale data systems (e.g. Spark) experience Strong leadership presence and senior stakeholder communication skills If you have the technical expertise, leadership experience and ambition to make an impact ...

Python and MLflow Strong AWS/cloud-native experience ( ideally with SageMaker exposure) React frontend proficiency Docker, Kafka, and large-scale data systems (e.g. Spark) experience Strong leadership presence and senior stakeholder communication skills If you have the technical expertise, leadership experience and ambition to make an impact ...