16 of 16 Apache Spark Jobs in the Midlands

should have specialist experience in one or more of the following technologies**Azure Databricks*** Design and build high-performance data pipelines: Utilize Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services.* Experience of Databricks ML and Azure … develop predictive models and drive business insights.* Proven expertise in Databricks, Apache Spark, and data pipeline development and strong understanding of data warehousing concepts and practices.* Experience with Microsoft Azure cloud platform, including Azure Data Lake Storage, Databricks and Azure Data Factory.* Azure Data Engineer Associate and Databricks ...

alignment with business priorities and best practice Implementing robust ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines and Azure Databricks (PySpark/Spark SQL) to support batch and streaming data processing Designing and delivering real-time and batch data ingestion solutions using Azure Event Hubs, Kafka, Databricks ...

/Redshift/SageMaker (Or relevant experience in another cloud technology). Specific tooling - Databricks Strong Data related programming skills SQL/Python/Spark Experience of industrialising and scaling machine learning models. Experience in deployment, monitoring, retraining and lifecycle management of ML models. Understanding of containerisation and infrastructure ...

Vetting (DV). Preferred technical and professional experience Experience with machine learning frameworks (TensorFlow, PyTorch, scikit-learn). Familiarity with big data technologies (Hadoop, Spark). Background in data science, IT consulting, or a related field. AWS Certified Big Data or equivalent. Demonstrated leadership and mentoring skills. Publication record ...

drift monitoring; versioning and release management. Cloud architecture (Azure/AWS/GCP), Kubernetes/Docker, serverless, IAM and network security. Data engineering depth (Spark/Databricks; ETL/ELT); cloud‐native data + AI architectures. Enterprise integration and SRE principles (SLIs/SLOs, runbooks, rollback). Consulting leadership ...

build out technology roadmapsCodebase ownershipTechnical Application ownershipQualificationsStrong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , HarnessStrong experience of developing on Apache Spark/HadoopStrong experience with AWS cloud services including AWS Glue, S3, StepStrong experience building data pipelines and data lakeExperience of the full ...

Building and maintaining data pipelines. Data modelling and warehousing fundamentals. dbt or similar modern transformation frameworks Experience with AWS data services. Exposure to Kafka, Spark, streaming data, or modern BI tools. Interest in AI, analytics engineering, and data products THE BENEFITS You will receive a salary ...

Building and maintaining data pipelines. Data modelling and warehousing fundamentals. dbt or similar modern transformation frameworks Experience with AWS data services. Exposure to Kafka, Spark, streaming data, or modern BI tools. Interest in AI, analytics engineering, and data products THE BENEFITS You will receive a salary ...

with cloud‐based data warehouses (e.g., Snowflake, GBQ). Familiarity with cloud‐based data pipeline services (e.g. AWS Glue) or distributed processing frameworks like Apache Spark. Equal Opportunity Employment Goldman Sachs is an equal opportunity employer and does not discriminate on the basis of race, colour, religion, sex, national ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

management and communication skills. Desirable Skills A postgraduate qualification in a numerate subject. Databricks notebooks, workflows and Declarative Automation Bundles (AKA Asset Bundles). Spark or PySpark. Training machine learning models in Databricks. Time-series forecasting and classification models. Applied large language model experience. Azure DevOps Repos, Boards ...

engines. Optimize processing engines for scalability and latency Expand capabilities to support new databases and cloud platforms Tech stack: Java, Python, Angular, Kubernetes, Spark, AWS, Azure, GCP AI Governance: Build features for AI compliance, connecting disparate systems and simplifying governance processes. Develop customizable platforms for managing AI compliance ...

Applied NLP and computer vision techniques Deployed AI models to production environments Implemented deep learning frameworks (TensorFlow, PyTorch) Worked with big data platforms (Hadoop, Spark) Collaborated with software developers for AI solutions Ensured scalability and performance of AI systems Documented model architecture and results Monitored deployed AI models ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey ...

Location: Northampton, UK --------------------------------------- JOB DETAILS Role Title: Sr Data Engineer • Minimum 6-8 years of experience working as Sr Data Engineer • Bigdata Technologies – PySpark, Spark • Coding Languages – Python, SQL • AWS services – Glue, Cloud formation, Lake Formation, IAM, Lambda, Redshift • Exposure to cloud technologies – Snowflake, Redshift, Apache Flink … Apache Kafka. • Work in an Agile environment and participation in scrum daily standups, sprint planning reviews and retrospectives. • Understand project requirements and translate them into technical solutions which meets the project quality standards • Ability to work in team in diverse/multiple stakeholder environment and collaborate with upstream/ ...

Software Engineer - Space Domain Awareness | Job DetailsAre you looking for an innovative and consolidated place for your professional development? GMV is an experienced technology group and we are looking for a highly motivated Software Engineer ...