51 to 75 of 115 PySpark Jobs in the UK

reservations, Event Hub TUs, serverless compute clusters, and Terraform deployments Required Skills * Azure Databricks* Git/GitHub* DevOps & Cloud Operations* Azure Functions* Data Streaming* PySpark, Python, SQL* Automation & Public Cloud* Cloud Infrastructure* Terraform* Palantir Foundry* PowerBI* Software Lifecycle Management If you meet the requirements please send me a copy ...

Data Science or Machine Learning solutions in a commercial setting. Strong programming skills in Python (preferred) or R, with experience using scikit-learn, pandas, PySpark or similar. Solid SQL skills and experience working with cloud environments (Azure advantageous). Hands-on experience with Databricks, model deployment and production workflows. ...

experience 2+ years hands-on Microsoft Fabric (Data Factory pipelines, Lakehouse) or Azure Synapse Proven experience extracting and consolidating master data from multiple ERPs PySpark and/or T-SQL Understanding of Delta Lake, medallion architecture and lakehouse patterns Understanding of data quality, data governance, and best practices Ability ...

able to learn new technologies or concepts quickly: Deep experience with Databricks on Microsoft Azure, including Delta Lake, Unity Catalog, Delta Live Tables, PySpark and Databricks SQL, alongside complementary services such as Azure Data Factory, Synapse Analytics, and Microsoft Fabric. Designing and implementing Lakehouse architectures in Databricks to unify … Databricks notebooks, Delta Live Tables, Databrick Jobs and integration with Azure Data Factory for enterprise-scale data ingestion and transformation. Proficiency in Python and PySpark for advanced data engineering, machine learning, and large-scale distributed processing within Databricks. Hands-on experience with Spark optimizations (Adaptive Query Execution, caching, partitioning ...

. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

across multiple time zones, including offshore development teams Skills required: Proven track record building scalable data pipelines (batch and streaming) in production Expert Python, PySpark, Celery and RabbitMQ skills; deep experience with AWS data stack (Glue, OpenSearch, RDS) Expert skills within SQL with experience in both transactional RDBMS systems ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Databricks-based data warehousing solution Designing and developing data models, ETL pipelines, and data integration processes Large scale data processing using PySpark Monitoring, tuning, and optimising data platforms for reliability and performance Upskilling the wider team in Databricks best practices, including modern architecture patterns Location: Atherstone (Hybrid – 3 days ...

timely manner. Key Experience, Skills and Knowledge: Proven 2 yrs+ experience as a Data Engineer or similar role, with a strong focus on PySpark, SQL, Microsoft Azure Data platforms and Power BI an advantage Proficiency in development languages suitable for intermediate-level data engineers, such as: Python/PySpark ...

timely manner. Key Experience, Skills and Knowledge: Proven 2 yrs+ experience as a Data Engineer or similar role, with a strong focus on PySpark, SQL, Microsoft Azure Data platforms and Power BI an advantage Proficiency in development languages suitable for intermediate-level data engineers, such as: Python/PySpark ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

extend AI-enabled workflows as part of the core product (e.g. decision support, automation, insight generation) Develop and maintain data pipelines using Python, PySpark, SQL , and distributed systems Make pragmatic engineering decisions that balance short-term delivery with long-term product quality Contribute to technical and architectural discussions ...

role focuses on designing, building and maintaining scalable data pipelines and products on a modern Databricks Lakehouse environment, using Delta Lake, Spark, Python (PySpark) and SQL to deliver high-quality, well-governed data across the business. You'll manage and optimise the cloud-based data lakehouse, leverage features like ...

World’s Most Ethical Companies for the 14th time. DUTIES (included, but not limited to): Monitor and maintain Microsoft Fabric data pipelines (Dataflows, PySpark Notebooks etc.) Develop and support self-service data transformation functionality using Powerquery (M Code) Work with front-end reporting developers to provide them with cleansed ...

deployed ML systems. Your Profile Essential Skills/Knowledge/Experience Strong programming skills in Python; experience with ML libraries such as Snowpark, PySpark or PyTorch. Experience with containerization tools like Docker and orchestration tools like Airflow or Astronomer. Familiarity with cloud platforms (AWS, Azure) and ML services (e.g. ...

background in data analysis, data processing, data modeling, with excellent mathematical and problem-solving skills. A background in commercial optimisation analytics. Proficiency in Python, PySpark and SQL for data extraction, transformation, and analysis. Expertise in data-driven storage solutions and handling large, complex datasets. Good understanding of classical statistics ...

experience with Azure Data Platform technologies, including Databricks and Data Factory Proven experience delivering complex ETL solutions Solid programming experience using Python, SQL and PySpark Strong stakeholder engagement skills with a diplomatic and collaborative working style Experience working within professional services or regulated environments is desirable If this role ...

STEM subject or equivalent experience with strong statistical understanding * Proven experience delivering machine learning models into production environments * Strong Python, SQL and PySpark skills for scalable, production-grade development * Hands-on experience establishing MLOps processes within Databricks * Experience building data xkybehq pipelines for structured and unstructured data * Strong stakeholder ...

strong focus on graph analytics. Proficiency in graph analytics tools and libraries such as Neo4j, NetworkX, GraphX, or similar. Strong programming skills in Python & PySpark, with experience in relevant machine learning libraries (e.g., scikit-learn, TensorFlow, PyTorch). Expertise in graph algorithms including shortest path, centrality, community detection ...

Data Engineer Data Pipelines | Python | PySpark | AWS | AWS Glue | London Location: London (hybrid - 2 days onsite per week) Salary: Up to £80K + Benefits We’re working with a tech-driven organisation looking to hire an experienced Data Engineer to help design and scale cloud-based data platforms that ...

different customer contexts and priorities Design and implement AI-enabled workflows that support decision-making and automation Build and maintain data integrations using Python, PySpark, SQL , and distributed systems Partner with customers to define success metrics and ensure solutions deliver real outcomes Feed learnings from the field back into ...

LightGBM Data Manipulation: Pandas, NumPy, LangGraph MLOps & Orchestration: MLflow, Apache Airflow Databases: Neo4j, MongoDB API Development: FastAPI Big Data (Beneficial): Apache Hadoop, Apache Spark (PySpark) What We Need From You 5-8 years of hands-on experience developing and deploying ML and/or Gen AI solutions in production. ...

Pipeline Builder Knowledge of end-to-end work - Ontology and Data connection Experience around Architectural principles Programming Skills - proficiency in Python, SQL and PySpark Data modeling and basic concepts of data science (classification, clustering, regression) Strong client-facing skills: You are comfortable interacting with both business and technical clients ...

just which tools you used Strong SQL/SSIS/Kingswaysoft and starting to use AI and automation for ETL. Python would be useful (PySpark/Pandas/NumPy) Desirable Experience migrating data into Dynamics (or similar ERP platforms) Experience working with messy, real-world data Why Join? High ...

engineering is done across the organisation. Skills & Experience Strong hands-on experience with Azure data platforms Advanced SQL skills Commercial experience using Databricks and PySpark Proven background building and maintaining scalable data pipelines If you're looking for a role where you can combine technical depth, ownership, and influence ...