101 to 125 of 382 PySpark Jobs in the UK

actually trust and use. Essential Skills & Experience: Hands-on experience with Databricks and Microsoft Fabric Experience building and running production-grade pipelines Strong PySpark and SQL skills Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi ...

grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. Build … ongoing evolution of Envitia’s data engineering capability. Essential Skills & Experience Proven hands‐on experience building and supporting production‐grade data pipelines using Databricks, PySpark, Spark SQL and Delta Lake. Experience designing and implementing Lakehouse architectures, including Medallion (Bronze, Silver, Gold) data models. Expertise in developing scalable ETL/ ...

with Databricks SQL endpoints for data access. Ensure performance, scalability, and security of backend systems. Databricks & Data Engineering Develop ETL pipelines and workflows using PySpark on Databricks. Work with Delta Lake, Unity Catalog, and Databricks REST APIs. Optimize queries and pipelines for cost efficiency in cloud environments (Azure/ ...

standards and documentation What we’re looking for 3–5 years of data engineering experience Strong Python & SQL skills Experience with Apache Spark/PySpark Ability to monitor, troubleshoot, and resolve pipeline failures Familiarity with lakehouse concepts Experience with Git, Databricks, streaming/CDC, AWS, Great Expectations Nice ...

Engineer Associate (DP-600), Power BI Data Analyst Associate (PL-300), or Azure Data Fundamentals (DP-900). Proficiency in Python and SQL, including PySpark and Spark SQL, used across our Microsoft Fabric data platform. Experience with ERP/finance systems data (NetSuite, SAP, Dynamics, etc.) Experience picking ...

years of relevant in data governance, compliance, or automation programs. Strong proficiency in Python for data engineering and automation. Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet . Advanced SQL skills; experience with Oracle databases . Experience designing ...

/reporting outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Hands-on exposure to DCRM tooling and operational exception management processes. Experience with governance/ ...

Azure Resource Manager/Bicep, or Google Cloud Deployment Manager) to build, maintain, and scale complex cloud environments Strong coding proficiency in Python and PySpark/Spark: you must demonstrate live coding, debugging, and solution‐building skills Deep expertise in distributed data systems architecture: designing scalable pipelines, streaming architectures ...

Azure Resource Manager/Bicep, or Google Cloud Deployment Manager) to build, maintain, and scale complex cloud environmentsStrong coding proficiency in Python and PySpark/Spark: you must demonstrate live coding, debugging, and solution-building skillsDeep expertise in distributed data systems architecture: designing scalable pipelines, streaming architectures, lakehouse patterns ...

would be beneficial, but we're open to candidates with transferable experience from other cloud platforms. (Desirable) Progress OpenEdge experience. (Desirable) Databricks, Spark or PySpark experience. (Desirable) Experience feeding data into AI systems, including RAG pipelines, vector stores, embedding workflows, or using Azure AI Foundry or similar AI/ ...

ingest, cleanse and model data for BI reporting, ML and AI, ensuring quality and governance. The role emphasizes strong Azure skills (ADF, ADLS, Databricks), PySpark, Python and SQL, with a DevOps and CI/CD mindset. Hybrid work with three days in the office weekly is offered. #J ...

troubleshooting, and collaboration with analysts and engineers to raise data quality. The ideal candidate has 6+ years in data engineering, strong SQL, Python/PySpark, and experience with Databricks and Azure, with a passion #J-18808-Ljbffr ...

pipeline development, data modelling, semantic layers and Power BI delivery. This role focuses on building a modern data platform using Azure Databricks, Python, PySpark and SQL, tying together marketing, sales and operations data to drive commercial performance. #J-18808-Ljbffr ...

tools, and emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated ...

Data Engineer Key skills: Python SQL Server/SQL Data Warehousing & Data Modelling Data Architecture Data Pipelines & Integration Solutions Microsoft Fabric Azure Cloud Technologies PySpark ETL/ELT Development Data Governance & Security Source Control (Git) Technical Leadership & Team Management You will be a Lead Data Engineer Developer with good ...

Data Engineer Key skills: * Python * SQL Server/SQL * Data Warehousing & Data Modelling * Data Architecture * Data Pipelines & Integration Solutions * Microsoft Fabric * Azure Cloud Technologies * PySpark * ETL/ELT Development * Data Governance & Security * Source Control (Git) * Technical Leadership & Team Management You will be a Lead Data Engineer Developer with good ...

that deeply cares about its values. To use a range of different tools and technologies depending on the specific task, these include: Databricks, Python, PySpark, R, Excel, Palantir Foundry, SQL and PowerBI. Qualifications Even if you don’t meet every requirement below, feel free to still apply ...

Lake and Lakehouse architecture for efficient data management; experience with ETL processes and optimizing data pipelines for performance Be strong in Python, SQL, and PySpark Be passionate about building data products for stakeholders and customers, ensuring they are stable, scalable, secure, accurate, observable, and performant Enjoy collaborating closely with ...

about its values Values | PA ConsultingWe use a range of different tools and technologies depending on the specific task, these include: Data Bricks, Python, PySpark, R, Excel, Palantir Foundry and PowerBI.QualificationsEssential requirements Even if you don't meet every requirement below, feel free to still apply ...

about its values Values | PA ConsultingWe use a range of different tools and technologies depending on the specific task, these include: Data Bricks, Python, PySpark, R, Excel, Palantir Foundry and PowerBI.QualificationsEssential requirements Even if you don't meet every requirement below, feel free to still apply ...

insights clearly to both technical and non-technical audiences. Some other highly valued skills include: Strong SQL and database experience. Knowledge of Python/PySpark and data analytics. Experience working in a Call Centre or Contact Centre environment. Strong communication and stakeholder management skills. Ability to lead technical teams ...

PostgreSQL, MySQL, and Aurora. Integrating and migrating data using AWS DMS, Glue Connectors, EventBridge, SNS, and SQS. Strong programming capability in Python and PySpark for large-scale data processing. Proficiency in writing complex SQL queries, analytical functions, and performance tuning for large datasets. Familiarity with NoSQL databases such ...

Associate in Internal Audit - Data Engineering to design and implement robust data pipelines and analytics. You will work with Python, Pandas, Polars and PySpark to deliver scalable data products and AI-ready insights for audit teams. You will collaborate with data owners, develop data models, and ensure data quality ...

looking for a Senior Data Engineer - Tech Lead (Databricks, PySpark) to join our team in London, UK, in a hybrid working mode. In this role, you will lead the design, development and optimization of scalable cloud-native data architectures, focusing on Azure Databricks, PySpark and Lakehouse principles. … design and build of large-scale data platforms using Azure Databricks and modern cloud technologies Implement and optimize ETL workflows and streaming pipelines with PySpark and Delta Live Tables following Lakehouse principles Enhance performance, manage cloud costs and ensure platform reliability for structured streaming workloads Define data governance, security ...

teamworkStrong problem-solving and analytical skillsPreferred qualifications, capabilities, and skillsExperience with modern data platforms such as Databricks or SnowflakeHands-on expertise with Spark/PySpark and big data processing technologiesKnowledge of open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration frameworks and model serving ...