76 to 100 of 104 PySpark Jobs in the UK

work with Kotlin Background in AdTech, marketing analytics, or similar data-rich domains is advantageous Experience with distributed data processing tools such as PySpark is beneficial What They Offer Salary of £70,000–£90,000 Flexible, hybrid working with minimal office requirements Opportunity to work on cutting-edge machine ...

together with reporting tools such as Tableau Desirable Experience Machine Learning and predictive modelling experience Experience working with large datasets and technologies such as PySpark or Hive Familiarity with version control systems such as Git Understanding of MLOps principles and production deployment practices Exposure to containerisation technologies such ...

dimensional modelling approaches such as Kimball or Data Vault Experience working in modern cloud environments, ideally within Azure Exposure to technologies such as dbt, PySpark, Python, or similar tools Experience with data modelling tools is beneficial. Strong preference for Sparx Enterprise Architect, or similar tools such as Visual Paradigm ...

ownership of data pipelines and automation of data workflows. Finally have 5 years Strong programming experience at an advanced level in Python, PySpark, and SQL scripting. Benefits Salary: 55,000 - 60,000 per annum 25 day's holiday Company car/Allowance Private medical insurance Variable annual bonus based … leveraging tools such as Foundry Code Workbooks, Pipelines, and Object Explorer for efficient data processing and integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability ...

ecosystem. This role will focus on building enterprise-grade data pipelines, migration frameworks and transformation processes using Azure Synapse Analytics, Azure Data Factory (ADF), PySpark, Python and SQL. You will work closely with technical and business stakeholders to ensure successful delivery of data migration and integration initiatives. Key Responsibilities … Data Factory (ADF). Develop Change Data Capture (CDC) solutions to support incremental and delta data loads. Create data transformation and cleansing processes using PySpark and Spark DataFrames. Develop Python-based utilities and automation scripts to support migration and validation activities. Design and optimise data models, SQL queries, stored ...

some of the following predictive modelling techniques preferred; GAMs, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Proficient in using WTWs Radar software Proficient at communicating results ...

modelling, Kimball, or Data Vault Exposure to modern cloud data platforms, ideally within the Microsoft ecosystem Familiarity with tools such as DBT, Python, or PySpark is beneficial Experience working in data transformation or modernisation programmes Understanding of architecture principles or a clear interest in developing into a Data Architect ...

Databricks, APIs, and integration platforms Strong understanding of MLOps and productionizing machine learning Experience leading and scaling engineering teams Tech stack: Azure, Databricks, Python, PySpark, SQL, JavaScript, Node, Bash THE ROLE: You’ll own the technical vision and delivery of our core platforms: Build world‐class platforms Architect cloud … technical leadership where required Define and own the engineering roadmap and align to resource needs Work with: Azure, Databricks, Bash, JavaScript, Node, Python, PySpark, SQL Integration & Ecosystem Enablement Own the strategy and delivery of integration services, including public and private APIs, event‐driven and asynchronous integrations, and authentication/ ...

join a leading organisation on a contract basis. This role will suit someone with strong experience across the Azure data platform , particularly Databricks and PySpark , who can take ownership of data engineering delivery from design through to implementation. This is a hybrid role requiring the candidate … data processing in a cloud environment. Key Responsibilities Design and build scalable data pipelines using Azure technologies Develop and optimise solutions in Databricks using PySpark Work with large and complex datasets, including streaming and batch data Build and maintain data platforms and data warehouses Implement ETL/ELT processes ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

lead for complex data initiatives, providing guidance on solution design, engineering standards and implementation approaches Design, build and optimise distributed data processing pipelines using PySpark, Python, Airflow and AWS services Drive engineering excellence through code reviews, technical coaching, design reviews and adoption of software engineering best practices Partner closely … Lead Data Engineer, Principal Data Engineer or Technical Lead within a large-scale, data-driven organisation Strong hands-on software engineering expertise with Python, PySpark and Apache Airflow Deep experience designing and building distributed data processing systems and large-scale data platforms on AWS Strong knowledge of modern ...

Azure experience to join their team for a contract until the end of the year. Job Responsibilities/Objectives Mandatory Skills: Databricks (PySpark/Spark SQL) , DBT , Azure DevOps (Repos, Pipelines, CI/CD), SQL ,Azure Data Services (Data Lake, Data Factory) Strong experience with Databricks (PySpark/ ...

Level Required in Years: 10 - 15 years Mandatory Required Skills Hands-on experience with Quantexa platform/Decision Intelligence Strong in Apache Spark/PySpark Proficiency in Python/Scala Experience in ETL, Data Engineering, Data Warehousing Knowledge of entity resolution, graph analytics, link analysis Experience working with large …/HDFS) Detailed Job Description Design and develop solutions using Quantexa Decision Intelligence Platform Build data ingestion, transformation, and ETL pipelines using Spark/PySpark Work on entity resolution, network analytics, and graph-based data models Integrate multiple structured/unstructured data sources for analytics use cases Ensure data ...

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time ...

with Infrastructure as Code (Terraform) Experience working with GitLab and implementing CI/CD pipelines Practical experience building data pipelines in AWS (e.g. EMR, PySpark) Solid understanding of ETL tooling (e.g. Informatica or similar) Strong background in data processing and transformation Proficiency in Python and PySpark Essential Requirements ...

days holiday, pension, healthCareer Progression + Modern Tech StackPyspark | ML | AI | Databricks | Kafka | AWSAre you a Data Engineer with strong Python, SQL and PySpark skills who is looking for the opportunity to step up and accelerate your career We're partnering with a market-leading technology business that … Build and optimise large-scale data pipelines Work with billions of data points generated from IoT devices Develop solutions using Python, SQL and PySpark Gain exposure to Databricks, Kafka and streaming technologies Contribute to Machine Learning and AI initiatives Work alongside Data Scientists and Software Engineers Influence architecture ...

Principal Consultant - Lead Data Analyst/Data Modeller Company: Transform Together Contract: 6-month Fixed Term Contract, with option to extend or become a permanent member of the team Compensation: OTE £90,000 Location: Hybrid ...

Staff Data Engineer | AWS | PySpark | Data Modernisation | London | Contract | Inside IR35 We're supporting a global, technology-led organisation undertaking a large-scale cloud data transformation programme. This isn't a traditional Data Engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns … responsible for Defining technical standards and engineering patterns for a large-scale Data Modernisation programme Re-engineering complex legacy data pipelines into production-grade PySpark Building scalable, reusable data movement frameworks across multiple engineering teams Driving best practice around AWS Glue, Apache Spark and Apache Airflow Designing high-quality ...

Robert Half Technology are assisting UK's market leading legal organisation to recruit a Data Engineer on a 12 month contract basis - Hybrid working - London based. A Data Engineer is required to join a growing ...

domain's data products, using the tools and guardrails provided by the Enablement team. \n Expected Skills: Advanced coding in Python and PySpark, strong SQL, expertise in building declarative pipelines (eg; Delta Live Tables), version control (Git), automated testing (eg; Pytest), and CI/CD deployment practices \n Expectations … automate data pipelines to ingest, transform, and model data specific to the squad's use cases. \n \nWrite structured, scalable code (eg; Python/PySpark/SQL) and apply software engineering best practices, including version control (Git) and automated testing.Work closely with the Platform Enablement team to package their ...

Architectural Leaders hipDefine and evolve the enterprise data architecture strategy, with a primary focus on Cloud-scale Distributed Data Processing using Apache Spark and PySpark at the core of the platf ormEstablish and maintain architectural artefacts, reference architectures, Architecture Decision Records (ADRs), technical standards, and architecture roadmaps … large, complex, data-led organisation, operating at an influence and strategy level rather than hands-on de liveryExpert-level understanding of Apache Spark and PySpark as the foundation of distributed data processing architectures, including tuning, optimisation, and enterprise-scale deployment pa tternsDeep expertise in designing and evolving Data LakeHouse ...

Doing Leading and mentoring a team of Data Engineers Designing and building scalable data pipelines in Databricks Remaining hands on with Python/PySpark development Working closely with product, frontend, and backend teams Integrating multiple data sources, including APIs Driving best practice across the data platform Helping shape … future data architecture What We’re Looking For Essential: Strong commercial experience with Databricks Advanced Python and PySpark Solid SQL skills Experience building production data pipelines Experience working with API based data ingestion Familiarity with Azure storage connectivity Prior technical leadership or mentoring experience Nice to have: Experience with ...

pricing models with advanced data science techniques using WTW Radar and Emblem. Query and engineer large datasets (e.g., Python/R/SQL/PySpark) on modern platforms (e.g., Azure Databricks). Create innovative data solutions and present actionable insights to stakeholders. Communicate results clearly to key decision-makers … Street Pricing. What we're looking for: Experience in general insurance pricing with strong modelling skills. Proficiency in programming languages such as R, Python, PySpark, SAS, or SQL. Good knowledge of Azure Databricks and modern data technologies. Experience with WTW Radar software is preferred. A quantitative degree in Mathematics ...

with Microsoft Fabric (Lakehouses, Pipelines, Notebooks) in a live production environment Building, enhancing Data Factory pipelines as part of an ongoing migration into Fabric PySpark for data transformation, engineering within Fabric notebooks SQL queries, transformation logic within Fabric CRM data (Dynamics 365/Dataverse), supporting integration, modelling, optimisation Business … experience Strong SQL skills, including complex queries, transformations Hands-on experience with Microsoft Fabric Experience building, maintaining data pipelines (Data Factory/Fabric Pipelines) PySpark experience for data engineering, transformation Exposure to CRM data, understanding of how CRM supports business processes Ability to work with stakeholders to identify data ...

Microsoft Fabric (Lakehouses, Pipelines, Notebooks) in a live production environment Building and enhancing Data Factory pipelines as part of the ongoing migration into Fabric PySpark for data transformation and engineering within Fabric notebooks SQL for querying and implementing transformation logic within Fabric Dynamics 365/Dataverse data, supporting integration … including complex queries and data transformations Hands-on experience with Microsoft Fabric Proven experience building and maintaining data pipelines (Data Factory/Fabric Pipelines) PySpark experience for data engineering and transformation Exposure to CRM data, with an understanding of how it underpins business processes Ability to engage with stakeholders ...