201 to 225 of 313 PySpark Jobs in the UK

Introduction At Gallagher, we help clients face risk with confidence because we believe that when businesses are protected, they’re free to grow, lead, and innovate. You’ll be backed by our digital ecosystem: a ...

Title**: **Data Engineer** **Python, PySpark, and SQL****Location**: Northampton**About the Job you are considering:**We are seeking an experienced Data Engineer with strong expertise in Python, PySpark, and SQL to design, develop, and optimize large-scale data processing solutions. The ideal candidate will have hands-on experience … best practices.**Skills*** 7–9 years of experience in Data Engineering or related roles.* Strong hands-on experience in Python development.* Proven experience with PySpark for large-scale data processing.* Strong knowledge of SQL, including query optimization and performance tuning.* Exposure to or working knowledge of Scala.* Experience working ...

consulting and technology services. The candidate will design, build, and maintain scalable data pipelines, data lakes, and data warehouse solutions on AWS, using PySpark, Spark, Python, and AWS services to enable #J-18808-Ljbffr ...

Data Engineer in the United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. ...

supporting the Intelligence Community or Department of Defense environments Experience with Python and working with databases Experience with big data frameworks such as Spark, PySpark, or Elastic Search Experience with data visualization platforms such as Kibana, Foundry Workshop, or Palantir Knowledge of secure coding practices and development in classified ...

experience, delivering measurable business value. You’ll work end-to-end from data exploration and feature engineering to model development and deployment, using Python, PySpark, SQL and Azure Databricks, and communicate insights to both technical and business stakeholders. #J-18808-Ljbffr ...

London seeks a Senior Data Engineer to design, build, and operate enterprise-scale data solutions. You will leverage Snowflake with hands-on Python, PySpark, AWS and Airflow to create scalable data platforms and robust analytics capabilities. The role emphasizes technical leadership, governance, and collaboration with cross-functional teams ...

contract. Contract Details: Inside IR35 Initial 6-month engagement Sheffield-based, 3 days per week on-site Key Requirements: Strong hands-on experience with PySpark and Databricks Deep understanding of Spark performance optimisation, including lazy evaluation, column pruning, groupBy operations, and window functions Proven ability to troubleshoot and resolve ...

drive Ab Initio deployments, and help migrate workloads to cloud services. You will design, build, and migrate Ab Initio workflows to AWS Glue/PySpark, write production-grade Python, and implement AI/LLM workflows with observability. This fast-moving role offers remote work with limited on-site days ...

balance of on-site and hybrid working arrangements. Ideal candidates have hands-on experience with Azure data services, strong SQL and Python/PySpark skills, and a passion for engineering best practices, governance, and performance tuning. #J-18808-Ljbffr ...

analytics in enterprise environments. Demonstrated experience extracting meaningful signals from high-volume, high-noise telemetry or behavioural datasets. Proficiency in Python (pandas, scikit-learn, PySpark) and statistical modelling. Experience building structured, repeatable analytical frameworks — use case scoring, feasibility assessments, or prioritisation models — not purely ad hoc analysis. Background ...

performance optimisation. Hands-on delivery experience within hospitality reporting and commercial reporting environments. Experience working with Databricks is useful. Ability to interpret Python/PySpark is beneficial. If you are interested in this Power BI Developer role please apply with your most recent CV or reach out on jordan ...

unsupervised techniques Experience with tree-based and boosting algorithms such as XGBoost, LightGBM, and related approaches Experience working with large-scale datasets, ideally using PySpark Strong analytical and SQL skills for handling, querying, and interpreting complex datasets Ability to communicate insights effectively through visualisations using tools such as Matplotlib ...

teamWhat could set you apartExperience in the use of credit bureau dataExperience using Google Cloud Platform and/or Google Cloud Engineer qualificationsExperience of PySpark, BigQuery, Git and any/all of: DataFlow, DataProc, Airflow (or equivalents)We believe great things happen when teams connect. Our schedule is built ...

teamWhat could set you apartExperience in the use of credit bureau dataExperience using Google Cloud Platform and/or Google Cloud Engineer qualificationsExperience of PySpark, BigQuery, Git and any/all of: DataFlow, DataProc, Airflow (or equivalents)We believe great things happen when teams connect. Our schedule is built ...

closely with SMEs and the Data Architect to shape our data management strategy and meet business needs. Ideal candidates bring deep experience with Databricks, PySpark, Azure ecosystems, and ELT pipeline development, with an emphasis on data modelling and effective collaboration across IT and business teams. #J-18808-Ljbffr ...

standards, architect scalable ETL/ELT frameworks, and provide technical leadership across large transformation programmes. The role requires deep expertise in ADF, Azure Databricks, PySpark, SQL, XML and governance, with strong stakeholder engagement and delivery oversight. #J-18808-Ljbffr ...

IR35 . Rate: £650 - £750 Location: Hybrid – 2/3 days per week on-site Contract: 6 months | Outside IR35 Key Skills: Data Lakehouse PySpark DBT Iceberg AWS Strong data engineering and pipeline development experience You’ll be responsible for designing, building and optimising scalable data pipelines and lakehouse ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

/ELT patterns, data modeling (dimensional + normalized), and batch scheduling dependencies. Ability to interpret data transformation logic from pipelines (Spark SQL/PySpark/Hive queries/orchestration configs). Strong documentation capability: source‐to‐target mappings, lineage diagrams, data dictionaries, metadata standards, and control evidence packs. Technical … Skills Strong proficiency in Python (data analysis/automation for metadata extraction, validation scripts, rule checks). Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases is preferred. Working knowledge of Autosys ...

including architecture, development, deployment, and optimization. Define Databricks workspace architecture, clusterstrategies, security models, and operational standards. Design and developscalable data pipelinesusing Databricks, Apache Spark, PySpark, SQL, andDelta Lake . Implement data ingestionframeworks supporting batchand real-timedata processing. Design Delta Lake architectures including: Delta tables Medallion architecture (Bronze/Silver … reviews. Skills & Experience Strong experience in dataengineering, big data, or solution architecture. Strong hands-onexperience with Databricks Platform . Expert knowledge of: Apache Spark PySpark/Scala Delta Lake SQL Data Lakehouse architecture ETL/ELT development Experience designing anddelivering enterprise Databrickssolutions. Strong experience withcloud platforms: Azure Databricks (preferred ...

designing, developing and maintaining cutting-edge data solutions within Microsoft Fabric. You'll build and optimise data pipelines, lakehouses and data warehouses, using PySpark, Python, T-SQL and DAX to deliver scalable, high-performing solutions that drive business intelligence and analytics. Working closely with stakeholders across the business … years' experience in Data Engineering and hands-on experience with Microsoft Fabric, Databricks, Snowflake or similar cloud platforms. You'll have strong Python/PySpark skills, experience integrating with APIs, and a background in Agile development. If you're a detail-oriented problem solver with excellent analytical skills ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

Function Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - Edinburgh, London, Manchester Skills Python & SQL: Advanced Python (PySpark GlobalLogic is seeking a Data & AI Architect working in our UK & Ireland Go to Market Team to drive pipeline growth and lead enterprise transformation pursuits across … mindset, with experience working across geographies including India, East Europe and North America. Proven track record of acceleration of Business growth Deep expertise in PySpark/Python, Advanced SQL, data modeling, batch/real-time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge ...

Function Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - Edinburgh, London, Manchester Skills Python & SQL: Advanced Python (PySpark GlobalLogic is seeking a Data & AI Architect working in our UK & Ireland Go to Market Team to drive pipeline growth and lead enterprise transformation pursuits across … mindset, with experience working across geographies including India, East Europe and North America. Proven track record of acceleration of Business growth Deep expertise in PySpark/Python, Advanced SQL, data modeling, batch/real-time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge ...