101 to 125 of 150 PySpark Jobs in England

data.Knowledge of Bayesian methods for causal analysis and decision-making.Experience in gaming, e-commerce, or subscription-based products.Experience working with large-scale data using PySpark or equivalent distributed data processing tools.Familiarity with production environments, MLOps, or data pipelines.Benefits: Discretionary bonus opportunityPrivate Medical InsuranceDental Scheme25 days holiday per yearOn Site ...

analytics capabilities. We welcome candidates from all backgrounds to apply, and we are committed to creating a diverse workforce.Key Responsibilities and Skills: - Proficiency in PySpark for effective data manipulation and transformation. - Experience with Microsoft B.I and DataBricks for advanced analytics solutions. - Strong problem-solving abilities to address complex challenges. ...

location strategy.“I am the person Capital Group is looking for.” Proven track record delivering solutions in complex organizations. Experience with AWS, Databricks, DBT, PySpark, React, JavaScript Terraform, etc. Fast learner, thrives in ambiguity, translates vision into product scope. Passion for automation and user experience innovation. Builds high-performing ...

play a key role in advancing our data capabilities and driving innovation. Job Responsibilities: Develop workflows and ELT data pipelines using Python, Spark/PySpark, and Databricks Onboard enterprise datasets into Palmos, including ingestion, transformation, and validation of data assets Build, test, and maintain scalable data pipelines and data … coding standards; contribute learnings and reusable patterns to improve broader team effectiveness Required Qualifications, Capabilities, and Skills: Hands-on experience with Databricks, Spark/PySpark, Python, and SQL Experience developing and maintaining data pipelines and data processing systems Understanding of the data lifecycle, including ingestion, transformation, storage, and consumption ...

DescriptionRole –Senior Databricks ArchitectTechnology – Databricks (Spark, PySpark/SQL) and data lake architecturesLocation – London, UKCompensation – Competitive (including bonus)Job DescriptionWe are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role focuses on integrating legacy … systems with cloud-native platforms to support analytics, reporting, and domain-owned data products.Primary Skills• Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures.• Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka.• Experience integrating legacy data sources such as DB2 (mainframe ...

Data Engineer £54-59,000 + bonus Hybrid Working – 1 day on-site per week West Midlands based I'm recruiting for a Data Engineer to join an established data team, working on the development ...

business questions and deliver custom data products seamlesslyThis is the perfect job for someone who has:At least 2 years experience with SQL/pySpark for data analysis - MustExperience in Python scripting - MustExcellent verbal and communication skills in English - MustAWS Infrastructure experience - Big AdvantageWorking with clients - Big Advantage.Proficiency ...

passion for realtime data, The Data Scientist must have: • Experience in building complex data pipelines/ETL/ELT scripts • Python/PySpark as our Data pipelines will be built in PySpark • SQL • Knowledge of AWS and components such as S3, Glue, Athena, RDS • Experience with Data Warehouse ...

platform using Microsoft Fabric, working within a wider Azure data environment. You'll design, build, and optimise scalable data pipelines using Fabric, Python and PySpark, while mentoring the existing BI team and helping shape best practice across the platform. Key Responsibilities Develop and maintain the enterprise data platform using … Lead Data Engineer who is passionate about building modern, scalable data platforms. You'll have strong hands-on experience with Microsoft Fabric, Python, PySpark and Azure, enjoy solving complex technical challenges, and thrive in a collaborative environment where you can make a real impact by setting best practice ...

technology sprawl while improving security, supportability and delivery speed. 4. Data Engineering and Integration Design and build reliable ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake. Implement full and incremental ingestion, CDC where appropriate, schema evolution, reconciliation, error handling and data quality controls. Design reusable integration … Databricks Serverless and compute/workload optimisation Azure identity, networking, security, secrets, monitoring and private connectivity Databricks SQL, Delta Lake and performance optimisation Python, PySpark and SQL Jobs/workflows, incremental processing, CDC and data quality REST/API and external data integration patterns FinOps, cost attribution, tagging, budgets ...

robust and standardised data platform experience.What you’ll be doingDesigning, building, and maintaining scalable data pipelines using Python and Scala, leveraging Spark and PySpark within an Azure ecosystem (including Azure Data Factory and Databricks).Developing and maintaining reusable data engineering templates, frameworks, and tooling to support data teams across … succeed.QualificationsStrong experience as a Data Engineer building scalable data platformsDeep expertise in Azure (ADF, ADLS, Databricks)Proficiency in Python and/or Scala(PySpark/Spark) for large-scale data processingHands-on experience with Databricks and Delta LakeSolid understanding of the end-to-end data lifecycle (ingestion transformation serving ...

Senior Data Scientist Welwyn Garden City, Hertfordshire or Slough, Berkshire | Hybrid (2 Days Office) | Python | SQL | PySpark | Azure | Databricks | Machine Learning | Statistics | Time Series Forecasting | Propensity Models Role: Senior Data Scientist, Lead Data Scientist, ML Scientist, AI Data Scientist, Statistical Data Scientist, Predictive Analytics Key Skills: Senior Data Scientist … Data Scientist, Machine Learning, Artificial Intelligence, Predictive Analytics, Statistical Modelling, Python, SQL, PySpark, Azure, Databricks, MLflow, GitHub Actions, CI/CD, Time Series Forecasting, Propensity Modelling, Feature Engineering, Data Engineering, Azure Machine Learning, Azure Data Platform, Experiment Design, A/B Testing, Hypothesis Testing, Predictive Modelling, Customer Analytics, Marketing ...

support migration planning, solution design and adoption. Data Engineering and Integration: Design, build and optimise scalable data ingestion and transformation solutions using Python, PySpark, SQL and Delta Lake. Implement batch and incremental processing patterns, including Change Data Capture (CDC), schema evolution, reconciliation processes, data quality controls and error handling. … Databricks SQL, Delta Lake and query/workload performance optimisation. Strong understanding of platform governance, operational support and best practices. Engineering, Delivery & Operations: Python, PySpark and SQL development expertise. Data engineering and pipeline delivery within enterprise-scale environments. Jobs, workflows, orchestration, incremental processing and CDC implementation. Data quality, reconciliation ...

closely with architects, senior engineers, and analysts to deliver scalable and reliable migration solutions, using technologies such as AWS Glue, Apache Iceberg, Python/PySpark, SQL, and YAML configurations. You should be comfortable working in a collaborative, delivery-focused environment and have a strong interest in data migration, cloud … pipelines on AWS services, including AWS Glue and Apache IcebergSupport validation of data pipeline migrations to AWS Data Lakehouse architecturesTest pipelines using:Python/PySpark transformationsSQL-based validation logicYAML-driven configurationsExecute and validate:Initial bulk data loadsIncremental/delta data processingWrite and run SQL queries to validate:Data completenessData ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

Responsibilities Design, develop and maintain modern data platform solutions. Build, optimise and manage data ingestion pipelines. Develop data transformation and modelling solutions using Python, PySpark and SQL. Support the implementation of data lakes, data warehouses and modern analytics architectures. Ensure high standards of data quality, security and performance. Work … date with emerging technologies. Requirements Previous experience working as a Data Engineer or within a similar role. Strong experience with Python or PySpark . Experience with cloud data platforms such as Microsoft Fabric, Databricks or Snowflake . Knowledge of SQL and data transformation techniques. Experience building and maintaining data ...

delivery. What You'll Be Doing Design, develop and maintain data pipelines and data engineering solutions using Databrick sBuild scalable data processing solutions using PySpark and SQL Work across Databricks Lakehouse environments, supporting implementation, optimisation and ongoing improvement Develop and optimise batch and data processing workflows Contribute to technical … transformation journeys What We're Looking For Strong commercial experience in Data Engineering Hands-on experience with Databricks and/or Apache Spark Strong PySpark and SQL development skills Experience designing, building and supporting data pipelines Experience working with Azure or AWS Good understanding of modern data platforms ...

Cyber Analytics team.You will need: A strong passion for big data, data modelling, data quality and ETLs. Solid programming experience with Python/PySpark along with proficiency in SQL. Extensive experience working with ETL and ELT frameworks and orchestration tools such as Airflow and dbt. Experience building robust data … Cyber Analytics team. You will need A strong passion for big data, data modelling, data quality and ETLs. Solid programming experience with Python/PySpark along with proficiency in SQL. Extensive experience working with ETL and ELT frameworks and orchestration tools such as Airflow and dbt. Experience building robust ...

Job Advertisement: Software Developer/Senior Data Engineer Location: London Contract Length: 12 Months Daily Rate: £700 (Umbrella) Are you ready to be part of a transformative journey? Our client is embarking on a major ...

Job TitleSenior Data Engineer EMEAJob Description SummaryAbout the teamWe are a Centre of Excellence within a larger EMEA team, with deep specialization in data engineering. Our mission is to build the platform that defines how ...

data transformationsTest framework and supporting toolingLead testing of AWS‐based Lakehouse and data pipeline solutions, including:AWS GlueApache IcebergAI‐enabled ETL acceleratorsNeo4jPython/PySpark, SQL and YAML‐driven configurationsQuality, Defect & Risk ManagementEnsure quality issues, risks issues and data defects are identified early, clearly documented and effectively triaged.Monitor, track … synthetic data creation, masking and environment readiness.What experience you'll bring:Extensive experience in data engineering/test management.Strong technical capability in: Python/PySpark, SQL, Oracle, HDFS and YAMLBPSS minimum, SC not mandated but strong likelihood may be requiredStrong experience in leading test automation in data testing, ideally ...

Ref491Vacancy typeITLocationLondonEmployment typeFull-timeClosing date19/08/2026DescriptionThe roleThe primary role of the Senior Data Analyst is to assist the firms Data Architect. The role can be based at any of our offices, although ...

required as well as a solid understanding of working in a cloud-based environment (AWS is preferred). Proficient using transformation tools such as PySpark/Pandas. Knowledge and practice of graph databases is preferred but not required. Key responsibilities/Skills and experience Familiarity using data quality frameworks ...

Senior Databricks Architect - London (Hybrid) - Databricks - Spark - PySpark - AWS - Up to £115,000 + Bonus Our client, a large-scale organisation undergoing significant investment in data and cloud technologies, is seeking an experienced Senior Databricks Architect to join its expanding Data & AI practice. This is an exciting opportunity … time data pipelines from legacy and modern source systems Implementing real-time and event-driven data processing solutions Developing data transformation frameworks using Spark, PySpark and SQL Designing Lakehouse architectures using Databricks and Medallion Architecture principles Creating scalable, reusable data products to support modern data operating models Integrating data ...

star schema and other best practices to optimise data usability.Sets best practices and standards on how we use SQL & dbt/Python/Pyspark and/or other languages to solve business problems.Becomes an expert in the business subject area the team are working on, holding expertise on both … team click. We’re a group that chooses to lift each other up and think smart every day.Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship ...