26 to 50 of 55 PySpark Jobs in the North of England

scalable applications. You will own data analysis, system design, and the full software lifecycle from requirements gathering to deployment. The role emphasizes strong Python, PySpark, SQL skills, and experience with ETL, data modeling, and regulatory reporting flows. Collaboration with the Axiom team is also key. #J-18808-Ljbffr ...

Partners is recruiting a hands-on Senior Data Engineer in the UK to advance a modern data capability built on Python/PySpark, Fabric, and Azure. You will contribute to lakehouse/medallion style designs, with a focus on code-led development and AI-ready data solutions. ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

Private equity-backed organisation with ambitious growth plans Essential Skills & Experience Strong experience as a Data Engineer within modern cloud data environments Expertise in PySpark, SQL and data transformation practices Experience building production-grade data pipelines Knowledge of Microsoft Fabric Experience working with Delta Lake and Lakehouse architectures ...

About the Opportunity Sun Chemical is seeking an experienced Data Architect II to join our Corporate IT team and drive the modernization of our enterprise data ecosystem. This is a unique opportunity to architect cutting ...

Data Engineer in the United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACTClient Advisory & Translation: Sit directly alongside clients, distinguish their goals ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark ark code repos - to building user-facing front-end applications and widgets in TypeScript or REACTClient Advisory & Translation: Sit directly alongside clients, distinguish their ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

teamWhat could set you apartExperience in the use of credit bureau dataExperience using Google Cloud Platform and/or Google Cloud Engineer qualificationsExperience of PySpark, BigQuery, Git and any/all of: DataFlow, DataProc, Airflow (or equivalents)We believe great things happen when teams connect. Our schedule is built ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

Hybrid, 3 days onsite). This 12-month fixed-term contract role focuses on building and supporting modern cloud-based data platforms using AWS, PySpark, and IaC. You will work with Terraform, GitLab CI/CD, and collaborate with engineers to deliver secure, scalable cloud solutions. You’ll join ...

plus 10% bonus and an extensive benefits package. What you'll be doingDesign, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake.Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems.Shape enterprise data … engineer and help mature a growing data function.What you need to bring“Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc)Advanced SQL and Python coding skills.A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

using both Microsoft Fabric tools and the Databricks Lakehouse Platform to deliver clean, production-ready data for enterprise analytics. Role Requirements Strong proficiency in PySpark, Databricks SQL, T-SQL, and handling Delta Lake formats Hands-on experience building production pipelines inside Databricks workspaces and Fabric capacities Practical knowledge … data processing Build and maintain the Medallion Architecture silver & gold layer across shared storage Write optimised transformations in Fabric Notebooks and Databricks clusters using PySpark and SQL Configure data access, masking, and governance policies within Databricks Unity Catalog. Optimise Lakehouse table performance using Z-Order indexing, liquid clustering ...

including model registry Unity Catalog and AI Gateway (or equivalent governance tooling) Strong general Databricks knowledge – compute management, logging and monitoring Strong Python and PySpark Excellent consulting skills – communication, people leadership and technical advisory, with the confidence to lead a client's team through a technical journey Desirable Machine ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

/ETL workflows within a Fabric Lakehouse environment Work with OneLake, Lakehouse architecture, and modern data engineering best practices Utilise PySpark, SQL and Databricks for complex data transformation and engineering workloads Create robust data ingestion, orchestration and monitoring processes Support the development of enterprise-wide analytics solutions and Power … with Microsoft Fabric, including Data Factory, Dataflows, Notebooks and Lakehouse solutions Experience building and supporting production-grade ELT/ETL pipelines Strong skills in PySpark and SQL Experience working with OneLake, Delta formats or modern Lakehouse architectures Exposure to Databricks and distributed data processing environments Understanding of data modelling ...

Location:West Yorkshire- multiple locations(Hybrid- 3 days working from onsite-Mandatory)Immediate startPermanent/Contract inside IR35/Fixed term contractWe’re looking for a Lead Data Engineer to join UST Data Team. As ...

Engineer role. They aren't looking for someone who wants to move away from development into pure management. You'll work across: Python/PySpark Microsoft Fabric Spark and notebook engineering Lakehouse/Medallion architecture Delta SQL Azure Git/CI/CD Automated testing Data quality and observability … Reusable engineering patterns They're particularly interested in people coming from Databricks/PySpark environments, so extensive Fabric experience isn't essential. At Senior level, you'll also have the opportunity to influence architecture, technical design, coding standards, CI/CD, testing and engineering best practice, while supporting other ...

modern data platform. Develop and maintain Medallion Architecture data layers, ensuring data is available for reporting and analytics. Create high-performance data transformations using PySpark, Databricks and Fabric Notebooks. Implement data governance and access controls through Databricks Unity Catalog. Support CI/CD deployment processes using Git integration … within a Data Engineering environment, delivering reliable and scalable data solutions. Strong hands-on experience with Microsoft Fabric and Databricks. Strong programming capability across PySpark and T-SQL. Experience working with Delta Lake, Lakehouse architectures and modern cloud data platforms. Understanding of data quality frameworks, testing approaches and governance ...

Data Engineer - Python/PySpark/Microsoft Fabric £55,000 - £65,000 + Benefits St Helens - 1-2 days per week This is an opportunity to join an organisation at a particularly interesting point in the development of its data capability. The business has invested heavily in modernising … procedures and drag-and-drop development. The team is deliberately moving towards a more code-led engineering approach, with a strong emphasis on Python, PySpark and notebook-based development. What you'll be working with The environment includes: Python/PySpark Microsoft Fabric Spark and notebook development Lakehouse ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

based data platforms. This is an excellent opportunity for an early to mid-career cloud or data engineer looking to develop expertise in AWS, PySpark, platform engineering, and Lakehouse technologies within a collaborative and technology-driven environment. The successful candidate will work closely with senior engineers to build, test … cloud data solutions and engineering best practices. Key Responsibilities Assist in the development, configuration, and testing of Lakehouse Core Platform components. Implement and maintain PySpark-based data transformation pipelines. Contribute to shared platform libraries and reusable engineering components. Support deployment and automation activities using Infrastructure as Code practices. Follow ...