1 to 25 of 30 PySpark Jobs in Manchester

enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform implementations Proven ability ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

tools and approaches. Hands-on experience with Microsoft Azure technologies, including MS Fabric, Data Factory, Azure Synapse & RDS. Experience with Python or PySpark for scripting and automating data tasks. Knowledge of CI/CD approaches for Data Platforms Knowledge of MS SQL Server, SSIS, Visual Studio, and SSDT projects. ...

API Gateway, and CloudWatch. Design, develop, and maintain data pipelines and ETL workflows using technologies such as Glue, Spark/PySpark, Kafka, or similar platforms. Implement and maintain Infrastructure as Code (IaC) using Terraform or equivalent tools. Drive observability, monitoring, logging, and operational readiness across software platforms and services. ...

Databricks Serverless configuration and workload optimisation* Databricks SQL and Delta Lake development* Cluster management, compute policies, autoscaling, and workload monitoring**Data Engineering*** Strong Python, PySpark, and SQL development skills* Design and build ingestion and transformation pipelines* Full and incremental data loading* CDC (Change Data Capture) implementation* Data quality controls ...

responsibilities. Ensure solutions are secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines … duties as required. Skills and Experience Strong understanding of data engineering principles, data architecture, and cloud-based data platforms. Proficient in Python (PySpark) and SQL for building and optimising data transformations at scale. Strong hands-on Databricks experience. Experience with Azure data storage technologies (Azure SQL, Data Lakes, Blob ...

knowledge of Matillion’s Data Productivity Cloud/Matillion ETL, or primary competitive data integration platforms. Big Data & Analytics Ecosystems: Experience with Spark/PySpark, modern Data Lakes/Lakehouses, and MPP database architectures (Teradata, Vertica, Netezza). Education: Bachelor’s or Master’s degree in Computer Science, Data ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query ...

experience with Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added ...

data ecosystem. You will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J ...

scalable applications. You will own data analysis, system design, and the full software lifecycle from requirements gathering to deployment. The role emphasizes strong Python, PySpark, SQL skills, and experience with ETL, data modeling, and regulatory reporting flows. Collaboration with the Axiom team is also key. #J-18808-Ljbffr ...

ideas flowing, typically once a quarter. Prefer being in the office more often? That works for us too. If you’ve strong experience with PySpark, Python and SQL, we’d love to hear from you. Core skills we’re looking for to succeed in the role: Cloud & Data Engineering … bring expertise in modern cloud platforms, delivering scalable and robust data solutions, with engineering and database skills in PySpark, Python, SQL Server and/or Azure (Data Factory, Synapse) and the associated ETL technologies. Problem‐Solving & Innovation: You’ll thrive on tackling complex challenges, finding creative solutions, and approaching ...

About the Opportunity Sun Chemical is seeking an experienced Data Architect II to join our Corporate IT team and drive the modernization of our enterprise data ecosystem. This is a unique opportunity to architect cutting ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

performance optimisation. Hands-on delivery experience within hospitality reporting and commercial reporting environments. Experience working with Databricks is useful. Ability to interpret Python/PySpark is beneficial. If you are interested in this Power BI Developer role please apply with your most recent CV or reach out on jordan ...

Function Data Analytics/Warehousing & Business Intelligence Experience 5-10 years Location United Kingdom - Edinburgh, London, Manchester Skills Python & SQL: Advanced Python (PySpark GlobalLogic is seeking a Data & AI Architect working in our UK & Ireland Go to Market Team to drive pipeline growth and lead enterprise transformation pursuits across … mindset, with experience working across geographies including India, East Europe and North America. Proven track record of acceleration of Business growth Deep expertise in PySpark/Python, Advanced SQL, data modeling, batch/real-time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge ...

Azure Data Engineer - 1/2 days onsite 18 month FTC Summary: Join a team building a modern Azure-based data platform. This hands-on engineering role involves designing and developing scalable, automated data pipelines ...

bonus and an extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise … help mature a growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ ...

interact with stakeholders, and mentor junior colleagues while maintaining governance. The role requires a quantitative degree and knowledge of insurance pricing concepts; proficiency in PySpark/SQL/Python/R is advantageous. #J-18808-Ljbffr ...

team About you: Experience within general insurance preferred Familiarity with retail pricing concepts Experience in statistical and data science programming languages (e.g. PySpark, SQL, Python, R) A good quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar software is preferred, but not essential ...

best practices and CI/CD within an agile, hybrid setup that supports collaboration across Haywards Heath, Salford Quays, and Guernsey. You’ll use PySpark, Python, SQL, and Azure-based tools to implement end-to-end data solutions, mentor junior engineers, and #J-18808-Ljbffr ...

Snowflake – Warehouses, Streams/Tasks, Snowpipe, Snowpark, UDFs/Stored Procedures, Time Travel and Cloning. Snowpark and end‐to‐end pipeline development. Experience with PySpark, Polars or Dask is also valuable. Python, including testing, build tooling and engineering best practices. SQL, including Snowflake‐optimised SQL. dbt, including Jinja templating … champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great opportunity to join a collaborative ...