26 to 50 of 104 Permanent PySpark Jobs

data modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … influence within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena ...

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...

performance, cost efficiency, and operational stability Advanced programming skills in Python and SQL, including working with large, distributed datasets (experience with Spark/PySpark is a strong advantage) Hands-on experience with Infrastructure as Code and CI/CD in cloud environments (CDK, Terraform, or CloudFormation; automated deployments ...

Doing Designing and delivering end‐to‐end data and analytics solutions across complex enterprise environments Building data pipelines and workflows using Python, SQL, PySpark, and low‐/no‐code tooling Supporting the development of user‐facing applications and dashboards (TypeScript/React exposure beneficial) Working closely with stakeholders ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality, governance and observability practices. Modernise legacy systems and support large-scale migrations into Azure. Provide technical leadership … Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

covered) Salary: £45,000 – £95,000 - dependent on level/experience (mid, senior, principle) Security Clearance: SC active Requirements: Databricks and Spark/PySpark (absolute must haves) Azure and AWS experience Data warehouses/lakehouse platforms CI/CD and modern engineering practices Public sector/Central government ...

landscapes and building workflows that support better decision making. The role is end to end. That means data engineering in Palantir Foundry, Python and PySpark, through to user-facing applications in TypeScript or React, and embedding AI workflows using AIP Logic and AIP Analysts. You will have ownership over ...

Data Engineer; Python, PySpark - Salford, Manchester, Onsite. We are seeking an experienced Data Engineer to join a growing Data Team within a fast-paced, e-commerce-led business that is growing at an astonishing rate. The company operates in the direct-to-consumer retail space, and the role will ...

into scalable technical solutions Building end-to-end data workflows using a mix of low/no-code platforms and code-based approaches (Python, PySpark, SQL) Developing user-facing applications, dashboards, and workflows using modern front-end tools such as TypeScript and React Supporting the design and evolution ...

technical outputs into clear commercial and operational insight • Delivering within agile, cross-functional teams on high-impact programmes Tech environment Python, SQL, R, Databricks, PySpark, GIS and geospatial tooling, optimisation frameworks, cloud platforms Looking for • Degree or higher in Data Science, Mathematics, Operational Research, Physics, Engineering, Geography or similar ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards … reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

engineering within distributed systems Proven ability to build and optimise data pipelines at scale Solid Python skills with exposure to tools such as Spark, PySpark, and Hadoop Strong understanding of data ingestion and batch processing methodologies Experience with text or unstructured data is advantageous but not essential A hands ...

other MLOps and ML Lifecycle stacks Experience with Docker and Kubernetes Experience in creating production grade coding Experience in programming languages (eg Python, PySpark, R, SAS, SQL) Experience in GitHub Strong communication skills both verbally and written ...

TensorFlow Serving, TorchServe). Familiarity with ML lifecycle tools such as MLflow, Kubeflow, or Azure ML Pipelines. Experience working with distributed data processing (e.g., PySpark) and SQL. Understanding of software engineering best practices, including version control (Git). Knowledge of CI/CD principles in ML environments. Experience with ...

joins, transformations, stored procedures, and performance basics 3+ years Azure Data Factory, including pipeline development, parameter usage, and basic templates 3+ years Databricks experience (PySpark, Delta Lake, notebooks) 2+ years Python Experience Experience working with data warehouses and structured modelling concepts Strong understanding of ETL/ELT practices … data centric engineering Desirable Exposure to metadata driven or config driven engineering practices Basic PySpark optimisation or Delta Lake performance tuning Experience using Azure DevOps (repos, pipelines) Understanding of data governance, data quality tooling, or lineage concepts Experience in regulated environments or financial services The Application Timeline A first ...

support reporting, analytics, and business operations. This includes developing ETL/ELT workflows using both low-code tools and code-based approaches (SQL, Python, PySpark), ensuring performance, scalability, and reliability across the data platform. You will take ownership of data modelling and architecture , designing scalable data warehouse and lakehouse … understanding of data modelling (star schema, dimensional modelling, lakehouse design) Experience building and optimising ETL/ELT pipelines using SQL, Python, and/or PySpark Knowledge of data governance, quality, and security best practices Experience implementing CI/CD pipelines and DevOps practices Ability to work independently and manage ...

Architecture . Build and optimize batch and real-time data pipelines across underwriting, pricing, claims, reinsurance, and bordereaux data. Develop scalable solutions using Python, PySpark, Databricks, and/or Snowflake . Integrate data from policy administration systems, claims platforms, broker systems, third-party providers, and market feeds. Ensure strong … background in Insurance , ideally within the London Market ecosystem. Deep knowledge of underwriting, claims, delegated authority, pricing, and reinsurance. Hands-on expertise in Python, PySpark, Databricks, Snowflake, SQL . Experience with Azure, AWS, or GCP . Strong understanding of batch/streaming pipelines and data modelling. Familiarity with DevOps ...