1 to 25 of 30 Remote/Hybrid Permanent PySpark Jobs

Engineer Hands-on experience with Microsoft Fabric (essential) Experience working within Azure data environments Strong SQL development skills Experience with Python and/or PySpark Knowledge of Azure Synapse Analytics and Azure Data Factory Experience building and maintaining data pipelines and data integration solutions Understanding of modern data warehousing ...

Engineer Hands-on experience with Microsoft Fabric (essential) Experience working within Azure data environments Strong SQL development skills Experience with Python and/or PySpark Knowledge of Azure Synapse Analytics and Azure Data Factory Experience building and maintaining data pipelines and data integration solutions Understanding of modern data warehousing ...

ideally in enterprise-scale environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. ...

scale data platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured, semi-structured and unstructured data ...

such as Claude Code or equivalent tools, to improve engineering productivity while maintaining high standards of quality, governance and security. Desirable experience Regulated environment. PySpark for large-scale data transformations. Migrating legacy data estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven systems. Data-quality ...

Experience of core MS Fabric features including: OneLake, Lakehouses, Warehouses, Data Pipelines, Dataflows, Gen2, Notebooks, Eventstreams + Eventhouses. Using SQL, T-SQL, Spark SQL, PySpark + Python for data ingestion, data transformation, data optimisation, data modelling, and converting structured/unstructured source data into validatd Delta tables. Using Power ...

accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls. Desirable experience Financial services or another regulated environment. PySpark for large-scale batch or streaming transformations. Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures. Customer Data Platforms or event-driven ...

architecture Experience with cloud data platforms (AWS preferred - Redshift, Athena, S3, Glue. Azure accepted) Understanding of workflow orchestration and automation tools Python experience (including PySpark) is beneficial but not essential Exposure to BI platforms such as Power BI, Tableau or QuickSight, with an understanding of how backend data models ...

Experience with audit, balance, and control frameworks - parameterized, modular pipeline governance at enterprise scale Familiarity with config-driven and automation-first pipeline patterns (YAML, PySpark, SQL-driven generation from mapping documents) Databricks & Lakehouse Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred) Strong Apache Spark skills … PySpark/Spark SQL) Experience migrating workloads from legacy data warehouse or Synapse environments to a Databricks Lakehouse Ability to re-implement governance and control frameworks natively in Databricks (audit logging, reconciliation, DQ checks) Experience with Delta Lake features: MERGE, CDC, time travel, schema enforcement Data Engineering & Development Strong ...

patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data ...

access trusted data at scale. Job description Roles and Responsibilities: Lead the team in designing, developing andmaintainingscalable ETL/ELT pipelinesusingastrong knowledge of PySpark and Python to ensure best practice and standards are met in optimizing,validatingand standardising all team data workflows as well as batch processes. Lead collection ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

Complex SQL Databricks/Data Lake Power BI & Data Visualization Data Quality, Validation & Reconciliation Dataset Development & Maintenance LeanIX/Enterprise Architecture Data Governance & Troubleshooting PySpark preferred Agile/SDLC & Production Run Support Must Have: Strong SQL, Databricks/Data Lake, Power BI, data validation, and data quality experience. LeanIX ...

growth opportunities and helping expand a consulting business. A naturally curious mindset and a genuine passion for solving challenging problems. **Additional experience with Databricks, PySpark, R, Power BI or Palantir Foundry would be advantageous.** Why consider it? Continued development:** Dedicated training and certification budget. Variety:** Work on complex, interesting ...

unexpected backgrounds. Desirable Experience Any exposure to Palantir Foundry or AIP is a bonus, but not required - this is where you will build it PySpark or experience with large-scale data processing Experience with ERP, supply chain, finance or commercial data in an industrial setting Consultancy or startup experience ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

query, validate and optimise data across cloud-based analytical platforms such as Athena, Redshift and Azure Synapse Build scalable data processing solutions using PySpark, Python and cloud-native orchestration services Implement data quality checks, validation processes, monitoring and logging to ensure trusted and reliable data delivery Support the performance … engineering, integration and analytics challenges Collaborate with experienced engineers, architects and data specialists in supportive, multidisciplinary teams Develop expertise across cloud-native data services, PySpark, SQL and modern engineering practices Access mentoring, knowledge sharing and development opportunities to help you grow your career Contribute to solutions that deliver measurable ...

ensure reporting is accurate and useful, and improve engineering standards while supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures … documentation, Git and CI/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with ...

data science, statistical modelling and using data to solve problems. Some experience with programming languages or analytical tools, such as Python, SQL, SAS or PySpark, gained through university projects, placements, internships or employment. An understanding of statistical concepts and an interest in predictive modelling or machine learning. Strong analytical ...

2026. A chance to work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve … understanding of role-based access, KMS encryption + diagnosing data access failures. Owning incidents from investigation through recovery and closure. Diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations + providing clear progress updates and timely escalation. Working knowledge of Git, code reviews, CI/ ...

personal technical development. To ensure data dictionaries and models are robust and maintained on a regular basis. Technical Skill Expert-level Databricks experience Advanced PySpark development Delta Lake architecture and optimisation Databricks SQL Data warehouse Dimensional modelling Data Lake Databricks Genie Python development Key Skills, Experience and Knowledge Extensive … within enterprise-scale data environments. Extensive hands-on experience delivering solutions using Databricks. Extensive experience designing and implementing Lakehouse architectures. Strong commercial experience using PySpark, Python, SQL and Delta Lake. Experience optimising Databricks workloads for performance, scalability and cost efficiency Previous experience delivering solutions within insurance, financial services ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...

Strong knowledge of migrating and transforming data from legacy and on-premise systems to cloud platforms. Expertise in automating ETL/ELT pipelines using PySpark and Spark SQL across Azure-based data environments. You and your role As a Data Engineer, you will design build and enhance scalable …/ELT pipelines using Python and PySpark, ensuring data solutions are reliable, efficient and aligned with engineering best practice. You'll work closely with stakeholders to understand complex business requirements, transforming data into trusted, high-quality datasets that enable valuable insights, reporting and decision-making. You will manage ...