1 to 25 of 43 Remote/Hybrid PySpark Jobs in the UK

tooling Key skills Strong experience with Microsoft Fabric (Data Factory, Lakehouse, OneLake, Pipelines, SQL Warehouse) Solid data engineering background using SQL, Python, PySpark, Delta Lake E xperience building and optimising end-to-end data pipelines CI/CD experience for analytics and data platforms (Azure DevOps or GitHub) Exposure ...

pipelines and working with lakehouse architectures Knowledge of Azure tools such as Data Factory and/or Fabric Data Pipelines Experience with Python or PySpark for data transformation Understanding of data modelling (dimensional modelling, medallion architecture) Strong communication skills and ability to work with non-technical stakeholders Desirable: Experience ...

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

from different sources. Good communication and teamwork skills. Experience with cloud data platforms such as Databricks or Snowflake on AWS. Strong skills in Python, PySpark, and SQL. Good understanding of ETL/ELT pipelines and data transformation. Experience with data modelling and database optimisation. Strong knowledge ...

Advanced SQL skills for complex query validation. Experience with Agile/DevOps delivery models and JIRA test management. Familiarity with automation frameworks (SQL/PySpark preferred). Knowledge of CI/CD and release management processes. Experience with Power BI, Alteryx, and VS Code , particularly in testing integrations with ...

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

Databricks Defining data models and Databricks architecture standards Establishing data governance and security frameworks Aligning data architecture with enterprise cloud strategy Supporting development using PySpark, SQL and Python within Databricks Working within CI/CD and DevOps environments (Git-based) About You: Strong technical background in Databricks and Azure ...

query performance Strong analytical and problem-solving skills Desirable Skills Knowledge of modern data architectures Experience with SQL, SSRS, SSIS and SSAS Experience using PySpark Familiarity with KQL Experience with data cataloguing tools Exposure to C# What you'll get in return Competitive Salary - 52,750 with a generous ...

Builder or Code Repositories. Experience with Delta Lake, Parquet, Iceberg, CSV and JSON. Data governance, lineage, secure data-sharing and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Configured and validated Foundry Databricks connector. ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena ...

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...

Palantir Foundry experience, especially Databricks connector, Data Connection, datasets, syncs, projects and permissions. Secure data-sharing, data governance, lineage and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Secure AWS S3 and Databricks access ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

genomic data. You have A BSc or MSc in computer science or related fields Expertise in Python, including popular Python libraries: NumPy, Pandas, PySpark and frameworks: Django, Django Rest Framework, FastAPI Hands-on experience with both relational (e.g. PostgreSQL) and non-relational databases (e.g. Elasticsearch, Redis) Extensive experience ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

complex technical architectures. Essential Skills Clearance: Active UK Government Security Check (SC). GenAI: Expertise in LLMs and RAG architectures. Python: Strong proficiency alongside PySpark for data engineering. Azure: Deep familiarity with native tools (AI Search, Document Intelligence). MLOps: Understanding of LLMOps frameworks for production deployment. 6 Months ...

Technical Environment : Palantir Foundry, Foundry Virtual Tables, Ontology, Workshop, Pipeline Builder, Code Repositories, AWS, Databricks, Terraform, IaC, Delta Lake, Parquet, Iceberg, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...

Technical Environment : Palantir Foundry, Foundry Virtual Tables, Ontology, Workshop, Pipeline Builder, Code Repositories, AWS, Databricks, Terraform, IaC, Delta Lake, Parquet, Iceberg, CSV, JSON, Python, PySpark, SQL, CI/CD, GitLab, GitHub Actions. ...