26 to 50 of 60 Remote/Hybrid PySpark Jobs

ideally in a multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform ...

Working closely with data engineering and platform teams • Translating complex outputs into clear, actionable insight • Agile, cross-functional delivery Tech Python, SQL, R, Databricks, PySpark, GIS tools, optimisation frameworks, cloud platforms Requirements Degree or higher in Data Science, Mathematics, Operational Research, Engineering, Physics, Geography, or similar Experience working with ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

ideally within Retail, QSR, or Consumer Services Strong academic background in statistics, mathematics, computer science, engineering, or equivalent practical experience Advanced programming skills (e.g. PySpark, SQL, Python) with the ability to develop and deploy production-level models and code. Experience with the Azure ecosystem (e.g. Azure Databricks, Azure DevOps ...

mentoring junior engineers and promoting engineering best practices Skills & Experience: Understanding of data processing using Apache Spark Use of Python, SQL, and familiarity with PySpark Experience using Apache Airflow for task orchestration Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena ...

modelling techniques Understanding of metadata management, data governance, and data quality frameworks Proficiency with SQL and ideally one or more programming languages (e.g., Python, PySpark) Experience with database technologies and data warehousing principles (star schema, snowflake, data vault) Experience working with cloud data services specifically with AWS (including Redshift ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

notebooks) Experience building or working within modern data warehouse architectures Ability to get up and running quickly without heavy onboarding Nice to have: Python, PySpark or Spark experience Exposure to APIs and external data integration Experience working in evolving or transforming data environments Most importantly, youll be someone ...

notebooks) Experience building or working within modern data warehouse architectures Ability to get up and running quickly without heavy onboarding Nice to have: Python, PySpark or Spark experience Exposure to APIs and external data integration Experience working in evolving or transforming data environments Most importantly, youll be someone ...

into scalable technical solutions Building end-to-end data workflows using a mix of low/no-code platforms and code-based approaches (Python, PySpark, SQL) Developing user-facing applications, dashboards, and workflows using modern front-end tools such as TypeScript and React Supporting the design and evolution ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

Strong experience designing modern cloud-based data platforms. Hands-on architectural experience with: AWS (essential): S3, EMR, Glue Kafka/event streaming architectures Python & PySpark-based data processing Experience designing data ingestion pipelines (batch and real-time). Proficiency in Infrastructure as Code (Terraform). Experience with GitHub-based ...

Technology Environment You will be working across a modern Azure data stack, including: Azure Databricks Azure Data Factory (ADF) Synapse/Fabric Snowflake Python & PySpark SQL DevOps & CI/CD tooling within Azure Experience Required: Strong experience in modern data engineering within trading or financial services (energy/commodities … experience highly desirable) Deep hands-on experience with Azure Databricks Strong Python programming capability (including PySpark) Advanced SQL skills Experience building and orchestrating scalable data pipelines Strong understanding of data modelling and architecture principles Familiarity integrating data platforms with Power BI and/or Tableau Appreciation for data governance ...

outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

engineering within distributed systems Proven ability to build and optimise data pipelines at scale Solid Python skills with exposure to tools such as Spark, PySpark, and Hadoop Strong understanding of data ingestion and batch processing methodologies Experience with text or unstructured data is advantageous but not essential A hands ...

direction, maintaining standards, and driving output Designing and delivering data solutions across platforms, including low/no-code tooling and code-based approaches (Python, PySpark, SQL) Overseeing the development of user-facing applications and workflows using TypeScript and React Acting as the primary client interface, shaping requirements and translating ...

Azure or AWS Developing modern lakehouse architectures using: Databricks (Spark, SQL, Delta Lake) Azure Data Lake Storage or Amazon S3 Writing high-quality PySpark, Python, and SQL for ingestion, transformation, and modelling Implementing Bronze/Silver/Gold data layers Embedding data quality, validation and monitoring within pipelines Supporting … Data Engineering roles Strong experience working with Azure or AWS cloud platforms Commercial, hands-on experience using Databricks in production Proven capability across: PySpark/Python/SQL Cloud data storage (ADLS or S3) ETL/ELT pipeline development Solid understanding of data architecture, modelling, and governance Experience building ...

national scale. You must already hold active SC clearance to be considered. Responsibilities Design, build and maintain Databricks data pipelines using Apache Spark (PySpark & Spark SQL) Implement Delta Lake/Lakehouse architectures (Bronze, Silver, Gold) Develop scalable ELT/ETL workflows for batch and streaming data Optimise performance, reliability … delivery environment Contribute to CI/CD pipelines and automated testing practices Essential Skills & Experience Proven commercial experience as a Databricks Data Engineer Strong PySpark and SQL capability Hands-on delivery within AWS, Azure or GCP Experience with Delta Lake , data lakes and data warehouses Strong understanding of data ...

frameworks Engineering Delivery Design and evolve scalable data pipelines (batch and incremental watermark-based ingestion) using: Microsoft Fabric Pipelines Azure Data Factory PySpark/Python Fabric Notebooks Implement and maintain a medallion lakehouse architecture using OneLake and Fabric Warehouses Ensure Direct Lake-ready semantic models for analytics Lead incident … Hands-on experience with Microsoft Fabric, including: Workspaces, OneLake, Fabric Warehouses, Fabric Notebooks and Data Pipelines and Direct Lake semantic models Strong Python and PySpark development capability Solid SQL skills for warehouse development and metadata layers Proven ownership of production data systems end-to-end Wider Azure data platform ...

GBMs, GAMs, Random Forests, Neural Networks and clustering techniques Knowledge of statistics and distributions commonly used in insurance Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Proficient at communicating results in a concise manner both verbally and written Behaviours: Motivated by technical excellence Driven to deliver iterative ...

ownership of data pipelines and automation of data workflows. Finally have 5 years Strong programming experience at an advanced level in Python, PySpark, and SQL scripting. Benefits Salary: 50,000 - 65,000 per annum 25 day's holiday Company car/Allowance Variable annual bonus based 5-15% Pension … leveraging tools such as Foundry Code Workbooks, Pipelines, and Object Explorer for efficient data processing and integration. Strong coding skills in Python (Pandas) and PySpark: Essential experience in using Python (Pandas) and PySpark for data manipulation, transformation, and pipeline development within Palantir Foundry, ensuring high performance and scalability ...

Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER – ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion Strong understanding … submit (subject to required skills) your application to our client in conjunction with this vacancy only. KEY SKILLS Data Engineering, Databricks, Apache Spark, PySpark, SQL, Python, Kafka, AWS, Delta Lake, ETL, Data Pipelines, CI/CD, Big Data ...

maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...

maintain scalable ETL/ELT data pipelines within Microsoft Fabric Lakehouse environments Develop and optimise data ingestion/transformation workflows Develop Python and PySpark processes for data transformation and large-scale processing Support the development and optimisation of Power BI datasets, reports and dashboards Build and maintain scalable semantic … data engineering solutions Skills and Experience Hands-on expertise with Microsoft Fabric, Lakehouse architecture and Data Pipelines Experience developing Python or PySpark solutions for large-scale data processing Understanding of ETL/ELT design patterns, data modelling and modern data platform architecture Experience developing Power BI datasets, semantic models ...