1 to 25 of 128 Remote PySpark Jobs

Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes of Python, Pyspark, SQL, Scala or Java. Experience operating in a cloud-native environment such as Azure, AWS, GCP etc ( Fabric experience would be beneficial ...

Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes of Python, Pyspark, SQL, Scala or Java. Experience operating in a cloud-native environment such as Azure, AWS, GCP etc ( Fabric experience would be beneficial ...

Essential Skills for the AWS Data Engineer: Extensive hands-on experience with AWS data services Strong programming skills in Python (including libraries such as PySpark or Pandas) Solid understanding of data modelling, warehousing and architecture design within cloud environments Experience building and managing ETL/ELT workflows and data ...

Azure Storage. Excellent SQL and data modelling (star/snowflake, dimensional modelling). Knowledge of Power BI dataflows, DAX, and RLS. Experience with Python, PySpark, or T-SQL for transformations. Understanding of CI/CD and DevOps (Git, YAML pipelines). Strong grasp of data governance, security, and performance ...

Azure Storage. Excellent SQL and data modelling (star/snowflake, dimensional modelling). Knowledge of Power BI dataflows, DAX, and RLS. Experience with Python, PySpark, or T-SQL for transformations. Understanding of CI/CD and DevOps (Git, YAML pipelines). Strong grasp of data governance, security, and performance ...

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

ideally in enterprise-scale environments Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake Hands-on experience with Spark (PySpark/SparkSQL) and distributed data processing Strong data modelling skills (Dimensional, ODS, Data Vault) and data warehousing concepts Proficiency in SQL, version control ...

data sources into Azure Databricks. Optimize pipelines for performance, reliability, and cost, incorporating data quality checks. Develop complex transformations and processing logic using Spark (PySpark/Scala) for cleaning, enrichment, and aggregation, ensuring accuracy and consistency across the data lifecycle. Work extensively with Unity Catalog, Delta Lake, Spark ...

Experience as a Senior Data Engineer or similar Strong knowledge of Azure Cloud architecture and Azure Databricks, DevOps and CI/CD. Experience with PySpark, Python, SQL and other data engineering development tools. Experience with metadata driven pipelines and SQL serverless data warehouses. Knowledge of querying API systems. Experience ...

Experience as a Senior Data Engineer or similar Strong knowledge of Azure Cloud architecture and Azure Databricks, DevOps and CI/CD. Experience with PySpark, Python, SQL and other data engineering development tools. Experience with metadata driven pipelines and SQL serverless data warehouses. Knowledge of querying API systems. Experience ...

pipelines to ingest, transform, and load data from diverse sources (APIs, databases, files) into Azure Databricks. Implement data cleaning, validation, and enrichment using Spark (PySpark/Scala) and related tools to ensure quality and consistency. Utilize Unity Catalog, Delta Lake, Spark SQL, and best practices for Databricks development, optimization ...

Fabric. Migrate on-prem data to Fabric Lakehouse/Warehouse, enabling Power BI analytics. Develop robust data pipelines, notebooks, and data models using: Python, PySpark, SQL, Synapse, and Data Warehousing principles. Implement CI/CD processes with GitHub integration for lifecycle management. Ensure governance, security, and best practices across ...

patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. Azure ...

patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. Azure ...

demand delivery Provide technical guidance for team members Providing 2nd or 3rd level technical support About You Experience using SQL, SQL Server DB, Python & PySpark Experience using Azure Data Factory Experience using Data Bricks and Cloudsmith Data Warehousing Experience Project Management Experience The ability to interact with the operational ...

master's degree. 2-3 years in data engineering or related field. Basic proficiency in SQL and Python. Familiarity with data processing frameworks (e.g., PySpark) is a plus. Knowledge and experience with Databricks and/or Snowflake. Exposure to Azure or other cloud platforms. Ability to understand data requirements ...

data models and analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear ...

Methodology or other similar dimensional modelling standards, is essential to the role. Technical experience building and deploying models and reports utilizing the following tools: PySpark Microsoft Fabric or Databricks Power BI Git CI/CD pipelines (Azure DevOps experience preferred) An understanding of the structure and purpose ...

manipulate data and building additional process to support data processing activities Experience manipulating, processing and storing data using tools such as SQL, dbt, pyspark and SSIS Comfortable building out and administrating data models, ensuring that they are accessible and used appropriately, leveraging platforms such as Databricks and Power ...

grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with Azure and Data Bricks You're excited ...

grades at A-level You have commercial Data Engineering experience working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You have a good understanding of modern data engineering best practices Ideally you will also have experience with Azure and Data Bricks You're excited ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...