1 to 25 of 85 Remote PySpark Jobs

Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes of Python, Pyspark, SQL, Scala or Java. Experience operating in a cloud-native environment such as Azure, AWS, GCP etc ( Fabric experience would be beneficial ...

Kafka, Databricks, DBT or similar Experience building, defining, and owning data models, data lakes, and data warehouses Programming proficiency in the likes of Python, Pyspark, SQL, Scala or Java. Experience operating in a cloud-native environment such as Azure, AWS, GCP etc ( Fabric experience would be beneficial ...

2+ years operating Azure Databricks in production (notebooks, jobs, cluster policies, Delta Lake), in combination with Unity Catalog Strong programming skills in Python/PySpark and expert-level SQL. Practical knowledge of CI/CD (Azure DevOps/Git), IaC (Terraform) Understanding of Enterprise Data Warehouse and Data Vault ...

Essential Skills for the AWS Data Engineer: Extensive hands-on experience with AWS data services Strong programming skills in Python (including libraries such as PySpark or Pandas) Solid understanding of data modelling, warehousing and architecture design within cloud environments Experience building and managing ETL/ELT workflows and data ...

Azure Storage. Excellent SQL and data modelling (star/snowflake, dimensional modelling). Knowledge of Power BI dataflows, DAX, and RLS. Experience with Python, PySpark, or T-SQL for transformations. Understanding of CI/CD and DevOps (Git, YAML pipelines). Strong grasp of data governance, security, and performance ...

Azure Storage. Excellent SQL and data modelling (star/snowflake, dimensional modelling). Knowledge of Power BI dataflows, DAX, and RLS. Experience with Python, PySpark, or T-SQL for transformations. Understanding of CI/CD and DevOps (Git, YAML pipelines). Strong grasp of data governance, security, and performance ...

security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations ...

data sources into Azure Databricks. Optimize pipelines for performance, reliability, and cost, incorporating data quality checks. Develop complex transformations and processing logic using Spark (PySpark/Scala) for cleaning, enrichment, and aggregation, ensuring accuracy and consistency across the data lifecycle. Work extensively with Unity Catalog, Delta Lake, Spark ...

traffic or fintech environments. You have an HBO+ degree in a relevant technical field. Your toolbox includes advanced SQL (including window functions), Python (Airflow, PySpark), and AWS services (EMR, Athena, Kinesis, Lambda). You're familiar with data modeling (star/snowflake schemas), infrastructure automation using Terraform, and GitLab ...

Experience as a Senior Data Engineer or similar Strong knowledge of Azure Cloud architecture and Azure Databricks, DevOps and CI/CD. Experience with PySpark, Python, SQL and other data engineering development tools. Experience with metadata driven pipelines and SQL serverless data warehouses. Knowledge of querying API systems. Experience ...

Experience as a Senior Data Engineer or similar Strong knowledge of Azure Cloud architecture and Azure Databricks, DevOps and CI/CD. Experience with PySpark, Python, SQL and other data engineering development tools. Experience with metadata driven pipelines and SQL serverless data warehouses. Knowledge of querying API systems. Experience ...

office. About you : You have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology ...

office. About you : You have strong Python backend software engineer skills You have experience working with large data sets You have experience of using PySpark and ideally also Apache Spark You believe in automating wherever possible You're a collaborative problem solver with great communication skills Other technology ...

pipelines to ingest, transform, and load data from diverse sources (APIs, databases, files) into Azure Databricks. Implement data cleaning, validation, and enrichment using Spark (PySpark/Scala) and related tools to ensure quality and consistency. Utilize Unity Catalog, Delta Lake, Spark SQL, and best practices for Databricks development, optimization ...

Fabric. Migrate on-prem data to Fabric Lakehouse/Warehouse, enabling Power BI analytics. Develop robust data pipelines, notebooks, and data models using: Python, PySpark, SQL, Synapse, and Data Warehousing principles. Implement CI/CD processes with GitHub integration for lifecycle management. Ensure governance, security, and best practices across ...

patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. Azure ...

patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data quality, data governance processes, Git version control + Agile development environments. Azure ...

demand delivery Provide technical guidance for team members Providing 2nd or 3rd level technical support About You Experience using SQL, SQL Server DB, Python & PySpark Experience using Azure Data Factory Experience using Data Bricks and Cloudsmith Data Warehousing Experience Project Management Experience The ability to interact with the operational ...

master's degree. 2-3 years in data engineering or related field. Basic proficiency in SQL and Python. Familiarity with data processing frameworks (e.g., PySpark) is a plus. Knowledge and experience with Databricks and/or Snowflake. Exposure to Azure or other cloud platforms. Ability to understand data requirements ...

data models and analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear ...

data models and analytics; exposure to predictive modelling and machine learning is a plus. Proficient in SQL and Python, with bonus points for PySpark, SparkSQL, and Git. Skilled in data visualisation with tools such as Tableau or Power BI. Confident writing efficient code and troubleshooting sophisticated queries. Clear ...

Methodology or other similar dimensional modelling standards, is essential to the role. Technical experience building and deploying models and reports utilizing the following tools: PySpark Microsoft Fabric or Databricks Power BI Git CI/CD pipelines (Azure DevOps experience preferred) An understanding of the structure and purpose ...

manipulate data and building additional process to support data processing activities Experience manipulating, processing and storing data using tools such as SQL, dbt, pyspark and SSIS Comfortable building out and administrating data models, ensuring that they are accessible and used appropriately, leveraging platforms such as Databricks and Power ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...

Continuously improve data systems and maintain awareness of best practice and emerging approaches Essential Skills Experience with large-scale analytics engines (e.g. Spark/PySpark) and scripting languages such as Python or Scala. Hands-on use of cloud data stacks (e.g. SageMaker Notebooks, S3, Glue, Athena) and modern storage ...