51 to 75 of 221 Remote PySpark Jobs

and data lake patterns , including ingestion, governance, and quality. Strong technical skills in BigQuery, DataProc, Dataform, Composer, Pub/Sub . Fluent in Python, PySpark, and SQL . Experience with BI tools like Looker or Power BI. Strong client-facing and communication skills — able to lead conversations with both More ❯

their data. Overview of responsibilities in the role: Design and maintain scalable, high-performance data pipelines using Azure Data Platform tools such as Databricks (PySpark), Data Factory, and Data Lake Gen2. Develop curated data layers (bronze, silver, gold) optimised for analytics, reporting, and AI/ML, ensuring they meet More ❯

to engage both technical and non-technical stakeholders. Desirable Criteria Experience with Matillion products and competitive ETL solutions. Knowledge of big data technologies (Spark, PySpark), data lakes, and MPP databases (Teradata, Vertica, Netezza). Familiarity with version control tools such as Git, and experience with Python. Degree in Computer More ❯

and managing data sets from multiple sources. 4+ years hands-on experience in key data management technologies, including Python but not limited to SQL, PySpark, Scoop, etc. Experience working on use cases for different file formats and database management systems including NOSQL databases Conceptual understanding of data management including More ❯

understanding of DevOps and CI/CD - Understanding of compliance and lifecycle of data management Desirable Skills - Coding experience in any relevant language (Python, Pyspark etc.) - Experience working with Azure Synapse, Databricks, and/or similar technology - Experience automating test cases - Experience working with non-relational databases (Cassandra, MongoDB More ❯

Ideal background: 3+ years in a Data Engineering position. Self-starter, who has experience with building data pipelines in the cloud. Excellent knowledge of PySpark, Python and SQL fundamentals. Familiar with Airflow, Databricks & BigQuery. Exposure with Palantir Foundry would be a huge plus but not essential. Ability to work More ❯

in Java Some knowledge inHadoop, hive, SQL, Spark Understanding of Unix Shell Scripting CI/CD Pipeline Maven or Gradle experience Predictive analytics (desirable) PySpark (desirable) Trade Surveillance domain knowledge (desirable) Education: Bachelor’s/University degree or equivalent experience What we’ll provide you: By joining Citi, you More ❯

valuable insights Desirable Experience working with enterprise data warehouse or data lake platforms Experience working with a cloud platform such as AWS Have used PySpark for data manipulation Previous exposure to game or IoT telemetry events and how such data is generated Knowledge of best practices involving data governance More ❯

experience working with Azure Git/DevOps Repos experience Demonstration of problem solving ability Synapse Analytics or similar experience - desirable Visual Files experience – desirable PySpark/Python experience – desirable Powershell experience – desirable What we offer: We aim to reward your hard work generously. You’ll be greeted in our More ❯

experience working with Azure Git/DevOps Repos experience Demonstration of problem solving ability Synapse Analytics or similar experience - desirable Visual Files experience - desirable PySpark/Python experience - desirable Powershell experience - desirable What we offer: We aim to reward your hard work generously. You'll be greeted in our More ❯

valuable insights. Desirable Experience working with enterprise data warehouse or data lake platforms. Experience working with a cloud platform such as AWS. Have used PySpark for data manipulation. Previous exposure to game or IoT telemetry events and how such data is generated. Knowledge of best practices involving data governance More ❯

experience working with Azure Git/DevOps Repos experience Demonstration of problem solving ability Synapse Analytics or similar experience - desirable Visual Files experience – desirable PySpark/Python experience – desirable Powershell experience – desirable What We Offer We aim to reward your hard work generously. You’ll be greeted in our More ❯

data pipelines by collecting high-quality, consistent data from external APIs and ensuring seamless incorporation into existing systems. Big Data Management and Storage : Utilize PySpark for scalable processing of large datasets, implementing best practices for distributed computing. Optimize data storage and querying within a data lake environment to enhance More ❯

data pipelines by collecting high-quality, consistent data from external APIs and ensuring seamless incorporation into existing systems. Big Data Management and Storage : Utilize PySpark for scalable processing of large datasets, implementing best practices for distributed computing. Optimize data storage and querying within a data lake environment to enhance More ❯

data pipelines by collecting high-quality, consistent data from external APIs and ensuring seamless incorporation into existing systems. Big Data Management and Storage : Utilize PySpark for scalable processing of large datasets, implementing best practices for distributed computing. Optimize data storage and querying within a data lake environment to enhance More ❯

Quality and Information Security principles Experience with Azure, ETL Tools such as ADF and Databricks Advanced Database and SQL skills, along with SQL, Python, Pyspark, Spark SQL Strong understanding of data model design and implementation principles Data warehousing design patterns and implementation Mainly home based working. Twice a month More ❯

in backend development. A strong foundation in data engineering , including designing practical data schemas and working with large-scale distributed systems. Proficiency in Python, PySpark, SQL, C#, .NET, and Typescript —and a willingness to quickly learn new technologies. Experience with Agile processes , source control (Git) , and delivering web-based More ❯

of MLOps and associated tools such as Azure DevOps/Github, MLFlow, Azure ML. Experience working with large datasets/big data architectures, particularly Pyspark/Databricks. Experience deploying container technologies (e.g. Docker, Kubernetes). Experience playing a lead role on technical AI projects. Excellent communication skills with both More ❯

apply if you have: Very strong experience working with Azure, Databricks, and large-scale data pipelines Strong proficiency in SQL, Python, and/or PySpark Understanding of data modelling, real-time streaming, and performance tuning Familiarity with DevOps practices, CI/CD, and IaC (Terraform, GitHub Actions) Experience with More ❯

occasional site visits to either London or Nottingham What you’ll bring Strong hands-on experience in data engineering within production environments Proficiency in PySpark and distributed data processing at scale Experience building and maintaining robust ETL pipelines and data lakes Solid understanding of cloud platforms such as Azure More ❯

Lake and Lakehouse architecture for efficient data management; experience with ETL processes and optimizing data pipelines for performance Be strong in Pandas, SQL, and PySpark Have a strong understanding of cloud networking principles, including Azure Virtual Networks, Private Endpoints, secure connectivity strategies Have experience or knowledge with GDPR compliant More ❯

analyze complex datasets and extract actionable insights • ETL pipeline development • Familiarity with machine learning workflows Optional skills: • Hands-on experience with Apache Spark (preferably PySpark) and Cloudera platform; • Practical knowledge of Elasticsearch for data indexing and querying Additional Information What do we offer you? International, positive, dynamic and motivated More ❯

Collaborate across technical and non-technical teams Troubleshoot issues and support wider team adoption of the platform What You’ll Bring: Proficiency in Python, PySpark, Spark SQL or Java Experience with cloud tools (Lambda, S3, EKS, IAM) Knowledge of Docker, Terraform, GitHub Actions Understanding of data quality frameworks Strong More ❯

Collaborate across technical and non-technical teams Troubleshoot issues and support wider team adoption of the platform What You’ll Bring: Proficiency in Python, PySpark, Spark SQL or Java Experience with cloud tools (Lambda, S3, EKS, IAM) Knowledge of Docker, Terraform, GitHub Actions Understanding of data quality frameworks Strong More ❯

Collaborate across technical and non-technical teams Troubleshoot issues and support wider team adoption of the platform What You’ll Bring: Proficiency in Python, PySpark, Spark SQL or Java Experience with cloud tools (Lambda, S3, EKS, IAM) Knowledge of Docker, Terraform, GitHub Actions Understanding of data quality frameworks Strong More ❯