301 to 325 of 387 Permanent PySpark Jobs

deep understanding of power and gas markets. Advanced Python and data analysis capabilities, alongside experience working with modern data frameworks such as SQL, PySpark, or cloud-based platforms. Expertise in quantitative techniques such as stochastic modelling, simulation methods, and time series analysis. Experience working collaboratively with trading desks ...

building data models and pipelines in Snowflake, migrating programme use cases, and collaborating with Data Scientists to validate model performance. Experience with Snowflake ML, PySpark, and DBT is essential. The candidate will manage the migration from Pre-Prod to Prod, support PoC phases, and document findings during the transition ...

existing pipelines. Working within a growing data team, you'll design scalable data models, build robust ELT pipelines in Databricks using SQL, Python and PySpark, and integrate data from multiple internal and third-party systems. You'll collaborate closely with Analytics Engineers to ensure data is reliable, well-governed … architecture and help define long-term data standards, rather than simply delivering against predefined requirements. You'll need experience with: Databricks, SQL, Python and PySpark Building and optimising ELT/data pipelines Data modelling, lakehouse architecture and modern data platforms Azure data services API and third-party system integration ...

Senior Data Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data Engineer, Data Engineer) Our client is a global innovator and world leader … with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience is also key, as are excellent communication skills ...

designing, developing, and implementing enterprise-scale data solutions. The ideal candidate will possess strong expertise in Snowflake and hands‐on experience with Python, PySpark, AWS, and Airflow for building scalable, high-performance data platforms. The role demands strong technical leadership, problem‐solving capabilities, and the ability to collaborate effectively … transformationsto support large‐scale data processing Build and optimize enterprise data warehouse solutions on Snowflake. Develop data ingestion, transformation, and orchestration workflows using Python, PySpark, and Airflow. Develop and optimize complex SQL queries, stored procedures, views, streams, and tasks in Snowflake. Implement scalable cloud‐based data solutions leveraging ...

Job Title Senior Data Engineer Job Description Summary Senior Data Engineer | Databricks | Agentic AI | EMEA & APAC We are the team leading the data transformation of our EMEA and APAC business, and we are hiring a ...

/ETL workflows within a Fabric Lakehouse environment Work with OneLake, Lakehouse architecture, and modern data engineering best practices Utilise PySpark, SQL and Databricks for complex data transformation and engineering workloads Create robust data ingestion, orchestration and monitoring processes Support the development of enterprise-wide analytics solutions and Power … with Microsoft Fabric, including Data Factory, Dataflows, Notebooks and Lakehouse solutions Experience building and supporting production-grade ELT/ETL pipelines Strong skills in PySpark and SQL Experience working with OneLake, Delta formats or modern Lakehouse architectures Exposure to Databricks and distributed data processing environments Understanding of data modelling ...

engineering, or a related field Experience analysing gambling marketing activities The ability to write clear, well-structured code Hands‐on experience with Python, SQL, PySpark, or similar tools A flair for problem‐solving and storytelling with data — and the confidence to present findings to clients Sound like ...

GBMs, Elastic Net GLMs, GAMs, Decision Trees, Random Forests, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW's Radar software Proficient at communicating results ...

GBMs, Elastic Net GLMs, GAMs, Decision Trees, Random Forests, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar software Proficient at communicating results ...

GBMs, Elastic Net GLMs, GAMs, Decision Trees, Random Forests, Neural Nets and Clustering Experience in statistical and data science programming languages (e.g. R, Python, PySpark, SAS, SQL) A quantitative degree (Mathematics, Statistics, Engineering, Physics, Computer Science, Actuarial Science) Experience of WTW’s Radar software Proficient at communicating results ...

facing team in London. You will mentor data scientists, oversee complex data products, and ensure analytical excellence. The ideal candidate has strong SQL and PySpark experience, excellent communication skills, and a Bachelor’s degree in a related field. This position offers a collaborative environment, competitive benefits, and a chance ...

Engineer role. They aren't looking for someone who wants to move away from development into pure management. You'll work across: Python/PySpark Microsoft Fabric Spark and notebook engineering Lakehouse/Medallion architecture Delta SQL Azure Git/CI/CD Automated testing Data quality and observability … Reusable engineering patterns They're particularly interested in people coming from Databricks/PySpark environments, so extensive Fabric experience isn't essential. At Senior level, you'll also have the opportunity to influence architecture, technical design, coding standards, CI/CD, testing and engineering best practice, while supporting other ...

team. Nice to have: A PhD or equivalent research training in a quantitative field, and/or familiarity with Databricks, dbt, Spark/PySpark, or similar large-scale data platforms. Experience modifying infrastructure-as-code (e.g. Terraform) or contributing to production data pipelines. Experience building shared tools or platforms … them. Experience with UTXO, EVM, or other blockchain data, graph computation, and/or the cryptocurrency ecosystem. Technologies we use: Python SQL Databricks Dbt PySpark/Spark Terraform AWS Postgres AI at Chainalysis AI is not a feature at Chainalysis - it is a new way of working. One that ...

support production-grade ETL/ELT pipelines that source, transform and curate data for Internal Audit. Develop maintainable Python solutions using Pandas, Polars and PySpark, selecting fit-for-purpose approaches based on scale, performance and operational needs. Write advanced SQL for data transformation, reconciliation, profiling and analysis across OLTP … computer science, engineering or a related quantitative discipline. A master’s degree is welcome. Hands-on proficiency in Python, including Pandas, Polars and PySpark, with experience writing testable and maintainable code. Strong SQL proficiency and practical understanding of OLTP databases, relational design and query performance. Strong understanding of data ...

data transformationsTest framework and supporting toolingLead testing of AWS‐based Lakehouse and data pipeline solutions, including: AWS GlueApache IcebergAI‐enabled ETL acceleratorsNeo4jPython/PySpark, SQL and YAML‐driven configurationsQuality, Defect & Risk ManagementEnsure quality issues, risks issues and data defects are identified early, clearly documented and effectively triaged. Monitor, track … creation, masking and environment readiness. What experience you'll bring: Extensive experience in data engineering/test management. Strong technical capability in: Python/PySpark, SQL, Oracle, HDFS and YAMLBPSS minimum, SC not mandated but strong likelihood may be requiredStrong experience in leading test automation in data testing, ideally ...

engineering organisation behind you. Specific responsibilities include, but are not limited to: Lead the migration of legacy SQL solutions into Databricks Jobs, Workflows, and PySpark/SQL pipelines, maintaining operational SLAs throughout Define the target-state data architecture - Lakehouse, Medallion, and domain-oriented data product patterns - to support reporting … work close to commercial ambiguity. Strong, hands-on experience with Databricks, delivering enterprise-scale pipelines, workflows, and Lakehouse architectures Advanced proficiency in Python, PySpark, and SQL A proven track record migrating SQL Server workloads, stored procedures, and scheduled jobs to modern, cloud-native data platforms A solid understanding ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role … integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such ...

scalable, secure, and high-performance data platforms.Key Responsibilities Design and implement scalable AWS Data Lakehouse solutions Develop and maintain data pipelines using Python and PySpark Work with AWS services including: S3, EMR, Lambda, IAM, EKS, MWAA Build and manage Apache Iceberg tables for efficient data storage and querying Implement … engineersRequired Skills & Experience Strong experience in AWS Data Engineering Hands-on expertise with: AWS services: IAM, Lambda, EKS, S3, EMR, MWAA Apache Iceberg Python & PySpark Solid understanding of data lakehouse architecture principles Experience with CI/CD pipelines and Terraform Knowledge of data quality frameworks and data security standards ...

driven outcomes. Develop interactive business applications using Foundry Workshop , Slate , and custom front-end solutions. Build robust applications and integrations using TypeScript , Python , and PySpark . Collaborate with business and technical stakeholders to translate requirements into scalable data solutions. Integrate AI/ML capabilities into data workflows and support … hands-on experience with Palantir Foundry , including: Data ingestion and integration Data transformations Foundry datasets Foundry applications and Workshop Strong programming expertise in: Python PySpark Experience building custom applications and user-facing solutions within Foundry. Strong understanding of distributed systems and microservices architecture. Experience with AWS services such ...

orchestration, experiment tracking, hyper-parameter tuning, and deploying automated model retraining pipelines. Programming & Data Engineering : Proficiency in object-oriented Python, advanced dataframes (Polars/Pyspark), and data versioning (DVC). Experience designing data storage solutions and using object-oriented SQL interfaces. Cloud & DevOps : Hands-on experience with at least … experience Advanced Education : A Master's degree or PhD in a relevant field is a strong plus. Parallelisation & Performance : Experience with parallelisation frameworks like Pyspark or Ray. Advanced Cloud & Infrastructure : Familiarity with serverless deployments (e.g., Fargate, Lambdas), infrastructure automation with Terratest or Ansible. ...

modern data platform. Develop and maintain Medallion Architecture data layers, ensuring data is available for reporting and analytics. Create high-performance data transformations using PySpark, Databricks and Fabric Notebooks. Implement data governance and access controls through Databricks Unity Catalog. Support CI/CD deployment processes using Git integration … within a Data Engineering environment, delivering reliable and scalable data solutions. Strong hands-on experience with Microsoft Fabric and Databricks. Strong programming capability across PySpark and T-SQL. Experience working with Delta Lake, Lakehouse architectures and modern cloud data platforms. Understanding of data quality frameworks, testing approaches and governance ...

focus for this role over the next few months will include: Designing, building and maintaining production-grade data pipelines and services across AWS, PySpark, Snowflake and SQL-based transformation frameworks Developing and maintaining modular, testable and high-quality data models and transformation pipelines using DBT or equivalent structured … Infrastructure-as-Code such as Terraform Your Profile Essential skills/knowledge/experience: Experience with AWS data stack (S3, EMR, EC2, etc.) PySpark & Snowflake & CI/CD & DevOps Performance optimisation and large-scale data processing Data modelling (RDV/BDV or similar patterns preferred) Query performance tuning ...

Principal Consultant - Lead Data Analyst/Data Modeller Company: Transform Together Contract: 6-month Fixed Term Contract, with option to extend or become a permanent member of the team Compensation: OTE £90,000 Location: Hybrid ...

Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across ...