76 to 100 of 195 PySpark Jobs in London

frameworks such as pytest Exposure to BDD (Behave) and/or TDD approaches Knowledge of ETL processes, data warehousing concepts, and tools such as PySpark (or similar) Experience working with cloud platforms such as AWS and/or Azure Familiarity with event-driven or data streaming technologies (e.g. Kafka ...

environment. What You'll Need: Strong experience as a Senior Azure Data Engineer An expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity Catalog Knowledge of Kimball data modelling, such as star schema, snowflake, etc. Strong SQL skills A flexible approach ...

experience with Databricks — running queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

experience with Databricks — running queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large‐scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

coding languages eg. Python, R, Scala, etc.; (Python preferred) Proficiency in database technologies eg. SQL, ETL, No‐SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques ...

Strong foundation in Python and SQL Experience building models for real commercial use cases (e.g. propensity, clustering, regression) Comfortable working with tools like PySpark, Databricks, Tableau or Power BI (nice to have, not essential, since the group uses a mix across its businesses) A track record of translating technical ...

interrogate data andverify correctness Hands-on experience with Databricks — running queries, navigatingUnity Catalog, reading Spark job outputs and understanding what they Working knowledge of PySpark or Spark SQL — enough to read pipelinecode, understand transformations, and trace where data Understanding of Lakehouse/medallion architecture (bronze-silvergold) and how data ...

language models Experience with experiment design and conducting A/B tests Experience building shared or platform-style ML systems Experience with Databricks and PySpark Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with ...

economists and data scientists. Based in London with a hybrid model (3 days on site), the role requires 8+ years in data engineering, strong PySpark/SQL, and active SC clearance. You will implement governance, Delta Lake, Unity Catalog, and CI/CD practices to ensure reliable #J ...

Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the entire value chain of organisations. We are proud ...

cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

Microsoft Azure. You will build production-grade data pipelines with Delta Lake, Unity Catalog, and Databricks tools. The role requires hands-on expertise in PySpark, Python, SQL, and Spark architecture, plus experience with GitLab-based CI/CD, Databricks Asset Bundles, and Azure Data Lake Storage. #J-18808-Ljbffr ...

through production deployment and monitoring — and know where each stage tends to go wrong.You're fluent in Python and SQL, and you've used PySpark or Spark to process data at scale.You've built agentic AI workflows — not just used off-the-shelf tools, but designed the automation yourself.You ...

troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental data loadingCDC (Change Data Capture) implementationData quality controls, schema evolution, reconciliation, and error ...

environment. What You'll Need: Strong experience as a Senior Azure Data Engineer An expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity Catalog Knowledge of Kimball data modelling, such as star schema, snowflake, etc. A flexible approach to getting things done ...

Требования Strong SQL skills, including window functions, CTEs, and aggregations; Hands-on experience with Databricks, Unity Catalog, and Spark job outputs; Working knowledge of PySpark or Spark SQL; Understanding of Lakehouse and medallion architecture; Familiarity with YAML-based configuration and structured test definitions; Comfortable with Git and basic engineering ...

Databricks/Delta Lake builds - ACID transactions, time travel, schema evolution, and performance tuning (OPTIMIZE, Z-ordering, VACUUM) Strong command of medallion architecture and PySpark, lazy evaluation, partitioning, broadcast joins, and diagnosing partition skew Solid Azure data platform experience; familiarity with Azure Analysis Services or a comparable semantic layer ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

interest in developing into an architectural role Experience working closely with Data Engineers, Analytics teams, and business stakeholders Exposure to Microsoft Fabric, Databricks, Python, PySpark, or dbt would be beneficial Experience with data modelling tools such as Sparx EA, Sparx Enterprise Architect, Enterprise Architect, ER/Studio, ERwin, Erwin ...

languages eg.?Python, R, Scala, etc.(Python preferred) Strong proficiency in database technologies eg.?SQL, ETL, No-SQL, DW, and Big Data technologies eg.?pySpark, Hive, etc. Experienced working with structured and also unstructured data eg.?Text, PDFs, jpgs, call recordings, video,?etc. Knowledge of machine learning modelling techniques ...

languages eg.?Python, R, Scala, etc.(Python preferred) Strong proficiency in database technologies eg.?SQL, ETL, No-SQL, DW, and Big Data technologies eg.?pySpark, Hive, etc. Experienced working with structured and also unstructured data eg.?Text, PDFs, jpgs, call recordings, video,?etc. Knowledge of machine learning modelling techniques ...

Declarative Pipelines with Expectations, including failure handling and observability Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns Production-level PySpark and SQL development skills Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM Experience with Databricks Workflows, orchestration ...

influencing skills. Ability to think creatively and propose innovative solutions. To qualify for this role, you must have: Proven experience and knowledge with PySpark and Apache Spark including the fundamentals of how it works. Experience with cloud platforms (e.g., AWS, Azure, GCP) and data lake architectures. Strong experience ...

engineering excellence through best practices, mentoring, and thoughtful technical direction. Job responsibilities Design scalable, reusable data processing and data quality frameworks using Python, PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow … transformations Model and transform data for analytics using SQL and dbt to support business intelligence and reporting workloads Write production-grade Python/PySpark code with disciplined testing, performance tuning, and maintainable object-oriented design Implement infrastructure-as-code (e.g., Terraform) to provision and manage cloud-based data platform ...