1 to 25 of 101 PySpark Jobs in England

tooling Key skills Strong experience with Microsoft Fabric (Data Factory, Lakehouse, OneLake, Pipelines, SQL Warehouse) Solid data engineering background using SQL, Python, PySpark, Delta Lake E xperience building and optimising end-to-end data pipelines CI/CD experience for analytics and data platforms (Azure DevOps or GitHub) Exposure ...

solutions Strong expertise in SQL, data modelling, and cloud data platforms Hands-on experience with Snowflake and Microsoft Fabric (OneLake) Proficiency in Python and PySpark , including notebooks and automation Experience building and maintaining ETL/ELT pipelines at scale Strong understanding of performance optimisation and cost management Knowledge ...

facing or consulting environment (or strong desire to develop this) Ambition to move into team leadership/people management Nice to have: Experience with PySpark/Databricks Experience with streaming technologies like Kafka or AWS Kinesis Knowledge of data governance and quality frameworks ...

pipelines and working with lakehouse architectures Knowledge of Azure tools such as Data Factory and/or Fabric Data Pipelines Experience with Python or PySpark for data transformation Understanding of data modelling (dimensional modelling, medallion architecture) Strong communication skills and ability to work with non-technical stakeholders Desirable: Experience ...

from different sources. Good communication and teamwork skills. Experience with cloud data platforms such as Databricks or Snowflake on AWS. Strong skills in Python, PySpark, and SQL. Good understanding of ETL/ELT pipelines and data transformation. Experience with data modelling and database optimisation. Strong knowledge ...

data services and integration systems (e.g. Azure Data Factory, Azure Data Lake) and how they complement Fabric/Databricks Practical experience with Spark/PySpark (desirable but strong Spark experience counts) Experience implementing platform security practices (secure connectivity, secrets management, RBAC concepts) aligned to client policies Experience implementing data ...

cloud, S3, cross-cloud connectivity, API Gateways GCP (BigQuery, GCS, Dataflow) Deep expertise in: Databricks (Lakehouse architecture, Delta Lake, Unity Catalog, Delta Sharing, Pyspark, Spark SQL etc.) Snowflake (data warehousing & data sharing, Iceberg, ) Strong understanding of: Data architecture patterns, industry benchmarking, Data integration patterns (ETL/ELT, streaming ...

modern UI patterns, develop framework is essential Back-end: Expertise in C# .NET Core, REST APIs, and microservices-based development is essential Python (pyspark): Experience with data pipelines and analytics is a plus. Power BI: Familiarity with developing dashboards and data models. Architecture: Knowledge of microservices, API design ...

backgrounds. The technology stack is flexible, but experience in some of the following are nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake ...

Azure ML. Key skills and responsibilities, Demonstrated hands-on experience with Microsoft Fabric and Power BI. Extensive knowledge in data modelling, DAX, Python/PySpark, SQL/KQL. Proven record of technical leadership and effective stakeholder engagement. Design and deliver comprehensive Fabric solutions, including OneLake, Lakehouse, Warehouses, and Power ...

Advanced SQL skills for complex query validation. Experience with Agile/DevOps delivery models and JIRA test management. Familiarity with automation frameworks (SQL/PySpark preferred). Knowledge of CI/CD and release management processes. Experience with Power BI, Alteryx, and VS Code , particularly in testing integrations with ...

experience in architecting and designing robust, secure and compelling data solutions using Microsoft Fabric, Purview and other Microsoft ecosystem toolsets. Programming skills, including Python, PySpark and T-SQL. Strong track record in business development, proposition shaping, and winning work. Proven experience leading and developing high-performing multi-disciplinary technology ...

Pega, Adobe, and similar tools) using APIs, connectors, or data exports. Build and maintain scalable ETL/ELT pipelines using SQL, Python, and PySpark for batch and near real-time processing. Work with AWS services such as DMS, Glue, Lambda, Step Functions, S3, Redshift, Athena, RDS, EC2, and CloudWatch. ...

data providers-ensuring data is reliable, auditable, and delivered at scale. Data Engineer Responsibilities Act as the subject matter expert across ADF, Databricks, Python, PySpark, and Spark SQL Manage the daily ingestion and processing of raw data into enterprise data platforms Collaborate with the wider team to define data ...

Databricks Defining data models and Databricks architecture standards Establishing data governance and security frameworks Aligning data architecture with enterprise cloud strategy Supporting development using PySpark, SQL and Python within Databricks Working within CI/CD and DevOps environments (Git-based) About You: Strong technical background in Databricks and Azure ...

including: CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/ ...

warehousing, OLTP systems, data integration and SDLC Strong experience in big data frameworks & working experience in Spark or Hadoop or Hive (incl. derivatives like pySpark (preferred), Spark Scala or Spark SQL) or Similar, along with experience in libraries/frameworks to accelerate code development Experience using major data modelling ...

exposure to enterprise-level data platforms and best practices. The role will involve developing and maintaining data pipelines, transforming data using Python, SQL and PySpark, integrating data from various internal and external sources, and supporting day-to-day data operations. You’ll also contribute to data quality initiatives, documentation ...

Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Mandatory Required Skills Python/PySpark/Scala, SQL and data warehousing concepts Preferred/Desired Skills Design and build data pipelines (ETL/ELT) Manage data infrastructure (databases, lakes ...

Hands-on experience with Microsoft Purview • Strong Azure Data Platform experience: ADLS Gen2, Azure Databricks, Data Factory • Hands-on data engineering experience (Spark/PySpark, Delta Lake) • Knowledge of security and compliance controls, GDPR, PII/DLP, RBAC/ABAC • Experience leading Agile teams and managing risks, dependencies ...

/Ansible Work closely with engineering teams to deliver real-world ML systems across multiple sectors. Tech Stack You’ll Work With Python, SQL, PySpark PyTorch, TensorFlow, Scikit‐learn MLflow (core focus), Databricks, Azure IaC: Terraform, ARM, Ansible Git, Azure DevOps, Jenkins What We’re Looking For ✔️ 3+ years ...

practices You have experience with Databricks, ideally also Azure You have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and Pandas You're excited to join a start-up in a role that you can shape and influence You're collaborative with excellent ...

Builder or Code Repositories. Experience with Delta Lake, Parquet, Iceberg, CSV and JSON. Data governance, lineage, secure data-sharing and access approval experience. Python, PySpark or SQL development experience. CI/CD experience using GitLab, GitHub Actions, Azure DevOps or similar. Key Deliverables Configured and validated Foundry Databricks connector. ...

making Your Skills and Experience Strong commercial experience as a data engineer in modern, cloud based environments Advanced Python and SQL, with experience using PySpark or similar distributed processing frameworks Experience building and maintaining data pipelines in AWS using services such as S3, Glue or EKS Hands on experience ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...