1 to 25 of 262 PySpark Jobs in England

e.g. Power BI) Deep understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability ...

Cloud Platforms:Deep experience with AWS (S3, Lambda, Glue, Redshift) and/or Azure (Data Lake, Synapse). Programming & Scripting:Proficiency in Python, SQL, PySpark etc. ETL/ELT & Streaming:Expertise in technologies like Apache Airflow, Glue, Kafka, Informatica, EventBridge etc. Industrial Data Integration:Familiarity with OT data schema ...

mentoring experience, with a genuine commitment to team growth and wellbeing. Strong hands-on skills in: AWS (or equivalent cloud platforms) Python/PySpark for data engineering and automation TypeScript, Node.js, React.js for full-stack development Solid grasp of distributed systems design, secure coding, and data privacy principles. Familiarity ...

. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Working with structured/semi-structured telecom datasets (CDRs, network logs, subscriber data) Data Lake Technologies: Expertise in Apache Iceberg table format … Ideal Candidate Profile 310+ years of experience in AWS Data Engineering/Support Engineering. Strong telecom domain understanding. Hands-on with Iceberg, Athena, Glue (PySpark), Python, Redshift, S3, ETL frameworks. Strong troubleshooting mindset and ability to work in 247 or rotational support environments (if required). ...

data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

platform migration efforts to Databricks. Requirements Strong hands-on experience delivering production-grade data engineering solutions. Expert-level SQL and proficiency in Python/PySpark; capable of building reusable, scalable code. Experience with modern orchestration tools (e.g., Airflow, AWS Step Functions). Experience with Infrastructure as Code (CloudFormation), Unit ...

Hive/Hadoop, Apache Spark, Kafka, Airflow Working with programming languages Python , Java, SQL. Working on building ETL (Extraction Transformation and Loading) solution using PySpark Experience in SQL/NoSQL database design Deep understanding in software architecture, object-oriented design principles, and data structures Extensive experience in developing microservices ...

ideally in enterprise-scale environments Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake Hands-on experience with Spark (PySpark/SparkSQL) and distributed data processing Strong data modelling skills (Dimensional, ODS, Data Vault) and data warehousing concepts Proficiency in SQL, version control ...

data engineering and 1+ experience as Lead Data Engineer Proven expertise with Microsoft Azure Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark . Expertise in Databricks for large-scale data engineering and analytics workloads. Design and implement scalable data architectures using Databricks Unity Catalog, Delta Lake ...

regulatory reporting. In this role, you’ll design, develop, and maintain data platforms using technologies such as Snowflake, Databricks, Synapse/Fabric, and PySpark, ensuring the scalability, security, and performance of all data systems. You will establish and champion best practices for data engineering while creating development environments that ...

data strategy, ensuring scalability, performance, and governance while working with Delta Lake, Data Catalog, and PySpark. Key Skills Required: Databricks Lakehouse Architecture Apache Spark, PySpark, Delta Lake Data Engineering & ETL/ELT pipelines Cloud platforms (Azure preferred; AWS/GCP acceptable) Strong experience in designing scalable data solutions ...

quality and structure of data Translate business needs into clear technical plans Experience required: Strong skills in data modelling SQL with either Python or PySpark Databricks or similar cloud data platform Experience communicating with both technical and non-technical users Leadership or mentoring experience Insurance, financial services or regulated ...

this Senior Data Engineering role, you’ll design, develop, and maintain data platforms using technologies such as Snowflake, Databricks, Synapse/Fabric, and PySpark, ensuring the scalability, security, and performance of all data systems. As the Senior Data Engineer your role will be to include establishing and championing best ...

developer, ideally within high-performance, data-heavy environments. Strong grounding in computer science fundamentals — algorithms, data structures, and system design. Experience with NumPy, Pandas, PySpark, or similar data frameworks. Exposure to financial data, trading systems, or quantitative research is highly advantageous (though not required). Passion for clean, performant ...

real-time insights for traders and quantitative teams. You will also work with complementary technologies such as Databricks, Snowflake, Azure Synapse/Fabric, and PySpark — but the focus is on extracting the full performance and scalability of ClickHouse within an enterprise-grade data ecosystem. What You’ll Do Design … with ClickHouse (schema design, cluster configuration, performance tuning, ingestion pipelines). Solid understanding of modern data engineering principles and lakehouse architectures. Strong Databricks and PySpark experience. Proficiency in Python and SQL. Experience working with Azure and DevOps/cloud-native tooling. Proven background in data orchestration, ETL/ ...

The Role We are looking for a skilled and experienced Senior Data Engineer to join our Data Science team. The team ingests large amounts of complex sensor data (billions of data points a day), combines ...

data solutions, leading data pipeline architecture, and ensuring best practices across cloud data systems. Key Responsibilities: Design and architect data platforms using Azure Databricks (PySpark), Data Lake, and Data Warehouse. Build and manage data pipelines using Azure Data Factory. Design integrations using Azure API Management, Function Apps, and Logic … Cosmos DB, and serverless services. Create and maintain strong data models and ensure efficient data flow across systems. Must-Have Skills: Azure Databricks, PySpark Azure Data Factory Azure API Management Azure Function Apps & Logic Apps Event-driven architecture (Event Hub, Service Bus, Cosmos DB) Data modeling, Data Lake, Data ...

operational information within related resolution and sampling timing and prescriptions for maintenance Implement prognostic algorithms with different proprietary technologies and coding languages such a Pyspark , Python, PowerBI Analyse diagnostics and maintenance data Monitor prognostic system performances and statistical analysis on collected data to identify critical trends and conditions. Support … Engineering Experience in railway domain (preferred) or other manufacturing industry such as avionics, automotive or R&D. Experience in coding with Python, SQL, preferably Pyspark, and versioning tools (SVN, Git) Familiar with PowerBI dashboard development. Able to interpret electrical drawings, system specification software specification (e.g., UML) and mechanical drawings ...

days per week onsite What you’ll be doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … environment What you’ll need: Proven experience as a Data Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery ...

solutions. This will be a long term project with scope for extensions and possible transition to permanent staff. Core Skills Needed: Strong Python development PySpark and SQL expertise Solid understanding of Azure infrastructure Knowledge of Containers, Microservices, and functional design patterns Experience working in Agile environments Terraform ...

years of hands-on experience in enterprise data modelling using tools such as ERwin, ER/Studio, or Power Designer Proven experience with Databricks, PySpark, Python, SQL, and modern data platform architectures including data lakes and data warehouses Knowledge of batch and streaming ingestion, CI/CD tools (Azure ...

years of hands-on experience in enterprise data modelling using tools such as ERwin, ER/Studio, or Power Designer Proven experience with Databricks, PySpark, Python, SQL, and modern data platform architectures including data lakes and data warehouses Knowledge of batch and streaming ingestion, CI/CD tools (Azure ...

years of hands-on experience in enterprise data modelling using tools such as ERwin, ER/Studio, or Power Designer Proven experience with Databricks, PySpark, Python, SQL, and modern data platform architectures including data lakes and data warehouses Knowledge of batch and streaming ingestion, CI/CD tools (Azure ...

years of hands-on experience in enterprise data modelling using tools such as ERwin, ER/Studio, or Power Designer Proven experience with Databricks, PySpark, Python, SQL, and modern data platform architectures including data lakes and data warehouses Knowledge of batch and streaming ingestion, CI/CD tools (Azure ...