326 to 350 of 387 Permanent PySpark Jobs

7Dxperts in the UK is seeking a Senior Data Engineer to design and optimize scalable data pipelines using Databricks, PySpark, Python and SQL, gaining hands-on experience with Delta Lake and lakehouse architectures. You will build ETL/ELT workflows, monitor performance, mentor juniors and contribute to cloud data ...

capabilities, and skills Hands-on experience delivering system design, application development, testing, and operational stability at enterprise scale Expertise in Python and/or PySpark Advanced knowledge of software application development and technical processes, with depth in disciplines such as cloud, AI/ML, or data engineering Experience … scalable solutions Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks ...

technical SME across Databricks, Fabric, Synapse, and Azure. Own solution design, implementation strategy, and technical delivery. Build and optimise pipelines using PySpark , Spark SQL , ADF , Synapse Pipelines , or Fabric Pipelines . Define engineering best practice, conduct code reviews, and uplift quality. Develop secure, performant SQL solutions. Manage Azure DevOps … What you’ll bring Proven leadership of Data Engineering teams. Strong commercial experience with Databricks , Microsoft Fabric , or Azure Synapse . Deep expertise in PySpark , Spark SQL , and SQL optimisation. ETL/ELT pipeline development and orchestration. Experience building scalable cloud-based data platforms. Nice to Have Snowflake experience. ...

data exploration skills Experience of working in Agile methodology Good knowledge and hands-on experience of developing Stored Procedures Experience of working with PySpark Notebooks for data ingestion and provision Practical experience of improving data quality and efficiency in data pipelines Solid understanding of DevOps and CI/… Understanding of compliance and lifecycle of data management Desirable Skills: Coding experience in any relevant language (Python, Pyspark etc.) Experience working with other Azure technologies such as Fabric, Power Automate Experience of working with non-relational databases (CosmosDB, MongoDB, etc.) and handling unstructured data in ETL pipelines Our values ...

cloud-native data solutions that ensure data is accurate, accessible, and trusted. Key responsibilities include: Designing and developing scalable data pipelines using Python and PySpark Building and optimising data workflows using AWS Glue, Athena, Redshift , and related AWS services Supporting modern lakehouse architectures, including Apache Iceberg Working with relational … solutions. Essential experience Experience as a Data Engineer, ETL Developer, Data Platform Engineer, or a similar role Strong hands-on experience with Python and PySpark Experience with AWS Glue, Amazon Athena, Amazon Redshift , and cloud-based data platforms Strong SQL skills and experience working with both relational and NoSQL ...

Snowflake – Warehouses, Streams/Tasks, Snowpipe, Snowpark, UDFs/Stored Procedures, Time Travel and Cloning. Snowpark and end‐to‐end pipeline development. Experience with PySpark, Polars or Dask is also valuable. Python, including testing, build tooling and engineering best practices. SQL, including Snowflake‐optimised SQL. dbt, including Jinja templating … champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great opportunity to join a collaborative ...

Data Engineer - Python/PySpark/Microsoft Fabric £55,000 - £65,000 + Benefits St Helens - 1-2 days per week This is an opportunity to join an organisation at a particularly interesting point in the development of its data capability. The business has invested heavily in modernising … procedures and drag-and-drop development. The team is deliberately moving towards a more code-led engineering approach, with a strong emphasis on Python, PySpark and notebook-based development. What you'll be working with The environment includes: Python/PySpark Microsoft Fabric Spark and notebook development Lakehouse ...

estate from Python/MySQL into a Databricks + Snowflake ecosystem built for scale, auditability and reliability. The Role Build & optimise pipelines in Databricks, PySpark, Python, Kafka Modernise legacy systems into a scalable, auditable cloud data platform Work closely with Product, DevOps & Regulatory teams You Bring Strong Python, PySpark ...

verbal communication skills, including the ability to explain complex methodology to diverse audiences. You might also have Familiarity with Databricks, DBT, Spark/PySpark, or similar large‐scale data platforms. Modified infrastructure‐as‐code (e.g. Terraform) or contributing to production data pipelines. Mentored junior data scientists or engineers. Knowledge … passion for, the cryptocurrency and blockchain space. Technologies we use Python SQL Databricks DBT PySpark/Spark Terraform AWS Postgres We encourage applicants across any race, ethnicity, gender/gender expression, age, spirituality, ability, experience and more. If you need any accommodations to make our interview process more accessible ...

capabilities, and skills Hands-on experience delivering system design, application development, testing, and operational stability at enterprise scale Expertise in Python and/or PySpark Knowledge of software application development and technical processes, with depth in disciplines such as cloud, AI/ML, or data engineering Experience in large … scalable solutions Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks ...

systems, including LLM-based applications and agentic architectures in regulated environments Proficiency in one or more programming languages, particularly Python and/or PySpark Advanced knowledge of software application development and technical processes in disciplines such as cloud, AI/ML, or data engineering Experience in large-scale data … analytical skills Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Hands-on expertise with Spark/PySpark and big data processing technologies Knowledge of open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model ...

Support data migration efforts from Azure to Databricks Use Terraform to manage and deploy cloud infrastructure Build robust data workflows in Python (e.g., pandas, PySpark) Ensure the platform is scalable, efficient, and ready for future AI use cases REQUIRED SKILLS & EXPERIENCE Strong experience with Azure and Databricks environments Advanced … Python skills for data engineering (pandas, PySpark) Proficiency in designing and maintaining ETL pipelines Experience with Terraform for infrastructure automation Track record of working on cloud migration projects, especially Azure to Databricks Comfortable working onsite in London 2 days/week and engaging cross-functionally Strong communication and problem ...

seeking a highly skilled, hands-on Databricks Lead with 8+ years of experience in data engineering , including deep expertise in Azure Databricks, PySpark, and Structured Streaming. This role is ideal for a senior engineer with a programming-first mindset, a strong understanding of distributed systems, and the ability … Responsibilities Lead the design and development of scalable, high-performance data pipelines, streaming tables, and Delta Live Tables (DLT) using Azure Databricks and PySpark Drive Spark performance tuning and implement cost-optimization strategies within the Databricks environment Build and manage real-time and batch workflows using Structured Streaming Leverage ...

pipelines to ingest, transform, and prepare data Develop and optimise datasets for reporting and analytics Implement data transformations using SQL and Python (incl. PySpark) Monitor, troubleshoot, and improve performance of data pipelines and queries Apply data quality controls and maintain clear documentation Contribute to ongoing improvements in data architecture … with AWS data services (e.g. Redshift, Athena, S3, Glue, Lambda) Strong SQL skills Exposure to SnapLogic or similar integration platforms Proficiency in Python (including PySpark) Familiarity with modern data tooling (e.g. open table formats, DuckDB beneficial) Experience with version control (Git/CodeCommit preferred) Awareness of BI tools (e.g. ...

data solutions. You’ll play a senior technical role in designing, building and optimising data pipelines and data products, using tools such as Databricks, PySpark and Python. About the role As Senior Data Engineer, you’ll help create, maintain and improve high-quality data products that support key business … mentor colleagues where appropriate. Key responsibilities will include: Designing, building and optimising data pipelines and data products. Developing data engineering solutions using Databricks, PySpark and Python. Creating and maintaining notebooks, frameworks and reusable engineering components. Contributing to data governance, technical standards and engineering best practice. Identifying opportunities to improve ...

star schema and other best practices to optimise data usability. Sets best practices and standards on how we use SQL & dbt/Python/Pyspark and/or other languages to solve business problems. Becomes an expert in the business subject area the team are working on, holding expertise … click. We're a group that chooses to lift each other up and think smart every day. Extensive experience in SQL/Python/Pyspark and/or other languages relevant to data processingExtensive experience in dbt and ideally Airflow (or other orchestration tools)Extensive experience in Entity-Relationship ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

JOB ID: FEQ427R333 Location: Leeds/Manchester, United Kingdom - remote contract Industry: Enterprise Retail or CPG (UKI Enterprise) The Role As a Sr. Solutions Architect, you are an expert-level technical leader responsible for the ...

Data Engineer Up to £60,000 We are working with an organisation that is going through an exciting period of growth and continued investment across its technology and data landscape. With data playing an increasingly ...

Data Engineer Up to £60,000 We are working with an organisation that is going through an exciting period of growth and continued investment across its technology and data landscape. With data playing an increasingly ...

Data Engineer Up to £60,000 We are working with an organisation that is going through an exciting period of growth and continued investment across its technology and data landscape. With data playing an increasingly ...

Data Engineer Up to £60,000 We are working with an organisation that is going through an exciting period of growth and continued investment across its technology and data landscape. With data playing an increasingly ...

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time ...

ability to contribute across the length and breadth of a projectProficiency in one or more of the following: Java/Spring Boot, Python/PySpark, GraphQL, or mobile development frameworks, with openness and aptitude to work across others as neededExperience with cloud platforms (preferably AWS) and working knowledge … knowledge of the financial services industry and their IT systemsPreferred qualifications, capabilities, and skillsExperience with high-volume batch and real-time data processing using PySpark or similar frameworks, including performance tuning and troubleshooting of large-scale data pipelinesIn-depth knowledge of relational and distributed databases (e.g., Oracle, Databricks) with ...

Engineering team focused on Customer value understanding and delivering scalable data solutions. You will design, build and maintain production-grade data pipelines across AWS, PySpark, Snowflake and SQL-based transformations, while driving best practices in CI/CD, DevOps and data modelling in a cross-functional environment. #J ...