1 to 25 of 58 PySpark Jobs in the North of England

Databricks ecosystem. What you’ll need: Strong experience building data pipelines using Azure Data Factory, Databricks and Azure Data Lake. Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation. Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming. Knowledge of data warehousing, lakehouse ...

deep Azure Synapse/Databricks experience with clear ability to specialise quickly in Fabric. Expert SQL/T-SQL plus strong Python or PySpark, with a track record of building maintainable ELT/ETL pipelines and analytical data models. Strong Power BI and DAX experience, including semantic modelling, incremental ...

analytics strategy AI & Automation Agentic AI patterns and intelligent automation solutions Document intelligence and AI-driven business use cases SQL Python and/or PySpark Data pipelines and transformation frameworks CI/CD concepts and deployment automation Azure DevOps and/or GitHub Infrastructure as Code awareness Environment promotion ...

hand over to Evoke teams Collaborate daily with Evoke ML engineering, the CF team, and AWS ProServe counterparts Skills & experience Must-have Python/PySpark - Expert. Production data pipeline development, not analysis-only AWS data/ML stack - Advanced. S3, Glue and/or EMR, IAM basics ...

hand over to Evoke teams Collaborate daily with Evoke ML engineering, the CF team, and AWS ProServe counterparts Skills & experience Must-have Python/PySpark - Expert. Production data pipeline development, not analysis-only AWS data/ML stack - Advanced. S3, Glue and/or EMR, IAM basics ...

OneLake, pipelines and notebooks, and how these can be used to build scalable enterprise data solutions. Strong experience with SQL, Python and/or PySpark and modern data engineering approaches. Strong understanding of data architecture and data modelling, including designing reusable and scalable data structures. Experience implementing modern software ...

premises data sources. Engineer reliable data pipelines and models: Design batch, incremental and near-real-time ETL/ELT using SQL, Python and PySpark; apply medallion, dimensional/Kimball, normalised and data-product patterns as appropriate; and use production-grade techniques such as CDC, SCD, schema-change handling, idempotency … implementation. Strong Microsoft Fabric experience across OneLake, Lakehouse, Warehouse, Data Pipelines, Notebooks and Dev/Test/Production workspace patterns. Advanced SQL, Python and PySpark, with practical experience of Delta/Parquet, medallion architecture and production-grade batch/incremental ETL/ELT. Strong Git and CI/ ...

technical leader and mentor , raising standards across the data engineering community What you’ll bring Deep expertise in Databricks (Spark/PySpark, Delta Lake, Unity Catalog, Workflows) Strong experience building ETL/ELT pipelines , both batch and streaming Advanced Python and SQL skills Hands-on experience with cloud platforms ...

tools and approaches. Hands-on experience with Microsoft Azure technologies, including MS Fabric, Data Factory, Azure Synapse & RDS. Experience with Python or PySpark for scripting and automating data tasks. Knowledge of CI/CD approaches for Data Platforms Knowledge of MS SQL Server, SSIS, Visual Studio, and SSDT projects. ...

Ability to manage workload independently and drive initiatives proactively. Experience designing and supporting enterprise-scale cloud data platforms. Skills ETL, Power Automate, Power BI, PySpark #J-18808-Ljbffr ...

Databricks Serverless configuration and workload optimisation Databricks SQL and Delta Lake development Cluster management, compute policies, autoscaling, and workload monitoring Data Engineering Strong Python, PySpark, and SQL development skills Design and build ingestion and transformation pipelines Full and incremental data loading CDC (Change Data Capture) implementation Data quality controls ...

Generation), Serverless, DWH models, Data Mesh, Data Fabric, Medallion and Data Products. Hands on mindset : Fluent in the language of data, whether SQL, Python, PySpark, R or Scala. Open formats including Parquet, Iceberg and JSON. Agile Ways of Working Run and contribute to agile ceremonies, helping to shape ...

data solutions to meet business needs. Essential Skills: Strong proficiency in T-SQL and experience developing complex queries and procedures. Expertise in Spark and PySpark for scalable data processing. Solid understanding of OLTP and OLAP architectures and data modelling principles. Experience with Microsoft Azure, including Fabric, Azure Data Factory ...

knowledge of Matillion’s Data Productivity Cloud/Matillion ETL, or primary competitive data integration platforms. Big Data & Analytics Ecosystems: Experience with Spark/PySpark, modern Data Lakes/Lakehouses, and MPP database architectures (Teradata, Vertica, Netezza). Education: Bachelor’s or Master’s degree in Computer Science, Data ...

actually trust and use. Essential Skills & Experience: Hands-on experience with Databricks and Microsoft Fabric Experience building and running production-grade pipelines Strong PySpark and SQL skills Experience with Delta Lake and Lakehouse architectures Good grasp of data modelling, orchestration and workflow automation Comfortable working with structured and semi ...

Data Engineer Key skills: Python SQL Server/SQL Data Warehousing & Data Modelling Data Architecture Data Pipelines & Integration Solutions Microsoft Fabric Azure Cloud Technologies PySpark ETL/ELT Development Data Governance & Security Source Control (Git) Technical Leadership & Team Management You will be a Lead Data Engineer Developer with good ...

Data Engineer Key skills: * Python * SQL Server/SQL * Data Warehousing & Data Modelling * Data Architecture * Data Pipelines & Integration Solutions * Microsoft Fabric * Azure Cloud Technologies * PySpark * ETL/ELT Development * Data Governance & Security * Source Control (Git) * Technical Leadership & Team Management You will be a Lead Data Engineer Developer with good ...

required as well as a solid understanding of working in a cloud-based environment (AWS is preferred).Proficient using transformation tools such as PySpark/Pandas.Knowledge and practice of graph databases is preferred but not required.Key responsibilities/Skills and experienceFamiliarity using data quality frameworks, ie. Great Expectations.Experience building ...

present insights clearly to both technical and non-technical audiences.Some other highly valued skills include:Strong SQL and database experience.Knowledge of Python/PySpark and data analytics.Experience working in a Call Centre or Contact Centre environment.Strong communication and stakeholder management skills.Ability to lead technical teams and mentor junior developers.You ...

insights clearly to both technical and non-technical audiences. Some other highly valued skills include: Strong SQL and database experience. Knowledge of Python/PySpark and data analytics. Experience working in a Call Centre or Contact Centre environment. Strong communication and stakeholder management skills. Ability to lead technical teams ...

solutions Requirements: Enterprise data warehouse architecture and design Microsoft Fabric, Azure Data Platform technologies and modern Lakehouse architectures Databricks, Unity Catalog and Delta Lake PySpark, Databricks, SQL Medallion Architecture implementation Data modelling approaches including Star Schema and Data Vault Data governance, security and access management Why This Opportunity Greenfield ...

engineering best practicesYou have experience with Databricks, ideally also AzureYou have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and PandasYou're excited to join a start-up in a role that you can shape and influenceYou're collaborative with excellent communication skillsApply ...

data and ETL processes Strong SQL, Python and pytest experience Experience with data validation and large datasets Knowledge of ETL, data warehousing and PySpark or similar Experience with AWS and/or Azure Familiarity with Kafka or other event-driven/data streaming technologies Understanding of CI/ ...

deliver dashboards that meet Barclays standards and business needs.Some other highly valued skills may include:Experience with SQL and database technologies.Knowledge of Python/PySpark and data analytics.Experience in a Call Centre or Contact Centre environment.Strong communication and stakeholder management skills.Ability to collaborate effectively across technical and business teams.You ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications• Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications• Advanced Data Management – Strong SQL design, query ...