101 to 125 of 255 PySpark Jobs in England

complex data led organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design ...

management, network design and access control; Databricks and OCI, or comparable platforms Strong Python and SQL, and experience building and monitoring production pipelines using PySpark or Spark Experience with infrastructure as code, CI/CD and a Git and GitHub workflow A track record of setting engineering standards ...

Experience Strong experience across both Data Science and Data Engineering Hands-on expertise with Microsoft Fabric and Databricks Advanced Python and SQL skills, with PySpark desirable Experience building data pipelines and machine learning models Knowledge of NLP, entity resolution, or document processing is advantageous Strong communication skills and commercial ...

experience with Databricks — running queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

experience with Azure AI Search or other vector/hybrid search solutions, including vector search, hybrid search, and retrieval evaluation. Experience with Databricks and PySpark for large‐scale data processing. Experience designing evaluation frameworks for LLM systems, including offline metrics, hallucination detection, and production monitoring. Experience implementing responsible ...

ODCS/ODPS), data product portals, lineage tools. Experience in a regulated financial services company. Experience leading a migration or a large platform change. PySpark or other distributed processing experience. We’re building a place where everyone truly feels they belong. Even if your past experience doesn't align ...

Platforms: Google Cloud Platform, Amazon Web Services, Microsoft Azure, and Databricks. Experience in one or more of the listed Languages such as: Python, R, PySpark Experience in one or more of the following Agentic & Gen AI Platforms: such as Microsoft Copilot Studio, Adept AI, UiPath, OpenAI GPT-5 Agents ...

language models Experience with experiment design and conducting A/B tests Experience building shared or platform-style ML systems Experience with Databricks and PySpark Experience working with AWS or another cloud platform (GCP/Azure) Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with ...

covering statistical analysis, A/B testing, time-series, regression, classification and optimisation problems. Proficient in SQL & Python. Familiarity with data manipulation libraries (e.g. PySpark, Pandas, Numpy) and machine learning frameworks (e.g. scikit-learn, TensorFlow or PyTorch). Experience with MLOps tools (e.g., MLflow, Databricks Model Registry, Unity Catalog ...

data platforms and analytics capabilities across the business. This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved in the design, development, optimisation and support of enterprise-scale data platforms, working alongside engineers … latest data engineering technologies and best practices. Key Responsibilities Data Engineering & Development Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark Build and optimise ETL/ELT solutions processing large-scale datasets Develop high-quality, reusable code and engineering frameworks Deliver robust, performant and maintainable data ...

doing Developing and maintaining Power BI reports, dashboards and semantic models Designing and improving ETL and ELT pipelines using SQL, Python and, where appropriate, PySpark or Spark Working with Microsoft Fabric, Azure data services, data warehouses, lakehouses and data lakes Translating business requirements into practical BI and data solutions … stakeholders It would be beneficial if you also have Knowledge of data modelling practices, including star and snowflake schemas and normalisation Experience with Python, PySpark or Spark, Azure, DevOps or Microsoft Dynamics 365 Finance and Operations An understanding of business operations A full driving licence, as travel to Group ...

Choose a partner with intimate knowledge of your industry and first-hand experience of defining its future.Your locationYour locationIndustriesChoose a partner with intimate knowledge of your industry and first-hand experience of defining its future.Birmingham ...

cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, machine ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query ...

Machine Learning – Understanding of AI/ML concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query ...

months Location: Northampton, UK --------------------------------------- JOB DETAILS Role Title: Sr Data Engineer • Minimum 6-8 years of experience working as Sr Data Engineer • Bigdata Technologies – PySpark, Spark • Coding Languages – Python, SQL • AWS services – Glue, Cloud formation, Lake Formation, IAM, Lambda, Redshift • Exposure to cloud technologies – Snowflake, Redshift, Apache Flink ...

Experience debugging moderately complex systems and improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code ...

troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental data loadingCDC (Change Data Capture) implementationData quality controls, schema evolution, reconciliation, and error ...

successful candidate will design and implement modern data lakehouse architectures, ensuring high performance and scalability. Key responsibilities include developing data pipelines using Python and PySpark, working with AWS services, and implementing CI/CD pipelines. Strong experience in AWS Data Engineering and knowledge of data quality frameworks are essential. ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

experience with Axiom (plus) and regulatory reporting flows (plus) Excellent communication; proven stakeholder engagement Ability to learn/adapt to new technologies Skilled PySpark, HDFS, SQL, Airflow, data modeling, and ETL processing. Familiarity with GitLab and Jupyter Notebook is preferred, and knowledge of Java will be considered an added ...

coding languages eg. Python, R, Scala, etc.; (Python preferred) Proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques ...

live. Strong Python and SQL skills, with experience writing version-controlled, tested and maintainable code. Experience with the Python geospatial ecosystem and working with PySpark on Databricks. Working knowledge of dbt, MLflow and production model deployment patterns . Confidence using AI-assisted development tooling appropriately as part ...

coding languages eg. Python, R, Scala, etc.; (Python preferred) Proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques ...

frame problems and present back to drive growth. Extensive experience working with large-scale Customer orientated datasets. Application of key analytical languages - SQL, PySpark, Python for analytics and automation. Power BI experience – Visualisation, DAX. Experience of shaping analytical direction, defining how success should be measured and developing new metrics ...