1 to 25 of 50 PySpark Jobs in England

understand consumers. Hands-on data engineering/development experience, preferably in a cloud/big data environment Skilled in at least one of Python, PySpark, SQL or similar Experience in guiding or managing roles in insight or data functions, delivering data projects and insight to inspire action and drive more »

month contract. Essential Skills: Insurance/Financial Services experience is highly desirable Design efficient and scalable data models. Azure Databricks (preferably), SQL, Python, PySpark, dimensional/star schema data modelling. Understanding of Conceptual, Logical and Physical Data Models Experience in ETL/ELT as well as Entity Relationship (ER more »

the Financial Services with experience in..... Statistical Models, Computer Vision, Predictive Analytics, Data Visualization, Large Language Models (LLM), NLP, AI, Machine Learning, MLOPs Python, Pyspark, Azure, Agile, MetaBase, then please apply. You can 📧 your cv to matt@hawksworthuk.com or message me on LinkedIn. Ideally you'll have plenty of more »

Must have experience of leading a team- Must also be hands on Apache JMeter with Groovy scripting TestCafe (Node Package Manager) – JavaScript/TypeScript PySpark – Python and Spark AWS Elastic Kubernetes Service (EKS) and Docker setup, administration, configuration AWS PostGres & Grafana setup, administration and configuration to store and visualise more »

A 6-month initial contract working on a range of greenfield projects, within a systematic trading firm based in the City. This is an Outside IR35 Position, offering circa £700 per day. You’re going to be playing a crucial more »

a generous benefits package. Technical Experience Required: Demonstrated expertise in data engineering with a focus on Azure services. Proficiency in SQL, Azure Databricks, and PySpark, handling unstructured/semi-structured data, with experience of schema evolution and/or serialisation. Extensive experience in building and optimizing data pipelines. Experience more »

enterprise strategy and overall industry trends. Big Data Technologies such as Hadoop, Spark, Kafka, etc. Hadoop: 5+ years Kafka: 3+ years Spark: 4+ years PySpark: 3+ years more »

Ideal Candidate Profile: Proven track record in big data engineering with a solid understanding of ETL pipelines and data system projections. Proficiency in Python, PySpark, SQL, and familiarity with data science tools like R, ML, AI. Strong foundation in database management (SQL and No-SQL databases such as Aurora more »

months to begin with & its extendable Location: Sheffield, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues. Skills: Spark Architecture – component understanding around Spark … Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans. Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations. Spark SME – Be able to review PySpark and Spark SQL jobs and make performance improvement recommendations. Spark – SME Be able … are Cluster level failures. Cloudera (CDP) – Knowledge of understanding how Cloudera Spark is set up and how the run time libraries are used by PySpark code. Prophecy – High level understanding of Low-Code No-Code prophecy set up and its use to generate PySpark code. more »

is seeking Azure Big Data Architect in Bristol, UK. This is a hybrid role. Must have skills: strong hands on technical Skills in Python, Pyspark, Azure Databricks, Spark, ETL, Cloud Azure Good to have knowledge on ADF CI CD Description: Extensive experience working with large data sets with hands … on technology skills to design and build robust Big Data solutions using Spark PySpark framework and Industry standard frameworks like Databricks Azure DevOps and other tools technologies on Azure Cloud Platform Key Roles and Responsibilities: Understand the requirement and analyze the source system data to come up with a … high performance Data Architecture Good to have the Retail functional Knowledge Must have Good Knowledge and Hands on experience in Python PySpark ADF and ADB Good to have Knowledge in ADF CI CD Experience in designing architecting and implementing large scale data processing data storage data distribution systems Ability more »

of this is a strong preference. However other Cloud platforms like AWS/GCP are acceptable. · Coding Languages - Experience using Python with data (Pandas, PySpark) would be an advantage. Other languages such as C# would be beneficial but not essential. · Machine Learning and Data Science Tools - Any experience in more »

modern NLP methods required. Specifically: Transformers models (ex. BERT), LLMs, RAG & Fine-Tuning, OpenAI Stack, Langchain etc. Experience with Big Data technologies a plus —PySpark, H20.ai, Cloud AI platforms, Kubernetes Must be able to translate business requirements into analytical problems Must have proven ability to merge and transform disparate more »

understand the benefits, pros and cons of various technical options. Required skills: Strong experience within a Data Engineering role Excellent understanding of Databricks and Pyspark Strong knowledge of Azure Cloud Services Excellent understanding of SQL Good exposure to Azure Data Lake technologies such as ADF, HDFS and Synapse Good more »

London, UK Do you strive for engineering excellence and are ready to lead a modern tech transformation ? Do you have experience as a Azure, Pyspark, Cloud? We have the perfect role for you- Data Engineer - Tech Lead Careers at TCS: It means more TCS is a purpose-led transformation … design, development, and maintenance of Azure-based data pipelines and analytical solutions using Databricks, Synapse, and other relevant services. Perform Data Transformations: Leverage SQL, PySpark, and Python to perform data transformations, aggregations, and analysis on large datasets. Architect Data Storage Solutions: Architect data storage solutions using Azure SQL Database more »

Principal Data Engineer – National Salary Circa £80,000. London Salary Circa £100,000. Do you like working with the latest technology and are interested in enhancing your tech abilities? We have an exciting opportunity for an Engineer Manager with significant more »

the Insurance and technology sector, renown for its outstanding working culture, in a Data Engineering role. The role will require someone who has strong Pyspark and SQL experience alongside an Azure background. Key responsibilities: Develop low to high complexity, secure, governed, high quality, efficient data pipelinesfrom a variety of … and reporting & visualisation solutions in recognised BI tools such as PowerBI. Skills required: Strong experience with SQL server and/or Azure (Synapse, Datafactory) Pyspark Experience of designing and building end to end data solutions A good understanding of the full data lifecycle The ability to coach/mentor more »

I have placed quite a few candidates with this organisation now, all have said glowing reviews about it, they're the leader in legal representation comparison and have rather interesting, unique data to work with. They are using modern Azure more »

segmentation , and AI , we want to hear from you! At E.ON Next, you'll have the opportunity to leverage your skills in Databricks and PySpark to tackle operational and customer experience-related challenges, driving impactful solutions in a dynamic and collaborative environment. Join us in revolutionising the energy sector … stakeholders and fostering strong business partnerships. Leveraging predictive modeling, segmentation techniques, and advanced AI algorithms to unlock valuable insights. Demonstrating proficiency in Databricks and PySpark to streamline data processing and analysis. A taste of what you’ll be doing: ● Consultative Leadership: Build a strategic understanding of the business, employ … Strong communication skills with the ability to engage with non-technical stakeholders. ● Expertise in predictive modeling, segmentation, and AI techniques. ● Proficiency in Databricks and PySpark for data manipulation and analysis. ● Experience solving operational or customer experience-related problems such as workforce management, demand forecasting, or root cause analysis. ● BSc more »

Lead Data Engineer: We need some strong Data engineer profiles… they need good experience with Pyspark, Python, SQL, ADF and preferably Databricks experience Job description: Building new data pipelines and optimizing data flows using the Azure cloud stack. Building data products from scratch. Support Business Analysts and Data Architects more »

candidate needs experience and confidence to speak up when several teams can be involved in delivering the project. There is a massive emphasis on Pyspark and Databricks for this particular role. Technical Skills Required: Azure (ADF, Functions, Blob Storage, Data Lake Storage, Azure Data Bricks) Databricks Spark Delta Lake … SQL Python PySpark ADLS Day To Day Responsibilities: Extensive experience in designing, developing, and managing end-to-end data pipelines, ETL (Extract, Transform, Load), and ELT (Extract, Load, Transform) solutions. Maintains a proactive approach to staying updated with emerging technologies and a strong desire to continuously learn and adapt more »

to 3 years of experience required Degree in Statistics, Maths, Physics, Economics or similar field Programming skills (Python and SQL are a must have, Pyspark is recommended) Analytical Techniques and Technology Experience with and passion for connecting your work directly to the customer experience, making a real and tangible more »

continual improvement of their performance. REQUIRED SKILLS AND EXPERIENCE Proficiency in Python, including its associated data and machine learning packages such as numpy, pandas, pyspark, matplotlib, scikit-learn, Keras, TensorFlow, and more. Experience working with object-oriented programming languages. Experience in Forecasting and propensity. Expertise in utilizing SQL for more »

team might face. Skills and Experience Strong stakeholder management Experience implementing data strategy Strong experience managing Data Engineering teams Hands on experience working with Pyspark and Azure What's in it for you? £90,000 - £100,000 Please send your CV in for initial screening. Data Engineering Manager Desired more »

Are you looking for an exciting opportunity in Solution Architecture ? Are you passionate about everything cloud and data ? Join us as an AWS Cloud Data Platform Solution Architect Careers at TCS: It means more TCS is a purpose-led transformation more »

LEAD DATA ENGINEER REMOTE UP TO £75,000 + BENEFITS PERMANENT Are you an expert at using Pyspark and Databricks? Do you have prior experience working within the healthcare industry? This role is a fantastic opportunity to work with a large organisation on some massive and challenging projects. THE … Azure Platform. Implement legacy data into the new data platform. YOUR SKILLS AND EXPERIENCE Healthcare experience is preferable. Advanced knowledge and commercial experience using Pyspark and Databricks. Cloud Experience, preferably with Azure. Strong communication skills. THE BENEFITS A salary of up to £75,000. Fully remote. Excellent Healthcare. Great more »