151 to 169 of 169 PySpark Jobs

Press Tab to Move to Skip to Content Link Select how often (in days) to receive an alert: For airports, for partners, for people. We are CAVU. At CAVU our purpose is to find new and better ways to make More ❯

Manager - Data Engineering Would you like to ensure the successful delivery of Data Platform and Software Innovations? Do you enjoy creating a collaborative and customer-focused working environment? A bout the Team : LexisNexis Intellectual Property, which serves customers in more More ❯

Mapbox is the leading real-time location platform for a new generation of location-aware businesses. Mapbox is the only platform that equips organizations with the full set of tools to power the navigation of people, packages, and vehicles everywhere. More ❯

and managing project changes and interventions to achieve project outputs. Documenting all aspects of the project for future reference and audits. Technical Responsibilities: Developing SQL scripts (store procedures) and PySpark notebooks. Creating and managing ingestion, ETL & ELT processes. Designing and configuring Synapse pipelines. Data modelling in various storage systems. Analysing existing data designs and suggesting improvements for performance, stability … Experience in Project Management within the Defence & Security sector. Strong technical skills in API, Java, Python, Web Development, SQL, and Azure. Proficiency in developing and managing SQL scripts and PySpark notebooks. Understanding of ETL & ELT processes and Synapse pipeline design and configuration. Experience in data modelling and improving existing data designs. Knowledge of real-time data processing. Capable of More ❯

and managing project changes and interventions to achieve project outputs. Documenting all aspects of the project for future reference and audits. Technical Responsibilities: Developing SQL scripts (store procedures) and PySpark notebooks. Creating and managing ingestion, ETL & ELT processes. Designing and configuring Synapse pipelines. Data modelling in various storage systems. Analysing existing data designs and suggesting improvements for performance, stability … Experience in Project Management within the Defence & Security sector. Strong technical skills in API, Java, Python, Web Development, SQL, and Azure. Proficiency in developing and managing SQL scripts and PySpark notebooks. Understanding of ETL & ELT processes and Synapse pipeline design and configuration. Experience in data modelling and improving existing data designs. Knowledge of real-time data processing. Capable of More ❯

practices. This is a fantastic opportunity for a curious, solutions-focused data scientist to help build out our capability, working with cutting-edge tools like Databricks, AWS data services, PySpark, and CI/CD pipelines. What's in it for you? You'll be joining a collaborative, supportive team with a real passion for data-led innovation. It's … business impact - we'd love to hear from you. About you: 2-5 years of experience in Data Science or a related field Strong programming skills in Python and PySpark Strong data science modelling skills across classification, regression, forecasting, and/or NLP Analytical mindset with the ability to present insights to both technical and non-technical audiences Experience More ❯

practices. This is a fantastic opportunity for a curious, solutions-focused data scientist to help build out our capability, working with cutting-edge tools like Databricks, AWS data services, PySpark, and CI/CD pipelines. What's in it for you? You'll be joining a collaborative, supportive team with a real passion for data-led innovation. It's … we can reach new heights. Together, we are CAVU. About You: 2-5 years of experience in Data Science or a related field Strong programming skills in Python and PySpark Strong data science modelling skills across classification, regression, forecasting, and/or NLP Analytical mindset with the ability to present insights to both technical and non-technical audiences Experience More ❯

teams to deliver robust, trusted, and timely data solutions that power advanced analytics and business intelligence. What You'll Do: Architect and build scalable data pipelines using Microsoft Fabric, PySpark, and T-SQL Lead the development of Star Schema Lakehouse tables to support BI and self-service analytics Collaborate with stakeholders to translate business needs into data models and … Mentor engineers and act as a technical leader within the team Ensure data integrity, compliance, and performance across the platform What You'll Bring: Expertise in Microsoft Fabric, Azure, PySpark, SparkSQL, and modern data engineering practices Strong experience with Lakehouse architectures, data orchestration, and real-time analytics A pragmatic, MVP-driven mindset with a passion for scalable, maintainable solutions More ❯

teams to deliver robust, trusted, and timely data solutions that power advanced analytics and business intelligence. What You'll Do: Architect and build scalable data pipelines using Microsoft Fabric, PySpark, and T-SQL Lead the development of Star Schema Lakehouse tables to support BI and self-service analytics Collaborate with stakeholders to translate business needs into data models and … Mentor engineers and act as a technical leader within the team Ensure data integrity, compliance, and performance across the platform What You'll Bring: Expertise in Microsoft Fabric, Azure, PySpark, SparkSQL, and modern data engineering practices Strong experience with Lakehouse architectures, data orchestration, and real-time analytics A pragmatic, MVP-driven mindset with a passion for scalable, maintainable solutions More ❯

teams to deliver robust, trusted, and timely data solutions that power advanced analytics and business intelligence. What You'll Do: Architect and build scalable data pipelines using Microsoft Fabric, PySpark, and T-SQL Lead the development of Star Schema Lakehouse tables to support BI and self-service analytics Collaborate with stakeholders to translate business needs into data models and … Mentor engineers and act as a technical leader within the team Ensure data integrity, compliance, and performance across the platform What You'll Bring: Expertise in Microsoft Fabric, Azure, PySpark, SparkSQL, and modern data engineering practices Strong experience with Lakehouse architectures, data orchestration, and real-time analytics A pragmatic, MVP-driven mindset with a passion for scalable, maintainable solutions More ❯

teams to deliver robust, trusted, and timely data solutions that power advanced analytics and business intelligence. What You'll Do: Architect and build scalable data pipelines using Microsoft Fabric, PySpark, and T-SQL Lead the development of Star Schema Lakehouse tables to support BI and self-service analytics Collaborate with stakeholders to translate business needs into data models and … Mentor engineers and act as a technical leader within the team Ensure data integrity, compliance, and performance across the platform What You'll Bring: Expertise in Microsoft Fabric, Azure, PySpark, SparkSQL, and modern data engineering practices Strong experience with Lakehouse architectures, data orchestration, and real-time analytics A pragmatic, MVP-driven mindset with a passion for scalable, maintainable solutions More ❯

Data Analyst (SQL, PySpark, Python) London - Hybrid (2 Days Onsite) Inside IR35 - Daily Rate Our client is seeking an experienced Data Analyst to support a key client engagement within their professional services division. This is an initial 1 year contract based in London , operating on a hybrid basis (2 days onsite per week) and falling inside IR35 . Key … Responsibilities: Independently manage data extraction, cleansing, and transformation activities Develop production-grade analytics code using SQL, PySpark, and Python Analyze large-scale datasets to generate actionable insights and visualizations Collaborate across technical and business teams to drive data-driven decision making Required Experience: 3–5 years' hands-on experience in data analysis, insight generation, and data visualization Proven expertise … in SQL and PySpark , with solid working knowledge of Python Background working with big data platforms , large datasets, and pipelines Strong analytical thinking and a collaborative, self-directed working style Experience in the Payments , FinTech , or financial services sector is highly desirable, though not mandatory More ❯

Data Analyst (SQL, PySpark, Python) London - Hybrid (2 Days Onsite) Inside IR35 - Daily Rate Our client is seeking an experienced Data Analyst to support a key client engagement within their professional services division. This is an initial 1 year contract based in London , operating on a hybrid basis (2 days onsite per week) and falling inside IR35 . Key … Responsibilities: Independently manage data extraction, cleansing, and transformation activities Develop production-grade analytics code using SQL, PySpark, and Python Analyze large-scale datasets to generate actionable insights and visualizations Collaborate across technical and business teams to drive data-driven decision making Required Experience: 3–5 years' hands-on experience in data analysis, insight generation, and data visualization Proven expertise … in SQL and PySpark , with solid working knowledge of Python Background working with big data platforms , large datasets, and pipelines Strong analytical thinking and a collaborative, self-directed working style Experience in the Payments , FinTech , or financial services sector is highly desirable, though not mandatory More ❯

time zone). Technical Business Analyst & Subject Matter Expert who will join the Data Solutions team. Must have a solid understanding of data engineering conecpets such as ETL pipeline, PySpark transformations & data lake. - ETL pipeline - Data lake - PySpark transformations - Public Financial Markets - Bachelors Degree in IT or Finance - Advanced SQL skill - Solid understanding of data modelling, including dimensional More ❯

the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance, scalability, and data governance needs. This includes work with complex … days per year for meaningful collaboration in either Leeds or Thame. Key Responsibilities Ensure end-to-end data quality, from raw ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to … management, metadata management, and wider data governance practices Help shape our approach to reliable data delivery for internal and external customers Skills & Experience Required Extensive hands-on experience with PySpark, including performance optimisation Deep working knowledge of Databricks (development, architecture, and operations) Proven experience working with Airflow for orchestration Proven track record in managing and securing PII data, with More ❯

Data Engineering Manager £110,000-£115,000 + 10% bonus ️Databricks, Snowflake, Terraform, Pyspark, Azure London, hybrid working (2 days in office) Leading property data & risk software company We are partnered with a leading property data & risk software company that contributes valuations, insights, and decisioning technology to over 1 million mortgage approvals each year. They are looking for a … visualization, and data modeling. Engage in projects that influence the company's bottom line. Drive the business forward by enabling better decision-making processes. Tech Stack : Databricks, Azure, Python, Pyspark, Terraform. What's in it for you 7.5% pension contribution by the company Discretionary annual bonus up to 10% of base salary 25 days annual leave plus extra days More ❯

code that is being written. Knowledge of the cloud software deployment process is also key, including familiarity with parallel processing in Python, large data processing using packages such as PySpark, and optimizing Python code for deployment as part of a containerized architecture. Role Highlights: Work with the Chief Engineer and Technical Leads to translate requirements into user stories to … Experience with interfacing with customer stakeholders to align on requirements and technical implementations Required Qualifications: Active Poly Minimum 3-5 years' experience with: Data Processing Python Libraries such as PySpark, Pandas and Numpy Experience with API development in Python using Python libraries such as FastAPI Experience with Unit Testing Frameworks in PyTest and Mocking Desired Qualifications: Experience with Python More ❯

TECHNICAL PROGRAMME MANAGER - DATA INGESTION (PHARMA/SNOWFLAKE) UP TO £560 PER DAY HYBRID (1/2 DAYS PER WEEK IN SPAIN & GERMANY) 6 MONTHS THE COMPANY: A global data and analytics consultancy are delivering a large-scale data ingestion More ❯

context Strong analytical mindset with the capability to structure complex business problems and translate them into statistically sound experimental designs and solutions Proficiency in programming with experience in Python, PySpark, R, or similar languages, sufficient to implement and test statistical models and simulations, with an emphasis on correctness over advanced coding skills Demonstrated ability to conduct independent research, quickly … design Proven expertise in hypothesis testing, design of experiments, and statistical methodologies critical to experimentation science Solid programming skills in at least one language such as Python, R, or PySpark, with the ability to write clean, maintainable code to support research and implementation Experience applying statistical rigor to real-world business problems, with the ability to independently lead and More ❯