176 to 200 of 321 Permanent PySpark Jobs

languages eg. Python, R, Scala, etc.(Python preferred) Strong proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies eg. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video,etc. Knowledge of machine learning modelling techniques ...

growth opportunities and helping expand a consulting business. A naturally curious mindset and a genuine passion for solving challenging problems. **Additional experience with Databricks, PySpark, R, Power BI or Palantir Foundry would be advantageous.** Why consider it? Continued development:** Dedicated training and certification budget. Variety:** Work on complex, interesting ...

fast-paced trading environment where insight is key to strategy Skills and experience you need as Data Analyst: Proficient in SQL & Python. PySpark & SparkSQL a bonus. Strong data viz skills using tools for presenting results such as Tableau or Power BI. Ability to write efficient code is essential ...

contract opportunity with a leading client in London.## Requirements* 8+ years of experience in Data Engineering* Strong expertise in Python and extensive experience with PySpark and distributed data processing* Expert-level knowledge of AWS services including EMR, Glue, Lambda, S3, Athena, Redshift, ECS/EKS, Step Functions, EventBridge … Science, Engineering, or a related field* Language(s): English (fluent)## Job Description* Design, develop, and maintain large-scale data pipelines using Python and PySpark* Build and optimise batch and near real-time data processing solutions* Design scalable data architectures on AWS* Develop ETL/ELT workflows for ingesting ...

Choose a partner with intimate knowledge of your industry and first-hand experience of defining its future.Your locationYour locationIndustriesChoose a partner with intimate knowledge of your industry and first-hand experience of defining its future.Birmingham ...

data platforms. Responsibilities Design, develop and maintain scalable data pipelines using Microsoft Fabric, Azure Databricks and Azure Synapse. Develop ETL/ELT solutions using PySpark and SQL. Build and optimise data integration solutions using Azure Data Factory and Microsoft Fabric. Design scalable lakehouse and data warehouse solutions. Optimise data … engineering standards. Troubleshoot production issues and continuously improve platform performance. Requirements Strong commercial experience as a Data Engineer (5+ years). Strong experience with PySpark . Advanced SQL skills. Experience with Microsoft Fabric , Azure Databricks and/or Azure Synapse Analytics . Experience building data pipelines using Azure Data ...

cloud‐native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands‐on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, #J ...

driving products. Collaborate with internal teams to understand requirements and build solutions with global geospatial data. Design and build data pipelines with PySpark and Databricks to enrich fleet data with mapping and road-network information at global scale. Build applications and datasets that support route planning and the preparation ...

data ecosystem. You will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J ...

Nova Hunte Ltd is seeking a Senior Data Engineer to design, build and maintain scalable data pipelines using Python and PySpark in a remote, long-term contract with a London-based client. You will work on AWS data services, data lakes, ETL/ELT, data modelling, and real-time ...

IR35 for initially 6 months. Hybrid to Glasgow (2 days a week in the office) £450 - £525 a day Technology: Python & SQL development (ideally PySpark) Databricks or similar cloud platforms Git, CI/CD, Azure DevOps (ideal) or GitHub Optimiser apps & battery storage systems (optional) Software quality assurance Unit ...

IR35 for initially 6 months. Hybrid to Glasgow (2 days a week in the office) 450 - 525 a day Technology: Python & SQL development (ideally PySpark) Databricks or similar cloud platforms Git, CI/CD, Azure DevOps (ideal) or GitHub Optimiser apps & battery storage systems (optional) Software quality assurance Unit ...

opportunities and shaping hypotheses through to data exploration, feature engineering, model development, and deployment. Working with large and complex datasets, you'll use Python, PySpark, SQL, and Azure Databricks to develop robust predictive models and analytical solutions. Just as importantly, you'll be able to explain your findings ...

embedded in the Ontology.- Collaborating with senior engineers on agent design, instructions, and evaluation using AIP's native features.· Strong technical skills in Python, PySpark, SQL, and data modelling.· Ability to work independently and engage directly with non-technical business users.· Strong problem-solving mindset and ownership of delivery. ...

unexpected backgrounds. Desirable Experience Any exposure to Palantir Foundry or AIP is a bonus, but not required - this is where you will build it PySpark or experience with large-scale data processing Experience with ERP, supply chain, finance or commercial data in an industrial setting Consultancy or startup experience ...

What We\’re Looking For Degree in Statistics, Mathematics, Physics, Economics, or a related quantitative field. Strong programming skills — Python and SQL are essential; PySpark experience is highly advantageous. Experience with version control tools (e.g. GIT). Solid understanding of analytical technologies, tools, and techniques. Excellent logical thinking ...

similarly complex digital platforms Our technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More ...

skills Excel proficiency Desirable but non-essential skills: Additional software engineering, cloud platform & data skills – (e.g. Linux, SQL, AWS (s3, CDK, Lambda, Glue, Athena), PySpark) Master's degree, PhD or other additional qualification WORKING AT ZENOBE We’re passionate about sustainability and are proud to offer Team Zenobē ...

production settings Experience with multimodal modelling (e.g. text + image) Familiarity with human-in-the-loop systems or moderation workflows Experience with Databricks, PySpark, or large-scale data processing systems Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with Bupa Subsidised counselling and coaching with ...

clients to gather requirements, embed CI/CD and IaC, and deliver cloud data solutions across Azure and AWS. Strong communication and hands-on PySpark/SQL are essential. #J-18808-Ljbffr ...

enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using:Apache Kafka Apache NiFi Apache Spark/PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment and reconciliation services Create canonical data models spanning multiple source systems … capabilities for compliance-critical processes Work closely with engineering, architecture, risk and operational stakeholders Software Engineering & Delivery Write clean, maintainable and testable Python and PySpark code Contribute to CI/CD pipelines and automated deployment processes Participate in code reviews and engineering standards initiatives Deliver solutions through Development ...

associated Azure technologies. Build scalable ETL/ELT pipelines to ingest, transform, and load data from multiple enterprise systems. Develop data transformations using PySpark, Spark SQL, SQL, and Python. Create and maintain curated datasets that support business intelligence, reporting, and analytics. Implement data quality, reconciliation, validation, and monitoring processes. … business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source systems. Knowledge of data modelling ...

small team of data engineers and own work end to end: talking to stakeholders, designing models, building and testing pipelines in Python and PySpark, and supporting BI. It's essentially remote, and the platform will grow from one business to the whole group, so there's plenty … conceptual, logical and physical data models for the new platform Building metadata-driven pipelines across Azure Synapse, Data Factory and PostgreSQL using Python and PySpark Migrating data from legacy reporting solutions onto the platform while protecting data integrity Releasing code through Azure DevOps and Git, and keeping documentation ...

Schedule: 40 hrs/week Location: United Kingdom (Remote) This opportunity offers the chance to work on complex data engineering initiatives focused on Databricks, PySpark, SQL and large-scale transaction and loyalty data pipelines. The role will support production-ready integration, migration stabilisation, orchestration, monitoring and data quality across … patterns, and operational handover into business-as-usual support Experience with integration testing, production validation and data completeness investigations Required Skills Strong SQL, Python, PySpark and Databricks pipeline engineering skills Hands-on ETL/ELT development across cloud-based data platforms Workflow orchestration and dependency management Idempotent processing, failure ...

modelling, cutover approach, reconciliation strategy and technical risk management. Design and build the real‐time data architecture using Amazon MSK/Apache Kafka and PySpark Structured Streaming (evaluating Apache Flink where it is the better fit), including event schema governance, exactly‐once semantics and stream‐to‐lakehouse integration. Define … Lambda, AWS S3, AWS Redshift, and Amazon EMR, with the ability to make and defend architectural decisions. Advanced proficiency in Python and SQL, including PySpark for large‐scale batch and streaming processing, and strong command of data modelling and data warehousing concepts. Production experience designing and operating streaming architectures ...