76 to 100 of 165 PySpark Jobs

Data Engineering Consultant

Hiring Organisation
Harnham - Data & Analytics Recruitment
Location
London, South East, England, United Kingdom
Employment Type
Full-Time
Salary
£65,000 - £75,000 per annum
complex operational challenges. You'll design and implement scalable generative AI workflows, often using technologies like Palantir AIP, while building robust data pipelines with PySpark, Python, and SQL. A key responsibility is executing sophisticated data integrations across distributed systems and enterprise environments, including ERPs and CRMs. You'll collaborate ...

Senior AWS Data Engineer opening in Columbus

Hiring Organisation
Robotics technology LLC
Location
Columbus, Ohio, United States
Employment Type
Any
Salary
USD Annual
Summary We are seeking a highly skilled Senior Data Engineer with strong expertise inAWS cloud services, Python, and PySpark to join our team in Columbus, OH. The ideal candidate will design, build, and optimize scalable data pipelines, ensuring high data quality, performance, and reliability. This role requires hands … Data Engineer. Strong hands-on expertise in AWS Data Services: Glue, EMR, Redshift, Athena, S3, Lambda, Step Functions, CloudWatch. Proficiency in Python and PySpark for ETL, transformation, and automation. Strong experience in SQL and data modeling (star schema, snowflake schema, dimensional models Experience building scalable, high-performance data pipelines ...

AWS Data Engineer

Hiring Organisation
Robotics technology LLC
Location
Dallas, Texas, United States
Employment Type
Any
Salary
USD Annual
designing, building, and deploying production-level data pipelines using tools from AWS Glue, Lamda, Kinesis using databases Aurora and Redshift.Experience with Spark programming (pyspark or scalaHands on experience with AWS components like (EMR, S3, Redshift, Lamdba, API Gateway, Kinesis ) in production environmentsStrong analytical skills and advanced SQL knowledge, indexing ...

L3 Data Engineer - Support

Hiring Organisation
Stackstudio Digital Ltd
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
GBP 250 - 270 Daily
Services : S3, Redshift, Glue, Athena, Lake Formation, IAM Data Engineering/ETL: Building and optimizing ETL pipelines Data ingestion, transformation & orchestration using AWS Glue (PySpark/Python) Worki click apply for full job details ...

Data Scientist - Contract - 12 months

Hiring Organisation
Randstad Technologies Recruitment
Location
London, United Kingdom
Employment Type
Contract
Contract Rate
£500 - £700/day Dependent on Experience
Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

Data Scientist - Contract - 12 months

Hiring Organisation
Randstad Technologies
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£500 - £700 per day
Dependent on experience. Required experience will include: Strong Python Programming skills. Strong design and development skills using Python. Python's libraries for machine learning: PySpark, Pytorch, scikit-learn and pandas. Technical experience using Databricks and MLFlow (Other Platforms can be considered). Good understanding of software code review standards ...

Senior Data Analyst with AI

Hiring Organisation
Robotics technology LLC
Location
Irving, Texas, United States
Employment Type
Any
Salary
USD Annual
visualize data specifically around experimental design, KPI calculation, and A/B testing. 4+ years of experience in manipulating big data using Python, PySpark, or SQL. Expert Experience with data visualization tools in Python, PowerBI, etc. Bachelors Degree or higher in Computer Science/Engineering/Math, or relevant ...

Machine Learning Engineer

Hiring Organisation
DXC Technology
Location
Bishopton, Renfrewshire, Scotland, United Kingdom
Employment Type
Permanent
TensorFlow Serving, TorchServe, ONNX, and TensorRT. Build and manage ML pipelines using MLflow, Kubeflow, and Azure ML Pipelines. Work with large-scale data using PySpark and integrate models into production environments. Monitor model performance and retrain as needed to ensure accuracy and efficiency. Collaborate with cross-functional teams ...

Lead Analytics Engineer

Hiring Organisation
Datatech
Location
Wimbledon, United Kingdom
Employment Type
Permanent
quality and structure of data ·Translate business needs into clear technical plans Experience required: ·Strong skills in data modelling ·SQL with either Python or PySpark ·Databricks or similar cloud data platform ·Experience communicating with both technical and non-technical users ·Leadership or mentoring experience ·Insurance or regulated sector experience ...

Lead Analytics Engineer

Hiring Organisation
Datatech Analytics
Location
South West London, London, United Kingdom
Employment Type
Permanent
quality and structure of data Translate business needs into clear technical plans Experience required: Strong skills in data modelling SQL with either Python or PySpark Databricks or similar cloud data platform Experience communicating with both technical and non-technical users Leadership or mentoring experience Insurance or regulated sector experience ...

Cloud Data Engineer

Hiring Organisation
Accenture
Location
Leeds, West Yorkshire, England, United Kingdom
Employment Type
Full-Time
Salary
Competitive salary
data for critical client projects. How to develop and enhance your knowledge of agile ways of working and working in open source stack (PySpark/PySql). Quality engineering professionals utilise Accenture delivery assets to plan and implement quality initiatives to ensure solution quality throughout delivery. As a Data ...

Lead Data Engineer - Nottingham City

Hiring Organisation
Nottingham Building Society
Location
Nottingham, UK
Employment Type
Full-time
strategy. About you - Extensive Technical Expertise: Strong knowledge of Microsoft Fabric components including OneLake, Lakehouse/Warehouse, Delta Lake, Direct Lake, Data Factory, Spark (PySpark/Scala) and Power BI (DAX and semantic modelling). Advanced Programming and Data Engineering Skills: Proficient in Python, SQL and T-SQL with … experience in PySpark; familiarity with KQL for real-time analytics and robust ELT design using medallion architectures. Proven Experience in Modern Data Platforms: Track record of building and operating Azure-based data platforms, ideally in greenfield or transformational contexts, with integration across on-premise, cloud and third-party sources. ...

Data Scientist Financial Services Domain

Hiring Organisation
Bitsoft International, Inc
Location
King Of Prussia, Pennsylvania, United States
Employment Type
Any
Salary
USD Annual
domain, specializing in building scalable data models and delivering actionable insights for risk management, payments, and customer analytics. Skilled in Python (pandas, scikit-learn, PySpark), SQL, and data engineering pipelines to enable predictive and prescriptive analytics. Adept at working with large financial datasets, ensuring data quality, and collaborating with … business decisions. Demonstrated success in developing machine learning models for fraud detection, credit scoring, and customer segmentation. Key Skills: Languages/Tools: Python, SQL, PySpark, R, SAS ML/AI: Scikit-learn, TensorFlow, NLP, Time-Series Forecasting Data Engineering: ETL, Data Wrangling, Data Pipelines Cloud Platforms: AWS, Azure ...

Palantir Data Technical Lead / Practice Lead

Hiring Organisation
Opus Recruitment Solutions Ltd
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£600 - £700 per day
engineers and influence data architecture across high-consequence programmes Build and optimise ontologies, pipelines, and semantic models Apply strong data engineering skills (Python, SQL, PySpark) Bring cloud expertise (AWS, Azure, or GCP) and proven team leadership If you have Palantir Foundry experience and want to take ownership of delivery ...

Data engineer

Hiring Organisation
Sanderson Government and Defence
Location
City, London, United Kingdom
Employment Type
Contract
Contract Rate
GBP 500 - 525 Daily
/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines click apply for full job details ...

Data engineer

Hiring Organisation
Sanderson Government and Defence
Location
City of London, London, United Kingdom
Employment Type
Contract
Contract Rate
£500 - £525 per day + Inside IR35
/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven ...

Data engineer

Hiring Organisation
Sanderson
Location
London, South East, England, United Kingdom
Employment Type
Contractor
Contract Rate
£500 - £525 per day, Negotiable, Inc benefits
/d Contract length: 6 months Base Location: London - 3/d a week Skills required: - Azure/Azure Data bricks - ADF/ETL - Pyspark/Scala Experience: Experience working on large data sets, and complex data pipelines. Understanding of Data Architecture and Design, and Data pipeline optimisation. Proven ...

Data Engineer

Hiring Organisation
Head Resourcing
Location
Glasgow, City of Glasgow, United Kingdom
Employment Type
Permanent
Salary
£45000 - £55000/annum
Sales, Customer Care and Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze ? Silver ? Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee … Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines, CI/CD Analytics: Power BI, Fabric What We're Looking For Experience Commercial and proven ...

Data Engineer

Hiring Organisation
Head Resourcing
Location
Glasgow, Lanarkshire, Scotland, United Kingdom
Employment Type
Full-Time
Salary
£45,000 - £55,000 per annum
Sales, Customer Care and Logistics. What You'll Do Lakehouse Engineering (Azure + Databricks) Build and maintain scalable ELT pipelines using Lakeflow Declarative Pipelines, PySpark and Spark SQL. Work within a Medallion architecture (Bronze Silver Gold) to deliver reliable, high-quality datasets. Ingest data from multiple sources including ChargeBee … Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines, CI/CD Analytics: Power BI, Fabric What We're Looking For Experience Commercial and proven ...

Senior Database & Cloud Engineer

Hiring Organisation
Conferma
Location
Manchester, UK
. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

Senior Database Engineer

Hiring Organisation
Conferma Ltd
Location
Manchester, North West, United Kingdom
Employment Type
Permanent
Salary
£70,000
. Maintain/implement data warehousing solutions and manage large-scale data storage systems (e.g. Microsoft Fabric) Build and optimise SQL queries, stored procedures, PySpark notebooks and database objects to ensure data performance and reliability. Migrate and modernise legacy databases to cloud-based architectures. Database Administration Administer, monitor … normalisation, indexing, query optimisation). Strong experience with ETL/ELT tools. E.g. Azure Data Factory, Databricks, Synapse Pipelines, SSIS, etc. Experience with Python, PySpark, or Scala for data processing. Familiarity with CI/CD practices. Experience with Data lake, Data warehouse and Medallion architectures. Understanding of API integrations ...

Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL)

Hiring Organisation
Capital One
Location
Cambridge, Massachusetts, United States
Employment Type
Permanent
Salary
USD Annual
Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part ...

Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL)

Hiring Organisation
Capital One
Location
York, Pennsylvania, United States
Employment Type
Permanent
Salary
USD Annual
Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part ...

Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL)

Hiring Organisation
Capital One
Location
Richmond, Virginia, United States
Employment Type
Permanent
Salary
USD Annual
Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part ...

Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL)

Hiring Organisation
Capital One
Location
Washington, Washington DC, United States
Employment Type
Permanent
Salary
USD Annual
Senior Data Engineer (PySpark/Spark, AWS, Databricks, EMR, ETL/SQL) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part ...