176 to 200 of 344 PySpark Jobs

Experience debugging moderately complex systems and improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure‐as‐code ...

adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience supporting data processing and integration work using Python and/or PySpark, such as batch jobs, reconciliation pipelines, or platform integrations Experience with data engineering and orchestration tooling such as Databricks and Airflow or similar platforms Exposure ...

troubleshooting of Azure DatabricksDatabricks Serverless configuration and workload optimisationDatabricks SQL and Delta Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental data loadingCDC (Change Data Capture) implementationData quality controls, schema evolution, reconciliation, and error ...

STACK Infrastructure EMEA is seeking a Senior Databricks Data Engineer to build and manage the enterprise data platform, applying PySpark, Python, SQL and Delta Lake in production. You will design reliable pipelines across Finance, Delivery, Operations, HR and PMO, implement governance with Unity Catalog, and optimise performance and costs. ...

This 12-month, onsite contract (5 days/week) offers an ASAP or 4-week start and focuses on enterprise data lakehouse architectures, Python, PySpark, and dbt to deliver analytics-ready data products. You will collaborate with data architects and stakeholders to ensure data governance, quality, and cost efficiency ...

trends using Tableau, Power BI and AWS Connect Customer, collaborating with business stakeholders to productionise solutions. The role requires strong SQL and Python/PySpark skills, plus experience leading technical teams and mentoring #J-18808-Ljbffr ...

understanding of DAG (Directed Acyclic Graphs) and their role in pipeline orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic ...

languages eg. Python, R, Scala, etc.(Python preferred) Strong proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies eg. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video,etc. Knowledge of machine learning modelling techniques ...

growth opportunities and helping expand a consulting business. A naturally curious mindset and a genuine passion for solving challenging problems. **Additional experience with Databricks, PySpark, R, Power BI or Palantir Foundry would be advantageous.** Why consider it? Continued development:** Dedicated training and certification budget. Variety:** Work on complex, interesting ...

fast-paced trading environment where insight is key to strategy Skills and experience you need as Data Analyst: Proficient in SQL & Python. PySpark & SparkSQL a bonus. Strong data viz skills using tools for presenting results such as Tableau or Power BI. Ability to write efficient code is essential ...

contract opportunity with a leading client in London.## Requirements* 8+ years of experience in Data Engineering* Strong expertise in Python and extensive experience with PySpark and distributed data processing* Expert-level knowledge of AWS services including EMR, Glue, Lambda, S3, Athena, Redshift, ECS/EKS, Step Functions, EventBridge … Science, Engineering, or a related field* Language(s): English (fluent)## Job Description* Design, develop, and maintain large-scale data pipelines using Python and PySpark* Build and optimise batch and near real-time data processing solutions* Design scalable data architectures on AWS* Develop ETL/ELT workflows for ingesting ...

Choose a partner with intimate knowledge of your industry and first-hand experience of defining its future.Your locationYour locationIndustriesChoose a partner with intimate knowledge of your industry and first-hand experience of defining its future.Birmingham ...

data platforms. Responsibilities Design, develop and maintain scalable data pipelines using Microsoft Fabric, Azure Databricks and Azure Synapse. Develop ETL/ELT solutions using PySpark and SQL. Build and optimise data integration solutions using Azure Data Factory and Microsoft Fabric. Design scalable lakehouse and data warehouse solutions. Optimise data … engineering standards. Troubleshoot production issues and continuously improve platform performance. Requirements Strong commercial experience as a Data Engineer (5+ years). Strong experience with PySpark . Advanced SQL skills. Experience with Microsoft Fabric , Azure Databricks and/or Azure Synapse Analytics . Experience building data pipelines using Azure Data ...

cloud‐native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands‐on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence, #J ...

driving products. Collaborate with internal teams to understand requirements and build solutions with global geospatial data. Design and build data pipelines with PySpark and Databricks to enrich fleet data with mapping and road-network information at global scale. Build applications and datasets that support route planning and the preparation ...

data ecosystem. You will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J ...

Nova Hunte Ltd is seeking a Senior Data Engineer to design, build and maintain scalable data pipelines using Python and PySpark in a remote, long-term contract with a London-based client. You will work on AWS data services, data lakes, ETL/ELT, data modelling, and real-time ...

IR35 for initially 6 months. Hybrid to Glasgow (2 days a week in the office) £450 - £525 a day Technology: Python & SQL development (ideally PySpark) Databricks or similar cloud platforms Git, CI/CD, Azure DevOps (ideal) or GitHub Optimiser apps & battery storage systems (optional) Software quality assurance Unit ...

IR35 for initially 6 months. Hybrid to Glasgow (2 days a week in the office) 450 - 525 a day Technology: Python & SQL development (ideally PySpark) Databricks or similar cloud platforms Git, CI/CD, Azure DevOps (ideal) or GitHub Optimiser apps & battery storage systems (optional) Software quality assurance Unit ...

opportunities and shaping hypotheses through to data exploration, feature engineering, model development, and deployment. Working with large and complex datasets, you'll use Python, PySpark, SQL, and Azure Databricks to develop robust predictive models and analytical solutions. Just as importantly, you'll be able to explain your findings ...

embedded in the Ontology.- Collaborating with senior engineers on agent design, instructions, and evaluation using AIP's native features.· Strong technical skills in Python, PySpark, SQL, and data modelling.· Ability to work independently and engage directly with non-technical business users.· Strong problem-solving mindset and ownership of delivery. ...

unexpected backgrounds. Desirable Experience Any exposure to Palantir Foundry or AIP is a bonus, but not required - this is where you will build it PySpark or experience with large-scale data processing Experience with ERP, supply chain, finance or commercial data in an industrial setting Consultancy or startup experience ...

What We\’re Looking For Degree in Statistics, Mathematics, Physics, Economics, or a related quantitative field. Strong programming skills — Python and SQL are essential; PySpark experience is highly advantageous. Experience with version control tools (e.g. GIT). Solid understanding of analytical technologies, tools, and techniques. Excellent logical thinking ...

similarly complex digital platforms Our technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More ...

skills Excel proficiency Desirable but non-essential skills: Additional software engineering, cloud platform & data skills – (e.g. Linux, SQL, AWS (s3, CDK, Lambda, Glue, Athena), PySpark) Master's degree, PhD or other additional qualification WORKING AT ZENOBE We’re passionate about sustainability and are proud to offer Team Zenobē ...