201 to 225 of 394 PySpark Jobs

seeking a skilled data engineer to enhance data processing frameworks and support Azure Databricks solutions. The ideal candidate will have extensive experience with PySpark, AWS, and strong programming skills in Python and SQL. Responsibilities include collaborating with architecture teams and delivering high-quality solutions while managing stakeholder expectations. This ...

successful candidate should have strong experience with AWS and/or GCP , alongside relevant data technologies such as SQL, Python, Power BI, and PySpark . Responsibilities will also include working closely with stakeholders, technical teams, and delivery partners to ensure solutions are secure, scalable, cost-effective, and aligned with ...

scale or time-series datasets Strong programming skills in Python, Rust and/or C++ Experience with tools such as Pandas, Polars, Dask or PySpark Experience building data platforms, ETL systems, data lakes, warehouses or lakehouse architectures Knowledge of Parquet, Arrow or similar columnar formats Experience with distributed systems ...

Physics, Statistics, or a related quantitative field. Core Technical Stack: Hands-on proficiency with Python, SQL, and Excel, alongside exposure to platforms like Databricks, PySpark, Palantir Foundry, or R. Specialized Experience: Proven track record in delivering advanced analytics, algorithm design, operational research, or big data engineering solutions. Energy & Utilities ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

instruments and market data symbology, with an interest in developing further expertise in these areas Proficiency with SQL and Python for exploratory data analysis (PySpark, pandas, Jupyter Notebooks) Ability to document and clearly explain complex data analysis findings Exceptional attention to detail and effective communication skills Ability to think ...

similarly complex digital platforms Our technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More ...

production settings Experience with multimodal modelling (e.g. text + image) Familiarity with human-in-the-loop systems or moderation workflows Experience with Databricks, PySpark, or large-scale data processing systems Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with Bupa Subsidised counselling and coaching with ...

able to translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) 26 days holiday (increasing with service) Most companies lose the detail. A large mobile network generates ...

Group Data & AI Office. The role focuses on hands-on data engineering, designing and building scalable data pipelines and data products using Databricks, PySpark and Python. You will collaborate with engineering teams, contribute to governance and standards, and mentor colleagues while delivering high-quality data solutions across the organisation. ...

Private equity-backed organisation with ambitious growth plans Essential Skills & Experience Strong experience as a Data Engineer within modern cloud data environments Expertise in PySpark, SQL and data transformation practices Experience building production-grade data pipelines Knowledge of Microsoft Fabric Experience working with Delta Lake and Lakehouse architectures ...

develop, deploy, and optimize scalable data solutions on Microsoft Azure. The ideal candidate will bring strong hands-on expertise in Azure Databricks, Apache Spark, PySpark, Python, and SQL, with proven experience building production-grade data pipelines, implementing governance through Unity Catalog, and automating deployments using GitLab-based CI/… Databricks Asset Bundles. Key Responsibilities Design, build, test, and maintain scalable batch and streaming data pipelines using Azure Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake. Develop reusable ETL and ELT frameworks for data ingestion, transformation, validation, and publishing across lakehouse layers. Design and manage Delta Tables, including ...

solutions, ensuring alignment with business priorities and best practice Implementing robust ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines and Azure Databricks (PySpark/Spark SQL) to support batch and streaming data processing Designing and delivering real-time and batch data ingestion solutions using Azure Event Hubs … Synapse Analytics, Azure SQL, Azure Data Lake, Datra bricks, Purview and Azure App Service is essential Proficiency in Python/SQL/SparkSQL/PySpark is essential Strong experience of Azure delivery experience including analysis/development/configuration/deployments essential Expert level technical skill ...

data platform while delivering scalable, secure, and high-quality data solutions. What You'll Do Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies. Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple … that support organisational growth and strategic decision-making. What You'll Bring Proven experience designing and delivering solutions within Azure Databricks environments. Strong Python, PySpark, SQL, and T-SQL development skills. Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms. Strong understanding ...

data platform while delivering scalable, secure, and high-quality data solutions. What You'll Do Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies. Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple … that support organisational growth and strategic decision-making. What You'll Bring Proven experience designing and delivering solutions within Azure Databricks environments. Strong Python, PySpark, SQL, and T-SQL development skills. Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms. Strong understanding ...

About the Opportunity Sun Chemical is seeking an experienced Data Architect II to join our Corporate IT team and drive the modernization of our enterprise data ecosystem. This is a unique opportunity to architect cutting ...

data platforms. Responsibilities Design, develop and maintain scalable data pipelines using Microsoft Fabric, Azure Databricks and Azure Synapse. Develop ETL/ELT solutions using PySpark and SQL. Build and optimise data integration solutions using Azure Data Factory and Microsoft Fabric. Design scalable lakehouse and data warehouse solutions. Optimise data … engineering standards. Troubleshoot production issues and continuously improve platform performance. Requirements Strong commercial experience as a Data Engineer (5+ years). Strong experience with PySpark . Advanced SQL skills. Experience with Microsoft Fabric , Azure Databricks and/or Azure Synapse Analytics . Experience building data pipelines using Azure Data ...

Phoenix enables digital transformation across the UK public sector, empowering organisations to innovate with cloud and hybrid infrastructures, data, AI, security and collaboration technologies. We are now hiring a Technical Consultant - Data & AI to join ...

analyse trends to help teams derive value from data. The role involves working with SQL and Python, plus Power BI or Tableau, with PySpark, SparkSQL, and Git experience considered a bonus. The position is permanent, offering £50,000 to £65,000 plus a bonus, and is mainly remote with ...

Data Engineer in the United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. ...

Pharos Resource Partners in the United Kingdom is seeking an experienced Data Engineer to design, build and operate data pipelines and models powering analytics across Investment teams and back-office functions. You will own data ...

attacker infrastructure, including account takeover, card testing, and credential stuffing. Hands-on experience with large-scale data-processing platforms, such as Databricks, Trino, or PySpark, to monitor and measure control performance across distributed systems. Experience incubating software features, establishing clear operational handoff criteria, and transitioning ownership to partner engineering ...

latest in AI to optimize processes, improve decision-making, and deliver measurable value. Job Details End-to-end product life cycle Python Spark/PySpark SQL/MySQL/SparkSQL Tableau Databricks AWS Glue Linear Regression - GLM - Random Forrest GenAI/LLM No sponsorship or C2C Interested in hearing ...

process improvements and automation Strong Python development skills (including Pandas and related data-processing libraries). Experience with big data technologies such as Spark, PySpark, Hadoop, and Hive Exposure to quantitative modelling or financial modelling is a plus but not required. Skills that will help: Global Risk Management experience ...

development of scalable data pipelines and to integrate diverse data sources into a cloud-based platform. You will guide data processing using Python, PySpark, and SQL within Azure and Kubernetes environments. The role emphasizes leadership, collaboration with stakeholders, and mentoring junior engineers while delivering robust data solutions ...