201 to 225 of 386 Permanent PySpark Jobs

instruments and market data symbology, with an interest in developing further expertise in these areas Proficiency with SQL and Python for exploratory data analysis (PySpark, pandas, Jupyter Notebooks) Ability to document and clearly explain complex data analysis findings Exceptional attention to detail and effective communication skills Ability to think ...

similarly complex digital platforms Our technology stack Python and associated ML/DS libraries (scikit-learn, NumPy, LightGBM, Pandas, LangChain/LangGraph, TensorFlow, etc.) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc MLOps/DevOps: Terraform, Docker, Airflow, MLFlow Monitoring and evaluation platforms including New Relic and Braintrust More ...

production settings Experience with multimodal modelling (e.g. text + image) Familiarity with human-in-the-loop systems or moderation workflows Experience with Databricks, PySpark, or large-scale data processing systems Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with Bupa Subsidised counselling and coaching with ...

able to translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) 26 days holiday (increasing with service) Most companies lose the detail. A large mobile network generates ...

Group Data & AI Office. The role focuses on hands-on data engineering, designing and building scalable data pipelines and data products using Databricks, PySpark and Python. You will collaborate with engineering teams, contribute to governance and standards, and mentor colleagues while delivering high-quality data solutions across the organisation. ...

Private equity-backed organisation with ambitious growth plans Essential Skills & Experience Strong experience as a Data Engineer within modern cloud data environments Expertise in PySpark, SQL and data transformation practices Experience building production-grade data pipelines Knowledge of Microsoft Fabric Experience working with Delta Lake and Lakehouse architectures ...

develop, deploy, and optimize scalable data solutions on Microsoft Azure. The ideal candidate will bring strong hands-on expertise in Azure Databricks, Apache Spark, PySpark, Python, and SQL, with proven experience building production-grade data pipelines, implementing governance through Unity Catalog, and automating deployments using GitLab-based CI/… Databricks Asset Bundles. Key Responsibilities Design, build, test, and maintain scalable batch and streaming data pipelines using Azure Databricks, Apache Spark, PySpark, Python, SQL, and Delta Lake. Develop reusable ETL and ELT frameworks for data ingestion, transformation, validation, and publishing across lakehouse layers. Design and manage Delta Tables, including ...

solutions, ensuring alignment with business priorities and best practice Implementing robust ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines and Azure Databricks (PySpark/Spark SQL) to support batch and streaming data processing Designing and delivering real-time and batch data ingestion solutions using Azure Event Hubs … Synapse Analytics, Azure SQL, Azure Data Lake, Datra bricks, Purview and Azure App Service is essential Proficiency in Python/SQL/SparkSQL/PySpark is essential Strong experience of Azure delivery experience including analysis/development/configuration/deployments essential Expert level technical skill ...

data platform while delivering scalable, secure, and high-quality data solutions. What You'll Do Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies. Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple … that support organisational growth and strategic decision-making. What You'll Bring Proven experience designing and delivering solutions within Azure Databricks environments. Strong Python, PySpark, SQL, and T-SQL development skills. Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms. Strong understanding ...

data platform while delivering scalable, secure, and high-quality data solutions. What You'll Do Design, develop, and optimise Azure Databricks solutions using PySpark, Python, SQL, and Azure-native technologies. Build and maintain enterprise-grade ETL/ELT pipelines supporting ingestion, transformation, CDC, validation, and data delivery across multiple … that support organisational growth and strategic decision-making. What You'll Bring Proven experience designing and delivering solutions within Azure Databricks environments. Strong Python, PySpark, SQL, and T-SQL development skills. Expertise building and supporting large-scale ETL/ELT pipelines and cloud-based data platforms. Strong understanding ...

About the Opportunity Sun Chemical is seeking an experienced Data Architect II to join our Corporate IT team and drive the modernization of our enterprise data ecosystem. This is a unique opportunity to architect cutting ...

data platforms. Responsibilities Design, develop and maintain scalable data pipelines using Microsoft Fabric, Azure Databricks and Azure Synapse. Develop ETL/ELT solutions using PySpark and SQL. Build and optimise data integration solutions using Azure Data Factory and Microsoft Fabric. Design scalable lakehouse and data warehouse solutions. Optimise data … engineering standards. Troubleshoot production issues and continuously improve platform performance. Requirements Strong commercial experience as a Data Engineer (5+ years). Strong experience with PySpark . Advanced SQL skills. Experience with Microsoft Fabric , Azure Databricks and/or Azure Synapse Analytics . Experience building data pipelines using Azure Data ...

Phoenix enables digital transformation across the UK public sector, empowering organisations to innovate with cloud and hybrid infrastructures, data, AI, security and collaboration technologies. We are now hiring a Technical Consultant - Data & AI to join ...

analyse trends to help teams derive value from data. The role involves working with SQL and Python, plus Power BI or Tableau, with PySpark, SparkSQL, and Git experience considered a bonus. The position is permanent, offering £50,000 to £65,000 plus a bonus, and is mainly remote with ...

Data Engineer in the United Kingdom to design, build, and maintain data pipelines on a modern AWS-native stack. You will use Spark/PySpark for distributed processing, Airflow for orchestration, and Terraform for infrastructure as code in a hybrid work arrangement with weekly travel to Newcastle office. ...

Pharos Resource Partners in the United Kingdom is seeking an experienced Data Engineer to design, build and operate data pipelines and models powering analytics across Investment teams and back-office functions. You will own data ...

attacker infrastructure, including account takeover, card testing, and credential stuffing. Hands-on experience with large-scale data-processing platforms, such as Databricks, Trino, or PySpark, to monitor and measure control performance across distributed systems. Experience incubating software features, establishing clear operational handoff criteria, and transitioning ownership to partner engineering ...

latest in AI to optimize processes, improve decision-making, and deliver measurable value. Job Details End-to-end product life cycle Python Spark/PySpark SQL/MySQL/SparkSQL Tableau Databricks AWS Glue Linear Regression - GLM - Random Forrest GenAI/LLM No sponsorship or C2C Interested in hearing ...

process improvements and automation Strong Python development skills (including Pandas and related data-processing libraries). Experience with big data technologies such as Spark, PySpark, Hadoop, and Hive Exposure to quantitative modelling or financial modelling is a plus but not required. Skills that will help: Global Risk Management experience ...

development of scalable data pipelines and to integrate diverse data sources into a cloud-based platform. You will guide data processing using Python, PySpark, and SQL within Azure and Kubernetes environments. The role emphasizes leadership, collaboration with stakeholders, and mentoring junior engineers while delivering robust data solutions ...

Analytics (DSA) for the Internal Audit function. You will design and implement production-grade data pipelines and data products, using Python (Pandas, Polars, PySpark), SQL, and modern data-lake architectures to support third-line risks. You will collaborate with audit teams and data owners to source, transform and curate ...

design and implement automated test solutions, validate data accuracy, and collaborate with cross-functional teams across AWS/Azure environments. Familiarity with PySpark, Kafka, and CI/CD will help drive reusable testing frameworks and shift-left testing #J-18808-Ljbffr ...

across global teams from a remote UK setting. You will work with a broad stack including AWS Glue, Redshift, EMR, Kinesis, and Python/PySpark to deliver real business impact for Fortune 1000 #J-18808-Ljbffr ...

more value from their data. You’ll need strong SQL and Python skills, along with experience using Power BI or Tableau. Experience with PySpark, SparkSQL, Git or LLM and AI tools would be a bonus. Permanent role with an ASAP start, paying £50,000 to £65,000 depending ...

genomic data. You have A BSc or MSc in computer science or related fields Expertise in Python, including popular Python libraries: NumPy, Pandas, PySpark and frameworks: Django, Django Rest Framework, FastAPI Hands-on experience with both relational (e.g. PostgreSQL) and non-relational databases (e.g. Elasticsearch, Redis) Extensive experience ...