201 to 225 of 344 PySpark Jobs

production settings Experience with multimodal modelling (e.g. text + image) Familiarity with human-in-the-loop systems or moderation workflows Experience with Databricks, PySpark, or large-scale data processing systems Additional Information Health + Mental Wellbeing PMI and cash plan healthcare access with Bupa Subsidised counselling and coaching with ...

Experience: 6-10+ Years Key Skills: MDM, Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Data Quality, Data Governance Role Summary We are looking for an experienced MDM/Databricks Data Engineer to build scalable master data and integration solutions on a modern Databricks Lakehouse click apply for full ...

Skills: MDM, Databricks, PySpark, SQL, Delta Lake, Unity Catalog, Data Quality, Data Governance Role Summary We are looking for an experienced MDM/Databricks Data Engineer to build scalable master data and integration solutions on a modern Databricks Lakehouse. The role will focus on common IDs, Golden Records, entity ...

clients to gather requirements, embed CI/CD and IaC, and deliver cloud data solutions across Azure and AWS. Strong communication and hands-on PySpark/SQL are essential. #J-18808-Ljbffr ...

enhance operational telemetry, monitoring and platform observability Streaming & Distributed Data Processing Build and optimise solutions using:Apache Kafka Apache NiFi Apache Spark/PySpark Hadoop ecosystem technologies Hive and large-scale data storage platforms Develop scalable matching, enrichment and reconciliation services Create canonical data models spanning multiple source systems … capabilities for compliance-critical processes Work closely with engineering, architecture, risk and operational stakeholders Software Engineering & Delivery Write clean, maintainable and testable Python and PySpark code Contribute to CI/CD pipelines and automated deployment processes Participate in code reviews and engineering standards initiatives Deliver solutions through Development ...

associated Azure technologies. Build scalable ETL/ELT pipelines to ingest, transform, and load data from multiple enterprise systems. Develop data transformations using PySpark, Spark SQL, SQL, and Python. Create and maintain curated datasets that support business intelligence, reporting, and analytics. Implement data quality, reconciliation, validation, and monitoring processes. … business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source systems. Knowledge of data modelling ...

small team of data engineers and own work end to end: talking to stakeholders, designing models, building and testing pipelines in Python and PySpark, and supporting BI. It's essentially remote, and the platform will grow from one business to the whole group, so there's plenty … conceptual, logical and physical data models for the new platform Building metadata-driven pipelines across Azure Synapse, Data Factory and PostgreSQL using Python and PySpark Migrating data from legacy reporting solutions onto the platform while protecting data integrity Releasing code through Azure DevOps and Git, and keeping documentation ...

Schedule: 40 hrs/week Location: United Kingdom (Remote) This opportunity offers the chance to work on complex data engineering initiatives focused on Databricks, PySpark, SQL and large-scale transaction and loyalty data pipelines. The role will support production-ready integration, migration stabilisation, orchestration, monitoring and data quality across … patterns, and operational handover into business-as-usual support Experience with integration testing, production validation and data completeness investigations Required Skills Strong SQL, Python, PySpark and Databricks pipeline engineering skills Hands-on ETL/ELT development across cloud-based data platforms Workflow orchestration and dependency management Idempotent processing, failure ...

modelling, cutover approach, reconciliation strategy and technical risk management. Design and build the real‐time data architecture using Amazon MSK/Apache Kafka and PySpark Structured Streaming (evaluating Apache Flink where it is the better fit), including event schema governance, exactly‐once semantics and stream‐to‐lakehouse integration. Define … Lambda, AWS S3, AWS Redshift, and Amazon EMR, with the ability to make and defend architectural decisions. Advanced proficiency in Python and SQL, including PySpark for large‐scale batch and streaming processing, and strong command of data modelling and data warehousing concepts. Production experience designing and operating streaming architectures ...

CreateFuture is seeking a Senior Data Engineer to lead production data pipelines built on Microsoft Fabric, Python, and PySpark. You will translate data requirements into robust, maintainable solutions while mentoring engineers and shaping architecture with ...

continuous integration/continuous delivery, application resiliency, and security Demonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, Databricks, PySpark, artificial intelligence, machine learning, mobile, etc.) Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding ...

Ceox is recruiting a Data Engineer to design and deliver scalable data pipelines and data models in a modern Azure-based environment. You will collaborate with architects, analysts and engineers to implement data flows aligned ...

production ML systems. You will collaborate with data scientists and ML engineers, mentor junior engineers, and elevate engineering practices. The role emphasises Python, PySpark, and platforms like Databricks, Snowflake, or AWS. You will design and maintain data pipelines, ensure reproducible, point-in-time datasets, and support model versioning ...

production ML systems. You will collaborate with data scientists and ML engineers, mentor junior engineers, and elevate engineering practices. The role emphasises Python, PySpark, and platforms like Databricks, Snowflake, or AWS. You will design and maintain data pipelines, ensure reproducible, point-in-time datasets, and support model versioning ...

production ML systems. You will collaborate with data scientists and ML engineers, mentor junior engineers, and elevate engineering practices. The role emphasises Python, PySpark, and platforms like Databricks, Snowflake, or AWS. You will design and maintain data pipelines, ensure reproducible, point-in-time datasets, and support model versioning ...

production ML systems. You will collaborate with data scientists and ML engineers, mentor junior engineers, and elevate engineering practices. The role emphasises Python, PySpark, and platforms like Databricks, Snowflake, or AWS. You will design and maintain data pipelines, ensure reproducible, point-in-time datasets, and support model versioning ...

Responsável por colaborar com analistas de negócios, produzir relatórios de conformidade e propor melhorias contínuas na plataforma. Requer experiência com Databricks, IaC, Git, Python (Pyspark) e SQL e vontade de aprender novas tecnologias. #J-18808-Ljbffr ...

embedded in the Ontology. - Collaborating with senior engineers on agent design, instructions, and evaluation using AIP's native features. Strong technical skills in Python, PySpark, SQL, and data modelling. Practical experience using or supporting AIP features such as RAG workflows, copilots, or agent-based applications. Ability to work independently ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Lead Client Advisory: Sit directly alongside clients, goals ...

Physics, Statistics, or a related quantitative field. Core Technical Stack: Hands-on proficiency with Python, SQL, and Excel, alongside exposure to platforms like Databricks, PySpark, Palantir Foundry, or R. Specialized Experience: Proven track record in delivering advanced analytics, algorithm design, operational research, or big data engineering solutions. Energy & Utilities ...

platform environments, ideally supporting production systems and external clients Advanced working knowledge of at least one data/query language (e.g. SQL, Python, or PySpark), with the ability to debug complex issues, analyse data, and build queries/scripts independently Deep experience working with data pipelines, transformations, and diagnosing ...

technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with an ethos of commitment ...

unlock business value. You will collaborate with data scientists, analysts and software engineers, implement governance and security, and contribute to engineering standards. Strong PySpark and SQL skills, plus experience with cloud platforms are essential for success. #J-18808-Ljbffr ...

seeking an experienced Data Engineer lead in Manchester to drive the next generation of its data estate. You will mentor junior engineers, deliver scalable PySpark pipelines on Databricks, and collaborate with product owners to shape data roadmaps. The role blends hands-on data engineering with technical leadership, governance practices ...

security and governance, and mentor engineers while partnering with product, security and analytics teams. Applicants should have 5+ years of experience with Snowflake, dbt, PySpark and AWS, and proven ability to lead #J-18808-Ljbffr ...