126 to 150 of 250 PySpark Jobs in the UK

datasets for modelling and analysis. Strong background in NLP and GenAI techniques, including text classification, summarization, and embeddings. Advanced proficiency in SQL, Python, and PySpark, with a focus on software development best practises. Experience deploying data science models in a cloud environment (Azure preferred). Familiarity with database management ...

data modelling experience. Strong understanding of ETL/ELT architecture and data integration frameworks. AWS experience including services such as S3, Glue, EMR, and PySpark . Experience defining data standards, governance frameworks, and architectural roadmaps. Desirable Experience with Iceberg and modern lakehouse architectures. Exposure to AI-enabled data platforms ...

functional teams to deliver high-quality technical solutions. Support the ongoing maintenance and optimisation of platform services. Essential Skills & Experience Hands-on experience with PySpark for data transformation and processing. Experience working with AWS data services , including: AWS Glue Amazon S3 Experience using Terraform for Infrastructure as Code. Experience ...

hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well-governed data assets … data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics, you will help deliver reliable ...

also contribute to engineering excellence through best practices, mentoring, and thoughtful technical direction.Job responsibilitiesDesign scalable, reusable data processing and data quality frameworks using Python, PySpark, and dbtBuild and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow … data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence and reporting workloadsWrite production-grade Python/PySpark code with disciplined testing, performance tuning, and maintainable object-oriented designImplement infrastructure-as-code (e.g., Terraform) to provision and manage cloud-based data platform ...

implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models (bronze/silver/gold), data quality checks … Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT patterns, schema evolution, partitioning, Z-ORDER, OPTIMIZE/VACUUM ...

Senior Data Engineer Who we areArtefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the entire value chain ...

continuous integration/continuous delivery, application resiliency, and securityDemonstrated knowledge of software applications and technical processes within a technical discipline (e.g., cloud, Databricks, PySpark, artificial intelligence, machine learning, mobile, etc.)Hands-on experience using enterprise-authorized AI-assisted software development tools within the work environment (e.g., for coding, testing ...

reliable, accurate, and delivered in a timely manner.Write clean, efficient, and well-documented code and test cases using tools like Python, SQL, and Pyspark (optional).Understands containerization and Kubernetes deployment using ArgoCD. Collaborate Across TeamsPartner with Senior Engineers in code reviews, design discussions, planning sessions and team ceremonies, asking ...

Data Manager/Lead Location : London (Hybrid) Comp : £115,000 P/A (FTC) Duration: 12 Months Skills: Python (Programming Language), SQL, Apache Spark, PySpark, Azure Databricks, Data Lakes, Data Warehouse Architecture, Leadership, Software Development, Management The role We're partnering with a large, data-driven organisation seeking ...

experience using Pandas. Experience working with SQL and cloud-based data warehousing technologies. Knowledge of Google BigQuery would be highly beneficial. Exposure to PySpark and distributed data processing environments. Ability to analyse, transform, and optimise complex datasets. Strong problem-solving and stakeholder communication skills. How to Apply ...

passion for solving complex data challengesWhile we advocate using the right tech for the right task, we often leverage the following technologies: Python, PySpark, the PyData stack, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions such ...

Kubernetes Experience in creating production grade coding and SOLID programming principles, including test-driven development (TDD) approaches Experience in programming languages (e.g. Python, PySpark, R, SAS, SQL) Experience in source-control software, e.g., GitHub Proficient at communicating results in a concise manner both verbally and written Experience in data ...

workflow orchestration, and machine learning operations.Excellent communication skills with the ability to influence cross-functional teams and stakeholders.Experience with Python, SQL, Java, HQL, Hive, Pyspark, or similar programming languages preferred.Well-versed in design thinking principles.Great problem-solving and analytical skills. The total cash range for this position in Seattle ...

Declarative Pipelines with Expectations, including failure handling and observability Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns Production-level PySpark and SQL development skills Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM Experience with Databricks Workflows, orchestration ...

implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models (bronze/silver/gold), data quality checks … Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT patterns, schema evolution, partitioning, Z-ORDER, OPTIMIZE/VACUUM ...

data platforms. Responsibilities Design, develop and maintain scalable data pipelines using Microsoft Fabric, Azure Databricks and Azure Synapse. Develop ETL/ELT solutions using PySpark and SQL. Build and optimise data integration solutions using Azure Data Factory and Microsoft Fabric. Design scalable lakehouse and data warehouse solutions. Optimise data … engineering standards. Troubleshoot production issues and continuously improve platform performance. Requirements Strong commercial experience as a Data Engineer (5+ years). Strong experience with PySpark . Advanced SQL skills. Experience with Microsoft Fabric , Azure Databricks and/or Azure Synapse Analytics . Experience building data pipelines using Azure Data ...

data platforms and influencing architecture direction across multiple engineering and data teams. Key experience: Enterprise-scale Data Architecture AWS cloud data platforms Spark/PySpark/Python Airflow & ELT/data processing Data Lakehouse & Data Products Streaming/Event-Driven Architecture Data Governance Architecture standards, patterns & technical design reviews ...

junior engineers while translating Vision and Data Strategy into IT solutions and driving delivery across programs. The role requires hands-on work with Databricks, PySpark, Python and SQL, with a focus on data modeling, integration and cloud data services. #J-18808-Ljbffr ...

within the Microsoft ecosystem or Azure Exposure to tools such as Sparx Enterprise Architect or similar modelling tools Familiarity with technologies such as DBT, PySpark or Python is beneficial Interest in progressing into a data architecture focused role Ability to engage with both technical and non technical stakeholders APPLY ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACTClient Advisory & Translation: Sit directly alongside clients, distinguish their goals ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACTClient Advisory & Translation: Sit directly alongside clients, distinguish their goals ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark code repos - to building user-facing front-end applications and widgets in TypeScript or REACT Client Advisory & Translation: Sit directly alongside clients, distinguish their ...

ownership of client problems, handling everything from deep data engineering knowing that may require low/no code Palantir Foundry tooling, or Python and PySpark ark code repos - to building user-facing front-end applications and widgets in TypeScript or REACTClient Advisory & Translation: Sit directly alongside clients, distinguish their ...