201 to 225 of 575 Apache Spark Jobs in the UK

drift monitoring; versioning and release management. Cloud architecture (Azure/AWS/GCP), Kubernetes/Docker, serverless, IAM and network security. Data engineering depth (Spark/Databricks; ETL/ELT); cloud‐native data + AI architectures. Enterprise integration and SRE principles (SLIs/SLOs, runbooks, rollback). Consulting leadership ...

application resiliency, and securityPractical cloud-native experiencePreferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or SnowflakeHands-on experience with Spark/PySpark and other big data processing technologiesDemonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence ...

learning, model development, and data engineering, with an appreciation of responsible AI principles. Experience with cloud environments (Azure, AWS, GCP) and tools such as Spark, Hive, and Redshift. Demonstrated ability to lead cross-functional teams and mentor junior practitioners. Ability to communicate complex technical concepts clearly to non-technical ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

application resiliency, and securityPractical cloud-native experiencePreferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or SnowflakeHands-on experience with Spark/PySpark and other big data processing technologiesDemonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial intelligence ...

security Practical cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial ...

such as MLflow, Airflow, model monitoring, API monitoring, data validation, data drift detection, autoscaling and access management. Strong Python experience, with helpful knowledge of Spark or PySpark for working with large‐scale data and machine learning systems. An understanding of feature stores and related data technologies used to support ...

Direct knowledge of Matillion’s Data Productivity Cloud/Matillion ETL, or primary competitive data integration platforms. Big Data & Analytics Ecosystems: Experience with Spark/PySpark, modern Data Lakes/Lakehouses, and MPP database architectures (Teradata, Vertica, Netezza). Education: Bachelor’s or Master’s degree in Computer Science ...

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business intelligence … distributed data processing and performance tuning Hands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

using Python, PySpark, and dbtBuild and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observabilityDevelop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformationsModel and transform data for analytics using SQL and dbt to support business intelligence … scale distributed data processing and performance tuningHands-on experience with modern data warehousing/lakehouse technologies (e.g., Redshift, BigQuery, Snowflake; and engines such as Spark, Flink, or Trino; and table formats such as Iceberg, Hudi, or similar)Strong SQL skills and experience with SQL-based transformation tooling (e.g. ...

help evolve platform standards and documentation What we’re looking for 3–5 years of data engineering experience Strong Python & SQL skills Experience with Apache Spark/PySpark Ability to monitor, troubleshoot, and resolve pipeline failures Familiarity with lakehouse concepts Experience with Git, Databricks, streaming/… Great Expectations Nice to have Exposure to YAML Experience building analytics data models in Spark What you’ll get in return Autonomy and ownership – we’ll set the vision and share the context, then we trust you to run with things – that’s why we hire you! Work alongside ...

this role leads the engineering of cloud-native data services for the firm's strategic platform, driving the migration to Databricks and delivering scalable Spark/Scala solutions on AWS.The overall objective of this role is to lead applications systems analysis and programming activities. Proven leadership in building … matching engines, AI/GenAI services and large-scale data platforms. This role will drive transformation of enterprise data capabilities using Databricks, scala/spark and cloud native architectures, ensuring platform is stable, scalable, resilient, secure and AI enabled. As platform landscape evolved towards cloud-native, AI enabled ...

restore methods, DDL generation and manipulation, database access management, schema and object-level high-performance data movement. Build solutions with pushdown optimization, ELT, Spark-based data processing and streaming, replication, partitioning, clustering, change data capture, and time travel. What you’ll bring to the role At least 4 years … scripts, with a particular focus on building and optimizing multi‐threaded, multi‐process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI‐powered solutions using Large Language Models (LLMs), retrieval ...

restore methods, DDL generation and manipulation, database access management, schema and object-level high-performance data movement. Build solutions with pushdown optimization, ELT, Spark-based data processing and streaming, replication, partitioning, clustering, change data capture, and time travel. What you'll bring to the role At least 4 years … scripts, with a particular focus on building and optimizing multi-threaded, multi-process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI-powered solutions using Large Language Models (LLMs), retrieval ...

security controls. Monitor, maintain and optimise data platforms to ensure performance and reliability. Skills and Experience Required: ETL development using Python or similar languages Apache Spark, NiFi, or Kafka Relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases AWS and cloud-based data services Palantir Foundry Stakeholder Engagement ...

Azure SQL Database, or Azure Data Factory Experience with Python or another object-oriented programming language Experience with Azure Databricks and/or Apache Spark Experience with reporting or visualisation tools such as Power BI, MicroStrategy, or similar platforms Experience with CI/CD workflows, including automated build ...

project stakeholders; Strong problem-solving skills; Ability to refine problem statements and propose solutions considering effort, impact, and scalability trade-offs; Nice to have: Apache Spark, Iceberg, Kafka, dbt, Scikit-Learn, XGBoost, PyTorch, MLFlow, GraphFrames, Ray, AWS S3, EMR, SageMaker, Lakeformation, Terraform, Docker, GitHub CI/CD, Knowledge ...

root‐cause analysis Experience building or supporting production data pipelines in a collaborative engineering environment Experience with distributed data processing frameworks such as Apache Spark Working knowledge of JSON, Avro and Parquet Nice to have: technical design ownership across multiple datasets or pipeline domains, experience guiding implementation standards ...

data technologies (e.g. Azure Cosmos DB, Azure SQL Database, Azure Data Factory) Python or another object‐oriented programming language Azure Databricks and/or Apache Spark Reporting or visualisation tools such as Power BI, MicroStrategy, or similar platforms CI/CD workflows, including automated build, test, and deployment ...

networking CI/CD The Atlassian Suite (JIRA, Confluence, Bitbucket, Bamboo) Azure Services Machine learning algorithms Big Data (Hadoop, Azure Data Lake, S3, Apache Spark) Additional programming languages – Python, Scala, Typescript, CSS 3/SASS (or other pre-processors) Other Information The successful candidate will be rewarded with ...

classification techniques, and algorithms Fluency in a programming language (Python, C,C++, Java, SQL) Familiarity with Big Data frameworks and visualization tools (Cassandra, Hadoop, Spark, Tableau) Desirable Skills & Experience Familiarity with MLOps, AI governance, and responsible AI practices. Experience with modern AI frameworks and tooling such as LangChain, Hugging ...

+ ETL/ELT experience Knowledge of Python/T-SQL Exposure to DBT Exposure to cloud platforms (Azure preferred) Nice to have: Databricks, Spark, CI/CD, big data This is a great opportunity to join a high-impact data team in a global organisation and shape ...

tools, and emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated ...

Ensure no significant commitments to other projects during this period. Qualifications Must‐Have Data Engineering : Experience with SQL , Python , ETL , ELT , and tools like Spark , Kafka , Airflow , dbt , Snowflake , BigQuery , or Databricks on AWS , GCP , or Azure . Sales : Experience as Account Executives, Account Managers, Customer Success Managers, SDRs ...

preparation. Machine learning and statistical methods, including model validation. Data analytics and visualisation techniques. Processing large datasets using batch or streaming tools such as Apache Spark. Data acquisition and fusion approaches. Cloud platforms (ideally AWS) and cloud-based data science solutions. DataOps principles and practices. Structured and unstructured databases. ...