151 to 175 of 259 Apache Spark Jobs in the UK

t. The ability to communicate model behaviour and limitations clearly to non‐technical stakeholders. Familiarity with data pipeline tooling (e.g. Airflow, dbt, Spark) and an understanding of how ML fits within a broader data platform. There’s always room to grow and learn with our roles — please ...

retrieval, and multi-modal representation learning. Technical Stack: Mastery of Python and deep learning frameworks (TensorFlow, PyTorch) alongside hands-on experience with distributed computing (Spark, Ray) and cloud infrastructure (AWS/GCP). Graph Neural Networks (GNNs): Strong experience applying GNNs, knowledge graphs, or graph embeddings to map complex ...

abuse detection, alongside a clear sense of when simpler rules-based approaches are the better choice Experience with cloud data tooling such as BigQuery, Spark, dbt, Airflow or similar Experience building tooling for operational or investigative teams, and partnering closely with the people who use it Interest ...

scaling patterns and reuse. Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with ...

help deliver robust, secure and high-performing data products. What youll be doing: Design and build scalable data pipelines using Python, SQL, Spark and Databricks. Develop, deploy and optimise machine learning models, including NLP and generative AI solutions. Build data ingestion pipelines across batch and streaming environments. Design data … visualisations using Power BI or Tableau. Mentor junior team members and promote engineering best practices. What you need: Commercial experience with Python, SQL, Spark and modern data engineering frameworks. Experience deploying machine learning models into production. Strong knowledge of cloud platforms such as Azure, AWS or GCP. Experience with ...

like ER/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record … with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability to appropriately prioritise ...

tenant isolation, row‐and‐column‐level security, and PII handling. Define the streaming and batch ingestion architecture, building on our investments in NATS JetStream, Apache Flink, and Debezium CDC patterns, and direct the engineering team on execution. Specify the catalog, lineage, and data‐quality foundation and own the operational … streaming vs batch). Hands‐on experience with: Dremio, Snowflake, or Databricks AWS data services (S3, Glue, Athena, Lake Formation, or equivalent) NATS/Apache Kafka or similar, and CDC patterns (Debezium or equivalent) Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation ...

Months contract 2 Days a month onsite - Expenses to Newcastle depends on their hub. Start 3 from offer Experience required AWS Services Terraform Apache Spark Python Airflow Docker Nice to haves Gitlab Security Scanning Agents (eg Trivy, Trend Micro, Wiz etc) Jupyter Notebooks Integration with DWP Integration Team ...

with cross-functional teams, and mentoring junior engineers. Ideal candidates will have over 7 years of experience in data engineering, proficiency in tools like Apache Spark and Azure Data Factory, and familiarity with AI and machine learning concepts. This role offers a dynamic work environment, promoting innovation ...

will lead requirements for the Treasury Analytics Group, support FIS Quantum & TBSM, and work on a new risk/analytics platform using Hadoop, Spark, and NoSQL tools. The role reqs 5+ years of experience, Linux/Oracle/MS SQL, ITIL, and scripting with Java/Python. Collaboration with ...

scalability, reliability, and observability. Strong knowledge of the JVM and server‐side Kotlin/Java programming. Solid understanding of the Hadoop ecosystem, including Spark, Hadoop, and Hive. Strong experience with cloud platforms and data infrastructure, especially AWS, including EMR, S3, and Glue. Experience working in Agile environments and contributing ...

infrastructure (Solr, Kubernetes). Knowledge of key‐value stores, Git, and basic Unix commands. Desirable Experience with large data processing technologies such as Apache Spark. Hybrid Working Hybrid working is available. Team members are required to work onsite on Mondays and Tuesdays, and from home for the remaining days. ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

looking for a mixture of the following skills: Python Kubernetes Docker CI/CD basics (Gitlab pipelines) Data science and MLOps knowledge (highly desirable) Spark (highly desirable) Scala (highly desirable) S3 (highly desirable) Graph databases (highly desirable) Elasticsearch (highly desirable) Vector stores (highly desirable) OIDC/OAuth (highly desirable ...

looking for a mixture of the following skills: Python Kubernetes Docker CI/CD basics (Gitlab pipelines) Data science and MLOps knowledge (highly desirable) Spark (highly desirable) Scala (highly desirable) S3 (highly desirable) Graph databases (highly desirable) Elasticsearch (highly desirable) Vector stores (highly desirable) OIDC/OAuth (highly desirable ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

Experience establishing reliability programs that deliver meaningful, measurable business outcomes. Experience with Kubernetes, cloud infrastructure, and large-scale distributed systems. Familiarity with Kafka, ClickHouse, Spark, Flink, BigQuery, or similar large-scale data platforms. Experience partnering with Product, Data Science, and Ads Engineering organizations. Experience supporting machine learning inference ...

with stakeholders. Preferred qualifications, capabilities, and skills Experience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray) Experience of big data technologies (e.g. Spark, Hadoop) Have constructed batch and streaming microservices exposed as REST/gRPC endpoints Familiarity with GraphQL We recognize that our people are our strength ...

with product stakeholders to align technical strategy with business goals and resolve blockers across the team. Apply efficient data transformations using distributed computing (e.g., Spark, Dask) and ensure robust test coverage for production systems. Mentor junior team members and utilise model explainability methods to drive feature performance improvements. About ...

leader, and/or technical program manager. Data Engineering Proficiency : Expertise in SQL, Python, Pandas, Matlab, or R. Bonus if familiarity with Influx, Grafana, Spark, Polars, Arrow, Kafka, Beam, and Flink. [Bonus] In-Field Hardware Mastery : Proficiency with hardware engineering tools such as DAQs, DDS, Nix, RTOS, MCAP ...

good fit Strong hands‐on experience of developing production‐grade solutions involving: Building Microservices (including scalable data pipelines using frameworks like Spark) Data technologies (Python, SQL) Large language models, fine tuning (closed & open source, OpenAI API) Solution design (mainly data applications using Python, SQL and other allied tech stack ...

system efficiency at scale Familiarity with a range of tools and technologies including: Development: Kotlin, Java Data storage/processing: DynamoDB, MySQL, Snowflake, Databricks, Spark Infrastructure as code and CI/CD: Terraform, GitLab CI/CD//You don't need to have experience in every area ...

business value in collaboration with an Account Executive and a Senior Solutions Architect. Gain excitement from clients about Databricks through hands‐on evaluation and Spark programming, integrating with the wider cloud ecosystem and third‐party applications. Contribute to building the Databricks technical community through engagement at workshops, seminars … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. EEO Statement At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. ...

business value in collaboration with an Account Executive and a Senior Solutions Architect. Gain excitement from clients about Databricks through hands‐on evaluation and Spark programming, integrating with the wider cloud ecosystem and 3rd party applications. Contribute to building the Databricks technical community through engagement at workshops, seminars … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs ...