376 to 400 of 569 Apache Spark Jobs in the UK

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits Databricks strives to provide comprehensive benefits ...

ensure data availability, quality, and efficiency across markets. This role is onsite in London and requires strong SQL and programming skills, plus experience with Spark, Hadoop, and cloud platforms. #J-18808-Ljbffr ...

ensure requirements are understood before design and development. The role focuses on data warehouse and large‐scale data processing using Big Data technologies (Hadoop, Spark), with programming in Scala and Python. You will guide developers, implement best practices, and drive automated testing and CI/CD to production. #J ...

stakeholders. Architecture design informed through the use of principles, patterns, technical radars, practices and standards. AWS Solution expertise and relevant certification - EMR/Spark (Hadoop)/AWS/Data Modelling-Mapping/GitHub/GitLab Technologies Azure AWS Power BI #J-18808-Ljbffr ...

software engineering skills (Python, Go, or similar) Extensive experience designing, building, and maintaining large-scale data pipelines (8+ years) Deep understanding of distributed systems (Spark, Kafka, or similar) Extensive experience with data storage technologies (data lakes, warehouses, object stores like S3) Experience running and maintaining production-grade infrastructure (Kubernetes ...

background)Testing and deployment pipelines, software and productivity tools, monitoring and middleware, data infrastructure. Integrating best-of-breed tools and technologies such as Spark, Kafka, Zookeeper, Mesos, Docker, ElasticsearchAssembling platforms and frameworks to automate business processesCollaborating with internal technology, research and portfolio management teamsPython Developers MUST have: A bachelor ...

depth actually gets used. AWS across the board, including S3, Glue, Redshift, Lake Formation and Kinesis Databricks and Snowflake running on top of AWS Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kinesis or Kafka You're expected to pick the right tools ...

data mining, information retrieval, and statistics Understanding of machine learning frameworks: Tensorflow, Pytorch, pyG, Keras, MXNet, Scikit-Learn Strong programming knowledge of Python and Spark; proficiency in vector operations using numpy, scipy; experience with distributed computation using multithreading, multi GPUs, Dask, Ray, Polars Familiarity with AWS Cloud services such ...

experience with additional cloud providers is a plus. Demonstrated success leading initiatives involving complex data pipelines and analytics platforms utilizing technologies such as Hive, Spark, Presto, Cassandra, Hadoop, and ML/NLP capabilities. IQVIA is a leading global provider of clinical research services, commercial insights and healthcare intelligence ...

looking for a mixture of the following skills: Python Kubernetes Docker CI/CD basics (Gitlab pipelines) Data science and MLOps knowledge (highly desirable) Spark (highly desirable) Scala (highly desirable) S3 (highly desirable) Graph databases (highly desirable) Elasticsearch (highly desirable) Vector stores (highly desirable) OIDC/OAuth (highly desirable ...

looking for a mixture of the following skills: Python Kubernetes Docker CI/CD basics (Gitlab pipelines) Data science and MLOps knowledge (highly desirable) Spark (highly desirable) Scala (highly desirable) S3 (highly desirable) Graph databases (highly desirable) Elasticsearch (highly desirable) Vector stores (highly desirable) OIDC/OAuth (highly desirable ...

Desirable Experience with cloud-based risk platforms and pricing engines Azure Kubernetes Service (AKS) Azure Data Lake Microservices and event-driven architectures Kafka, Redis, Spark, HBase, and HDFS Market Risk platforms covering VaR, Stress Testing, and FRTB Risk exposure reconciliation and reporting platforms Enterprise scheduling tools such as Control ...

powered services into production. Strong Python and Java, both of which are in active use across the team's production services. Experience with Snowflake, Spark, Databricks or similar technologies, CI/CD pipelines, and modern DevOps tooling. Solid understanding of SecOps practices and security-conscious system design. Demonstrable track ...

networking, storage, to applications Work in Python to design and deliver open source code Architect cloud infrastructure solutions like OpenStack, Kubernetes, Ceph, Hadoop and Spark either On-Premises or in Public Cloud (AWS, Azure, Google Cloud) Coach and develop your colleagues where you have insights Grow a healthy, collaborative ...

with product stakeholders to align technical strategy with business goals and resolve blockers across the team. Apply efficient data transformations using distributed computing (e.g., Spark, Dask) and ensure robust test coverage for production systems. Mentor junior team members and utilise model explainability methods to drive feature performance improvements. About ...

with product stakeholders to align technical strategy with business goals and resolve blockers across the team. Apply efficient data transformations using distributed computing (e.g., Spark, Dask) and ensure robust test coverage for production systems. Mentor junior team members and utilise model explainability methods to drive feature performance improvements. About ...

customer lifetime value, risk scoring, financial crime or marketing analytics. Technical proficiency with SQL databases and analytical tools such as Python, R or Spark, with an openness to using approved AI-assisted capabilities to improve productivity and code quality where appropriate. Strong experience developing dashboards and data visualisations using ...

record of building AI solutions that solve real business problems and of taking them from prototype to production. Practical experience with Databricks, MLflow and Spark based data processing. Working knowledge of responsible AI methods, including bias and fairness testing, explainability techniques, and model risk. The ability to explain complex ...

large generative models. AV/robotics/simulation; multi-sensor driving data (video, telemetry; LiDAR a plus). Productionising research: Flyte/Ray/Spark-style jobs, dataset lineage, training mix configuration. Reward models, offline RL, or closed-loop evaluation of driving policies. Cloud GPU fleets (Azure/ ...

provenance. Hands‐on experience with data lineage/metadata tooling in enterprise environments (e.g., Collibra, Alation, Informatica EDC/IDMC, IBM Infosphere, Microsoft Purview, Apache Atlas, Amundsen, DataHub or similar). Proven ability to build lineage for complex platforms: data lakes, warehouses, marts, and distributed processing (Spark‐based … Working knowledge of ETL/ELT patterns, data modeling (dimensional + normalized), and batch scheduling dependencies. Ability to interpret data transformation logic from pipelines (Spark SQL/PySpark/Hive queries/orchestration configs). Strong documentation capability: source‐to‐target mappings, lineage diagrams, data dictionaries, metadata standards ...

enhance collaboration and delivery. What We're Looking ForStrong proficiency in Python, Scala, or Java, with hands-on experience of Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks. Experience building real-time event streaming solutions using technologies such as Kafka, Spark Streaming, or Amazon Kinesis … technical solutions. Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets. Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation. Knowledge of automating ingestion and transformation frameworks within cloud-native data platforms. Experience delivering solutions within highly ...

environmentFocus on delivering secure and scalable solutionsPreferred qualifications, capabilities, and skillsExperience with modern data platforms such as Databricks or SnowflakeDeep hands-on experience with Spark/PySpark and other big data processing technologiesExpertise in open-source table formats and catalog services such as Apache IcebergExperience with LLM orchestration ...

secure and scalable solutions Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with ...

improve collaboration and delivery. What We're Looking ForStrong proficiency in Python, Scala, or Java with hands-on experience in Big Data technologies including Spark, Hadoop, or similar distributed processing frameworks. Practical experience building real-time event streaming pipelines using technologies such as Kafka, Spark Streaming, or Amazon … client environments. Bonus Points ForExpertise in data modelling, schema design, and working with structured and semi-structured datasets. Experience with distributed technologies including Hadoop, Spark, HDFS, Hive, Databricks, or AWS Lake Formation. Knowledge of automating cloud-based ingestion and transformation frameworks. Experience delivering data engineering solutions within highly regulated ...

scaling patterns and reuse. Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with ...