176 to 200 of 234 Apache Spark Jobs in the UK

market data normalisation. Experience handling time-series challenges such as timestamp precision, sequence gaps, duplicate and out-of-order events. Experience with distributed processing (Spark, Polars, Dask, Ray or similar). Experience with Kafka or similar streaming technologies. Familiarity with PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Docker, Linux … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

from idea to scaled adoption, with KPI definition, testing/experimentation, and benefits tracking. Build on modern data/ML platforms (e.g., Databricks/Spark and Snowflake) with CI/CD, monitoring, and operational controls. Operate in a controlled environment with strong model governance (model risk, compliance, and controls … productionisation, monitoring, and support. Operationalise analytics/ML with trusted data, model governance, and delivery controls (CI/CD, deployment, monitoring) on Databricks/Spark and Snowflake. Lead senior stakeholders (Sales, Product, Risk/Compliance, CDO/CTO, SMAD/Quants, engineering/data) to align priorities, secure decisions ...

appropriate technologies for scalable analytics, trading strategy development, tick or time-series data, workflow automation and AI/ML-enabled use cases.• Prefer Databricks, Spark and modern lakehouse patterns for scalable analytics where suitable, while recognising transferable experience with Snowflake and other enterprise data platforms.• Ensure delivery choices balance … define OKRs, shape roadmaps and deliver enterprise data or analytics products.• Hands-on understanding of data technologies and modern analytics platforms, preferably Databricks/Spark and lakehouse patterns; Snowflake experience is relevant and transferable but Databricks experience is preferred.• Experience with Python, SQL and production analytics workflows; familiarity with ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

proven experience with enterprise-scale data solutions and includes maintaining architecture documentation. The ideal candidate will have strong hands-on experience with technologies like Apache Spark, Azure, and AWS. This position is a 6-month contract located in Sheffield, with an attractive pay rate of £515 per day. ...

technology teams to deliver scalable, production-grade Python services and APIs for research and operations. The role emphasizes modern cluster compute technologies (Kubernetes, Spark, Hadoop Yarn, Ray, Airflow) and strong proficiency in Python. A background in mathematics or computing is advantageous. #J-18808-Ljbffr ...

scale. The position sits in a team delivering high-impact ML across marketing and advertising, working on both real-time and batch workloads with Spark, Python, Java, and #J-18808-Ljbffr ...

APIs used by researchers and engineers across the firm. Ideal candidates have deep Python expertise, hands-on experience with modern cluster compute technologies (Kubernetes, Spark, Hadoop/Yarn, Ray, Airflow) and a track record of delivering scalable infrastructure in a multi-user #J-18808-Ljbffr ...

streamline operations. At Chainalysis, we are at the forefront of this development. Technologies in Use Technologies we use include: AWS serverless architectures Kubernetes PostgreSQL Spark Typescript Terraform Kafka Github (including Github Actions) Java Data Platform Development We are building the data platform for blockchain, cryptocurrency, and web3. The Protocol ...

focusing on backend systems for large‐scale data storage, streaming, and warehousing. Experience with database technologies (MongoDB, Postgres), streaming/processing solutions (Kinesis, Flink, Spark), indexing/caching (ElasticSearch, Redis), and query engines (Trino, Presto, Snowflake). Track record of mentoring and leading teams in successful projects. Excellent communication ...

Java, data structures and concurrency, not just framework knowledge Hands-on experience with Kafka and event-streaming frameworks such as Flink, Kafka Streams or Spark Practical Kubernetes experience, including troubleshooting production issues Experience tuning JVM performance - garbage collection, CPU hotspots and heap dump analysis Why you'll love this ...

Java, data structures and concurrency, not just framework knowledge Hands-on experience with Kafka and event-streaming frameworks such as Flink, Kafka Streams or Spark Practical Kubernetes experience, including troubleshooting production issues Experience tuning JVM performance - garbage collection, CPU hotspots and heap dump analysis Why you'll love this ...

related to building large-scale data storage, streaming, and warehousing systems. Experience in various database technologies (MongoDB, Postgres), streaming/processing solutions (Kinesis, Flink, Spark), indexing/caching (ElasticSearch, Redis), and various data query engines (Trino, Presto, Snowflake, etc.). Show a track record of mentoring and leading teams ...

matter – packed with personal touches and delivered with care. We’re not just about selling cards or gifts – we’re here to spread joy, spark smiles and make every celebration feel extra special. And with values that guide how we work and support one another, we’ve built ...

society’s experience with software. Come join one of the fastest‐growing startups, supported by best‐in‐class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. Pendo Core Values: Bias to Act, Hone Your Craft, The Team is Pendo, and Maniacal Focus. Location: Pendo is a hybrid ...

Help shape the technical roadmap for data engineering across Elliptic. Leverage and deploy AI and agentic systems to operate more effectively. Tech environment Scala | Spark | Databricks | AWS | Airflow | Kubernetes | Terraform | Functional Programming You will thrive here if you Enjoy end‐to‐end ownership, from writing the code to architecture … programming. Ability to design, build, and maintain distributed data systems in a cloud‐based environment. Hands‐on experience with big data frameworks such as Spark or Databricks. Knowledge of cloud infrastructure (AWS, GCP, or Azure). Judgement to balance scalability, performance, and maintainability. Experience with data modelling and workflow ...

needs of the department. This is an exciting opportunity to work on cutting edge technologies including Big Data and NoSQL databases (Hadoop, HBase, Hive, Spark and MongoDB) to allow the business to gain advanced insight into their portfolios and valuation metrics. Key responsibilities include:* Lead business requirements elicitation, analysis … documentation efforts* Assist technology team in Azure cloud solution implementation; write Spark SQL queries to analyze and transform data; provide feedback to User Interface team; testing and documentation* Analyze solution requirements; understand project scope; determine project priorities; ensure efficient and on-time delivery of project tasks and milestones; follow ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs ...

Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least one Deep experience with distributed computing with Apache SparkTM and knowledge of Spark runtime internals Familiarity with CI/CD for production deployments Working knowledge of MLOps, ML/AI models ...

infrastructure projects , is highly desirable What we can offer you At Nscale, you'll find a collaborative, supportive, and innovative environment where your contributions spark real impact. We're building something extraordinary, and we want you at the core. Highly competitive US compensation package (base + bonus + equity ...

toolset is preferred. Having built ELT or ETL pipelines processing large amounts of data. Experience with dbt or Airbyte is preferred. Experience developing production Spark or Hadoop applications Gained experience with a workload scheduling and orchestration tool such as Apache Airflow/Cloud Composer Familiarity with modern DataOps ...

from ingestion and streaming through to warehousing, serving, and cost optimisation You have deep experience with streaming architectures – Kinesis, Kafka/MSK, Flink, or Spark Streaming – and know when each is the right choice You can model the cost implications of architecture decisions and advise clients on trade‐offs … between throughput, latency, durability, and spend AWS certification at professional or specialty level is strongly preferred Mastery of Python, SQL, and Spark – and strong opinions on how they should be used in production data pipelines Deep experience with dbt, Airflow, or equivalent orchestration – including how to build pipelines that ...

from ingestion and streaming through to warehousing, serving, and cost optimisation You have deep experience with streaming architectures – Kinesis, Kafka/MSK, Flink, or Spark Streaming – and know when each is the right choice You can model the cost implications of architecture decisions and advise clients on trade‐offs … between throughput, latency, durability, and spend AWS certification at professional or specialty level is strongly preferred Mastery of Python, SQL, and Spark – and strong opinions on how they should be used in production data pipelines Deep experience with dbt, Airflow, or equivalent orchestration – including how to build pipelines that ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling a modern data mesh architecture. The role … focuses on integrating legacy systems with cloud-native platforms to support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such ...

model lifecycle: data prep, experimentation, deployment, and monitoring. Build and optimize feature pipelines and scoring jobs using Python, Databricks, Spark, Delta Lake, and AWS. Use AWS services (S3, Redshift, Lambda) for data automation, orchestration, and scalable processing. Ensure data quality, observability, lineage, and documentation across data and ML pipelines. … Direct experience with OpenAI APIs, LLM workflows, and prompt engineering. Solid machine learning fundamentals, including supervised learning, NLP, and feature engineering. Experience with Databricks, Spark, and Delta Lake. Strong SQL skills with experience working on large datasets. Experience with AWS, including S3 and Lambda. Familiarity with Redshift, Snowflake ...