76 to 100 of 124 Apache Spark Jobs in England

writing, talks, or threads about building with AI. MLOps and model serving experience (BentoML, MLflow, Vertex AI, SageMaker). Streaming and batch ingestion pipelines (Spark, Airflow, Beam, Glue). Healthcare or life sciences domain exposure. AWS Professional certification or other relevant industry certifications. What We Offer At Newpage ...

managing big data platforms, and transforming high-volume structured and unstructured financial data into actionable insights. Proficient in Hadoop ecosystem technologies including HDFS, Hive, Spark, Sqoop, Kafka, Oozie, and YARN , along with SQL and cloud-based data platforms Hybrid working: The places that you work from … home 100% of the time. Your Role: Design, develop, and optimize Hadoop-based data ingestion and processing pipelines. Build ETL/ELT workflows using Spark, Hive, Sqoop, Kafka, and other big data tools. Integrate data from banking, trading, payments, risk, and customer management systems. Ensure data quality, integrity, security ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Databricks. Familiarity with cloud‐based application development (AWS & Azure). Familiarity with one or more ML frameworks and technologies: scikit‐learn, xgboost, TensorFlow, PyTorch, Spark, SageMaker, Vertex AI, Kubeflow, Seldon, Triton. Strong communication skills, able to express ideas clearly and collaborate across teams. #J-18808-Ljbffr ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...

Role : Senior Data Manager/Lead Location : London (Hybrid) Comp : £115,000 P/A (FTC) Duration: 12 Months Skills: Python (Programming Language), SQL, Apache Spark, PySpark, Azure Databricks, Data Lakes, Data Warehouse Architecture, Leadership, Software Development, Management The role We're partnering with a large, data-driven ...

audit trails are essential. Desirable Skills & Experience Databricks Certified Data Engineer Professional or equivalent Databricks certification. Experience with adjacent data tooling such as dbt, Apache Spark, or AWS Glue. Familiarity with data product management tooling and data marketplace patterns. Previous experience in a technology modernisation or platform migration ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

teams Ensure solutions meet security, compliance, and regulatory standards Key Skills Required: Experience in solution design/architecture Strong background in big data tools (Spark, Airflow, Trino, etc.) Knowledge of cloud platforms (AWS/Azure/GCP) Understanding of cybersecurity principles (IAM, encryption, secure design) Ability to work hands ...

experience Term structure modelling, stochastic calculus or Monte Carlo methods Interest rate curve bootstrapping Algorithmic execution experience Data lake or warehouse experience (Snowflake, Iceberg, Spark etc.) We’re people without borders—without judgement or prejudice, too. We want to work with the best people, no matter their background. ...

Python and a solid understanding of software development lifecycle principles Experience in modular code, unit testing, regression testing, and distributed computing (e.g., MapReduce, Spark DataFrames) Proven track record of leading and executing enterprise‐wide initiatives Strong communicator able to explain complex methodologies to senior stakeholders, auditors and regulators Skills ...

like ER/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record … with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability to appropriately prioritise ...

like ER/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record … with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability to appropriately prioritise ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … technologies such as Iceberg, Spark, PostgreSQL, and cloud-native infrastructure on AWS (or equivalent). You write clean, readable, and testable Python code, applying sound abstractions, naming, and review discipline to build systems that scale beyond the first version. You bring strong analytical and problem-solving skills ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

with stakeholders. Preferred qualifications, capabilities, and skills Experience designing/implementing pipelines using DAGs (e.g. Kubeflow, DVC, Ray) Experience of big data technologies (e.g. Spark, Hadoop) Have constructed batch and streaming microservices exposed as REST/gRPC endpoints Familiarity with GraphQL We recognize that our people are our strength ...

business value in collaboration with an Account Executive and a Senior Solutions Architect. Gain excitement from clients about Databricks through hands‐on evaluation and Spark programming, integrating with the wider cloud ecosystem and third‐party applications. Contribute to building the Databricks technical community through engagement at workshops, seminars … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. EEO Statement At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. ...

business value in collaboration with an Account Executive and a Senior Solutions Architect. Gain excitement from clients about Databricks through hands‐on evaluation and Spark programming, integrating with the wider cloud ecosystem and 3rd party applications. Contribute to building the Databricks technical community through engagement at workshops, seminars … data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs ...

Data Engineer Location: London (Hybrid) Salary: Up to £75,000 + Bonus + Benefits Technology: SQL, Python, Spark - Azure, AWS, GCP - ETL/ELT - Data Governance The Opportunity Are you a Data Engineer looking for an opportunity to work on large-scale data transformation programmes whilst helping shape enterprise … data lineage. Support Data Governance initiatives, ensuring data security, privacy and compliance standards are maintained. Work with large and complex datasets using SQL, Python, Spark, Hadoop and modern data technologies. Collaborate with technical and business stakeholders within Agile delivery environments. Support data architecture and data modelling activities across enterprise ...

understand how they’re designed, scaled, and why they fail. Hands‐on experience building large‐scale data processing systems using technologies such as Apache Spark and processing very large datasets. Deep experience designing high‐performance data architectures using PostgreSQL, DuckDB, Redis, and similar technologies. Experience building distributed, event … over genes, variants, diseases, medications, and scientific literature. Design high‐performance data models, indexing strategies, and storage architectures across PostgreSQL, Qdrant, Redis, DuckDB, and Apache Spark. Develop guardrails and deterministic reasoning layers that keep AI responses scientifically accurate, explainable, and grounded in genomic evidence. Translate complex bioinformatics and genomic ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits We strive to provide comprehensive benefits and perks that meet the needs of all our employees. ...

architecture and deployment. Key Responsibilities Data Pipeline Development Build, optimize, and maintain scalable data pipelines using Java, plus exposure to Python, Flink, Kafka, or Spark click apply for full job details ...

compute cluster capabilities Demonstrate advanced proficiency in Python with extensive experience building production applications and services Experience with modern cluster compute technologies (e.g., Kubernetes, Spark, Hadoop Yarn, Ray, Airflow) Experience leading work in a formal software development environment in collaboration with other engineers Strong experience building platforms ...