201 to 225 of 372 Apache Spark Jobs in England

migration context. Strong SQL skills for complex joins, window functions, anomaly detection and source-to-target reconciliation. Working knowledge of Python, Azure Databricks or Spark SQL, and familiarity with Git and dbt. Ability to present evidence, scale and business impact clearly to technical and non-technical stakeholders. A responsible ...

Reporting Analytical and Critical Thinking Risk Management and Regulatory Knowledge Stakeholder Engagement and Communication Data Governance and Model Validation Transformation Programme Support Other Skills Spark/SAS and Large-Scale Data Processing Power BI/Tableau Data Pipeline Development Scenario Analysis and Stress Testing Cross-Functional Collaboration Qualifications ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing. Experience with containerization (Docker) and scalable data systems (e.g., Spark, Kafka). Strong leadership presence, excellent communication, strategic thinking, and empathy with a hands-on execution mindset. Desirable Requirements: Experience with AWS SageMaker ...

batch processing, and scalable enterprise workloads, integrating with clients' incumbent technology estates. Implement hybrid‐cloud and multi‐cloud strategies using modern data infrastructure (Kafka, Spark, data lakes), MLOps practices, and deployment platforms. Ensure compliance with data privacy regulations (including UK GDPR) and responsible AI standards, incorporating governance, observability ...

Scale processing across distributed cloud warehouses/storage via container orchestration (e.g. Kubernetes), High-Performance GPU Compute (e.g. Slurm), or distributed compute frameworks (e.g. Spark, Ray). Contribute to an engineering culture that values maintainability, testing, robust system design, and deep collaboration, but allows flexibility for rapid prototyping ...

experience and a strong understanding of high-concurrency, high-throughput, mission-critical, and zero-downtime systems. Think in distributed systems: Apply practical knowledge of Spark, Hadoop, HBase, HDFS, Hive, Impala, and adjacent data-processing ecosystems to solve complex customer problems. Win technical evaluations: Guide enterprise prospects through demanding architecture ...

other teams with AI. By 12 months You’re putting AI to work in product discovery, building rapid prototypes that test ideas quickly and spark new ones. Working software you’ve built is landing regularly and being used across OE and by our clients, with a short, safe path ...

Public writing, talks, or threads about building with AI. MLOpsand model serving experience (BentoML,MLflow, Vertex AI, SageMaker). Streaming and batch ingestion pipelines (Spark, Airflow, Beam, Glue). Healthcare or life sciences domain exposure. AWS Professional certification or other relevant industry certifications. What We Offer At Newpage ...

laptop. Nice to have requirements An ontology, or a business entity model, on top of a semantic layer. A distributed engine such as Spark or Trino, used on data that did not fit a single warehouse query. Fintech, payments, or another regulated environment. Success in the role looks like ...

processes. Engage in hands-on development with Scala, Java, or Python through system integration projects or as a data scientist. Utilise technologies such as Spark, Elastic, Restful APIs, JSON, and Tomcat. Build reusable libraries and shared capabilities to expedite time to market for Data Engineering delivery teams. Continuously improve … Python, and SQL. Well-versed in software engineering and systems design principles. Hands-on experience with Azure cloud data platforms is essential. Familiarity with Spark and Apache Airflow is required. Proficient in Linux/Unix/*nix environments. Strong knowledge of ETL tools and data modelling techniques. Candidates ...

including target modelling, cutover approach, reconciliation strategy and technical risk management. Design and build the real‐time data architecture using Amazon MSK/Apache Kafka and PySpark Structured Streaming (evaluating Apache Flink where it is the better fit), including event schema governance, exactly‐once semantics and stream … large‐scale batch and streaming processing, and strong command of data modelling and data warehousing concepts. Production experience designing and operating streaming architectures with Apache Kafka or Amazon MSK, including stream processing with Spark Structured Streaming. Hands‐on experience with open table formats, such as Apache Iceberg ...

such as RDF, OWL and SPARQL. Familiarity with feature store design and MLOps/DataOps pipeline integration. Experience with stream processing at scale using Apache Kafka or Apache Flink. Background in master data management or data mesh architecture. Consulting or comparable client‐facing delivery experience. Exposure to some … with all of them: Cloud data platforms, warehouses and lakehouses: Databricks, Snowflake, Microsoft Fabric, Azure Synapse, Google BigQuery, Amazon Redshift Data engineering and orchestration: Spark, dbt, Airflow, Azure Data Factory, AWS Glue, Dataflow, Kafka, Flink Governance, catalogue and lineage: Microsoft Purview, Collibra, Informatica, Alation, Atlan, OpenLineage Graph, ontology ...

unify data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits Databricks strives to provide comprehensive benefits and perks. For specific details in your region, refer ...

bronze/silver/gold layers, and partner with analytics, ML, and trading teams to deliver trusted data assets. You will work with dbt, Spark, Airflow, Kafka, and streaming technologies to ensure data quality, governance, and efficient storage. #J-18808-Ljbffr ...

develop pipelines for analytics, ML and business intelligence in a security-conscious environment. The role requires 3+ years in data engineering, strong experience with Spark, Python, Kafka, SQL and Airflow, and a willingness to work hybrid in London or Cheltenham. You will contribute to client engagements and growth ...

orchestration, standardize data across real-world teleoperation and synthetic datasets, and collaborate with a team building general-purpose Physical AI. Strong Python/Go, Spark, Kafka, Kubernetes, and Terraform experience required. #J-18808-Ljbffr ...

senior stakeholders and contribute to business development initiatives. Provide thought leadership through industry engagement and innovation. Requirements Strong Python, SQL, Pandas, Scikit-Learn and Spark experience. Experience delivering real-world AI and machine learning systems. Understanding of statistics and machine learning methodologies. Experience with cloud platforms and large-scale ...

senior stakeholders and contribute to business development initiatives. Provide thought leadership through industry engagement and innovation. Requirements Strong Python, SQL, Pandas, Scikit-Learn and Spark experience. Experience delivering real-world AI and machine learning systems. Understanding of statistics and machine learning methodologies. Experience with cloud platforms and large-scale ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

understanding of data structures, algorithms, and software design principles Experience working with large-scale data processing, distributed computing, or cloud-based platforms (e.g., Hadoop, Spark, AWS) is highly desirable Familiarity with ESG investing principles and market trends is a plus Excellent problem-solving, analytical, and debugging skills Strong communication ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

large language model (LLM) systems, including retrieval augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, so data is high quality, time-stamped, and ready for both training and real-time inference. Design and deploy data collection systems ...

large language model (LLM) systems, including retrieval-augmented generation (RAG) and fine-tuning. Build scalable ETL/ELT pipelines using tools such as Spark or dbt, ensuring data is high quality, time-stamped and ready for both training and real-time inference. Design and deploy data collection systems ...

pipelines to collect, transform and deliver datasets for analytical decision making • Work with cloud platforms, big data technologies, and modern programming languages (Python, SQL, Spark etc) to solve complex data challenges • Support data modelling, database design and integrated data sources • Apply data quality, governance and security controls in line ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...