176 to 200 of 413 Apache Spark Jobs in England

work. Technologies and Tools Python Machine learning frameworks (e.g. PyTorch, TensorFlow, scikit-learn) ML/MLOps tooling (e.g. SageMaker, MLflow, TFServing) Spark and Databricks AWS services (e.g. IAM, S3, Redis, ECS) CI/CD tooling and best practices Streaming and batch data systems (e.g. Kafka, Airflow, RabbitMQ) Additional Information ...

libraries (e.g., scikit-learn, pandas, PyTorch or TensorFlow). Practical experience with Databricks, including MLflow for experiment tracking and model management, and Spark-based data processing. Familiarity with Generative AI concepts and tooling, including working with LLMs, prompt engineering, RAG patterns, and vector search. A solid understanding of MLOps ...

evolving platform stack — from data ingestion and event processing to REST services and internal tooling. Build and maintain streaming and batch processing systems (Flink, Spark, Kafka) that power real‐time and analytical workloads. Take responsibility for all aspects of software engineering: design, implementation, QA, and maintenance, with … Willingness and ability to take on new technologies as our stack evolves. Nice to Have Experience with stream processing frameworks such as Flink or Spark Streaming. Experience with task orchestration tools such as Apache Airflow or similar systems. Familiarity with additional technologies: GraphQL, React, HTML5, JavaScript, CSS, Postgres ...

engineering experience; Eagerness to mentor and technically guide engineering teams; Experience guiding technical decision making; Extensive, hands‐on experience building distributed data pipelines using Apache Spark; Knowledge of how to build and optimize data pipelines, architectures and data sets with the ability to drive additional learning for knowledge ...

years' experience delivering enterprise data platforms or analytics solutions. Strong commercial experience delivering Databricks solutions. Consulting or customer-facing delivery experience. Strong understanding of Apache Spark, modern Lakehouse architecture and Unity Catalog. Experience with Azure, AWS or GCP. Experience implementing CI/CD and deployment automation. Excellent communication ...

inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open‐source frameworks Familiarity with AWS‐native data services such as Glue, Lake Formation, Redshift, or Athena Exposure to data governance ...

Monitor and maintain - Monitor data systems for performance issues and make any necessary updates. What You’ll Bring ETL processing languages such as Python Apache Spark/NiFi/Kafka Relational databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB, Cassandra) AWS and data-related services Palantir Foundry ...

build out technology roadmapsCodebase ownershipTechnical Application ownershipQualificationsStrong experience with DevOps & IaC tooling - Terraform, CI/CD pipelines, Git, Jenkins , HarnessStrong experience of developing on Apache Spark/HadoopStrong experience with AWS cloud services including AWS Glue, S3, StepStrong experience building data pipelines and data lakeExperience of the full ...

architecture, distributed data systems, orchestration, governance, and scalable data product design Experience with technologies and ecosystems such as GCP, AWS, Azure, Databricks, Snowflake, Kafka, Spark, dbt, Kubernetes, Airflow, ML/AI enablement, or similar modern platform tooling Strong commercial acumen and ability to engage credibly with senior technical ...

with cloud‐based data warehouses (e.g., Snowflake, GBQ). Familiarity with cloud‐based data pipeline services (e.g. AWS Glue) or distributed processing frameworks like Apache Spark. Equal Opportunity Employment Goldman Sachs is an equal opportunity employer and does not discriminate on the basis of race, colour, religion, sex, national ...

Java, Scala or Go) Extensive experience working with operational data stores, data warehouse, large‐scale data technologies and data lakes Experience using distributed frameworks (Spark, Flink, Beam, Hadoop) Good knowledge of containers (Docker, Kubernetes etc) and experience with cloud platforms such as GCP, Azure or AWS Strong experience working ...

about you: Strong Python knowledge. Ability to read through code. Demonstrable experience collaborating with engineers. Experience with big data technologies such as Hadoop, Spark, or similar. Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and data warehousing solutions. Strong knowledge in at least a few of the following areas ...

experience in software engineering, MLOps, and deploying machine learning models at scale.* Proficiency in SQL, Python, and familiarity with big data technologies (e.g., Kafka, Spark) and/or cloud platforms (e.g., GCP, AWS, or Azure).* Industry knowledge: Experience in gaming or digital entertainment is a strong plus. ...

Experience with Infrastructure as Code (Terraform, CloudFormation, or similar). Strong experience with cloud platforms, ideally AWS. Experience with distributed data processing technologies (e.g. Spark) and streaming platforms (e.g. Kafka). Ability to operate at both a strategic and hands-on level when required. Strong communication and stakeholder management ...

including feature engineering, model training, and evaluation metrics Familiarity with ML frameworks, e.g., PyTorch, TensorFlow, PySpark Experience with large scale data processing systems, e.g., Spark Work in a way that works for youWe promote a healthy work/life balance across the organization. We offer an appealing working prospect ...

minimum bar is 2+ years of Databricks experience in recent years, demonstrating strong familiarity with modern capabilities and current platform patterns (not legacy Spark-only experience). What you';ll be doing: Own the end-to-end Databricks solution architecture across ingestion, transformation, governance, serving, and operations. Define ...

designing and enforcing data contracts and quality SLAs in distributed data ecosystems. Proficiency in Python, Java, or Scala, and experience building pipelines with Databricks, Spark, or Kafka. Strong understanding of data governance principles, privacy regulations (GDPR, CCPA), and secure data handling practices. Familiarity with metadata management and data catalog ...

like Terraform or CloudFormation. Previous exposure to additional technologies like Seldon, KServe, RayServe, Kubernetes, MLFlow, Sagemaker, Kubeflow, Anyscale, Valohai, ArgoCD, Docker, Python, Java, Ray, Spark, Pandas, Argo Workflow, Postgres, Snowflake and BigQuery is highly desirable. Good to have: previous knowledge managing complementary DBs like vector and/or graph ...

Proficiency in the Python data science ecosystem (e.g. NumPy, Pandas, scikit‐learn, PyTorch, TensorFlow). Experience with big data and distributed analytics platforms (e.g. Spark, Hadoop). Hands‐on experience with Natural Language Processing, graph analytics and advanced analytical methods. Experience using cloud analytics platforms, with strong ...

Proficiency in the Python data science ecosystem (e.g. NumPy, Pandas, scikit-learn, PyTorch, TensorFlow). Experience with big data and distributed analytics platforms (e.g. Spark, Hadoop). Hands-on experience with Natural Language Processing, graph analytics and advanced analytical methods. Experience using cloud analytics platforms, with strong ...

working with databases. Comfortable working with a variety of data sources, both structured and unstructured and very large datasets using distributed computing (e.g. spark). Experience working with LLMs to deliver value in a commercial organisation, including how to manage and monitor LLM-based applications to maximise performance. Experience ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

software engineers and product managers to deliver robust data solutions. The role requires 7+ years in data engineering, including leadership, and strong expertise in Apache Spark and Azure Data Factory, with hands-on cloud experience across Azure/AWS/GCP. #J-18808-Ljbffr ...