126 to 150 of 372 Apache Spark Jobs in England

scale datasets Preferred Qualifications Knowledge or interestin investment banking or financial instruments Experience with big data concepts, such as Lakehouse, ETL data pipelines (e.g. Spark, AWS Glue) Experience with near real time and event‐based systems like Kafka ABOUT GOLDMAN SACHS At Goldman Sachs, we commit our people, capital ...

including data quality, data lineage, model testing, prompt testing, access controls, monitoring, explainability, human oversight and governance. Strong data analytics capability, including Python, SQL, Spark, Power BI, Tableau or equivalent tools. Ability to translate complex technical concepts into practical business risks, control implications and clear audit conclusions. Internal Audit ...

modern cloud engineering practices. Wherever you land, you’ll be working with some of the biggest datasets in the UK — using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. Cloud-native AI engineering and large-scale data platforms, leveraging Google Cloud technologies and modern ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

modern cloud engineering practices. Wherever you land, you'll be working with some of the biggest datasets in the UK - using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Cloud-native AI engineering and large-scale data platforms ...

experience building, deploying, and scaling AI/ML solutions using frameworks and platforms such as Scikit-learn, PyTorch, TensorFlow, Hugging Face, LangChain, Databricks, Spark, vector databases, cloud services (AWS/Azure), and MLOps tools including MLflow and Kubeflow. Proven ability to develop production-grade, large-scale AI systems with ...

able to offer visa sponsorship for this role at present Desirable (Nice to Have) Experience with streaming and big‐data tooling – Apache Kafka, Spark, or Flink Experience with MLOps platforms – MLflow, Kubeflow, Vertex AI, SageMaker, or Weights & Biases Experience evaluating and optimising RAG systems at scale (retrieval quality ...

support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … ease. Understanding of Data Lakehouse Architecture:You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions:Youpossessexperience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or evenon-premiseSpark clusters, usingPySparkor Spark SQL to manage ...

support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … ease. Understanding of Data Lakehouse Architecture:You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions:Youpossessexperience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or evenon-premiseSpark clusters, usingPySparkor Spark SQL to manage ...

into actionable insights, and power high-impact trading and AI decisions. Ideal Candidate: 5+ years of experience in data engineering with MPP databases or Apache Spark Strong skills in Python and SQL, with solid data modeling experience Extensive knowledge of AWS/cloud services (ECS, EKS, Lambda, Kinesis ...

languages like Python, or functional programming languages like Scala Demonstrated experience in design and development of big data applications using tech stacks like Databricks, Apache Spark, HDFS, HBase and Snowflake Commendable skills in building data products, by integrating large sets of data from hundreds of internal and external ...

hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

Analytics, Artificial Intelligence or a related field, or equivalent experience, would be effective, although practical experience is equally valuable. “Nice to Have” Azure, Databricks, Apache Spark or comparable cloud and data platforms. Data pipelines, ETL and workflow‐orchestration technologies. Machine‐learning methods such as forecasting, prediction or classification. ...

GenAI infrastructure and workflows Skills and experience: Passionfor building scalable, resilient cloud data platforms Strongexperience with Databricks or Snowflake on AWS ProvenPython skills, including Spark and Airflow expertise AdvancedSQL skills and end‐to‐end data modelling experience Experiencebuilding batch and real-time data integrations Hands-onCICD skills with ...

problem‐solving skills to collaborate across global teams, and translate complex data insights into actionable outputs Familiarity with data platforms such as Databricks, Spark, Snowflake, or similar technologies Ability to communicate technical concepts clearly to non‐technical stakeholders Basic understanding of artificial intelligence concepts, with curiosity and enthusiasm ...

Nice to have: Proven experience leading projects and managing teams, Experience in presales, client workshops, or building Proof-of-Concepts (PoCs), Familiarity with Hadoop, Spark, Research or R&D background. Soft Skills Ability to structure projects teams, make technical decisions, and own outcomes, Ability to manage multiple projects ...

Python and SQL with strong knowledge of data modeling and ETL/ELT processesSolid understanding of Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processingExperience with Azure Data Factory (ADF), Qlik Replicate and modern data architecture practicesExcellent communication, stakeholder management, problem-solving skills ...

/ML offerings. Strong programming skills (Python, Java, C++) and experience with DevOps practices (CI/CD). Knowledge of big data technologies (Hadoop, Spark, Flink). Knowledge of NoSQL databases (MongoDB, Cassandra, DynamoDB), message queueing systems (Kafka, RabbitMQ), and version control systems (Git). Preferred Skills: Hugging Face ...

Platform. Have a thorough understanding of Industrial systems - ERP (Microsoft Dynamics, SAP), MES, SCADA Be proficient with big data tools - familiarity with tools like Spark, Kafka, or Hadoop. Be a self-starter, confident to work both independently, scoping ideas from a business led design brief as part ...

security Practical cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial ...

Python and SQL; APIs; test automation; data structures; distributed processing; Git; CI/CD; containers; infrastructure awareness; production debugging and performance optimization. Data engineering: Spark or Databricks, Kafka or equivalent streaming, ETL/ELT, batch and real-time pipelines, data contracts, lineage, cataloguing, quality rules, and scalable cloud storage. ...

libraries Strong background in applied machine learning, model development and data engineering Experience with cloud environments (Azure, AWS, GCP) and tools such as Spark, Hive, Redshift Demonstrated ability to lead cross-functional teams and mentor junior practitioners Ability to communicate complex technical concepts clearly to non-technical audiences Bonus ...

operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets Experience with big data technologies such as: Hadoop, Hive, Spark, EMR Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting decisions ...

skills in Python, and experience with Java or Scala. Data Engineering at Scale: Solid ELT/ETL design experience and hands-on work with Apache Spark. Bonus points if you have: Multi-Cloud Exposure: Experience with Azure alongside GCP. Container Orchestration: Experience packaging and deploying workloads with Helm ...