26 to 50 of 93 Apache Spark Jobs in England

best practices for data security and compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse and ETL/ELT development in More ❯

observability. Preferred Qualifications Exposure to machine learning workflows, model lifecycle management, or data engineering platforms. Experience with distributed systems, event-driven architectures (e.g., Kafka), and big data platforms (e.g., Spark, Databricks). Familiarity with banking or financial domain use cases, including data governance and compliance-focused development. Knowledge of platform security, monitoring, and resilient architecture patterns. More ❯

architecture, integration, governance frameworks, and privacy-enhancing technologies Experience with databases (SQL & NoSQL - Oracle, PostgreSQL, MongoDB), data warehousing, and ETL/ELT tools Familiarity with big data technologies (Hadoop, Spark, Kafka), cloud platforms (AWS, Azure, GCP), and API integrations Desirable: Data certifications (TOGAF, DAMA), government/foundational data experience, cloud-native platforms knowledge, AI/ML data requirements understanding More ❯

experience with Trino/Starburst Enterprise/Galaxy administration/CLI. Implementation experience with container orchestration solutions (Kubernetes/OpenShift). Knowledge of Big Data (Hadoop/Hive/Spark) and Cloud technologies (AWS, Azure, GCP). Understanding of distributed system architecture, high availability, scalability, and fault tolerance. Familiarity with security authentication systems such as LDAP, Active Directory, OAuth2 More ❯

experience with Trino/Starburst Enterprise/Galaxy administration/CLI. Implementation experience with container orchestration solutions (Kubernetes/OpenShift). Knowledge of Big Data (Hadoop/Hive/Spark) and Cloud technologies (AWS, Azure, GCP). Understanding of distributed system architecture, high availability, scalability, and fault tolerance. Familiarity with security authentication systems such as LDAP, Active Directory, OAuth2 More ❯

practices for data infrastructure, fostering a culture of collaboration and knowledge sharing. (Required) Kubernetes and Orchestration: Manage and optimize Kubernetes clusters, specifically for running critical data processing workloads using Spark and Airflow. (Required) Cloud Security: Implement and maintain robust security measures, including cloud networking, IAM, encryption, data isolation, and secure service communication (VPC peering, PrivateLink, PSC/PSA). More ❯

version control, e.g., Git. Knowledge of OO programming, software design, i.e., SOLID principles, and testing practices. Knowledge and working experience of AGILE methodologies. Proficient with SQL. Familiarity with Databricks, Spark, geospatial data/modelling Exposure to MLOps, model monitoring principles, CI/CD and associated tech, e.g., Docker, MLflow, k8s, FastAPI etc. are a plus. Additional Information What’s More ❯

stakeholders in a fast-paced environment Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Experience performing tasks such as writing scripts, extracting data using APIs, writing SQL queries etc. Experience in processing large amounts of structured and unstructured data, including integrating data More ❯

modern data platforms (Databricks, Snowflake, Kafka), container orchestration (Kubernetes/OpenShift), and multi-cloud deployments across AWS, Azure, GCP Advanced knowledge of Big Data ecosystems (Hadoop/Hive/Spark), data lakehouse architectures, mesh topologies, and real-time streaming platforms Strong Unix/Linux skills, database connectivity (JDBC/ODBC), authentication systems (LDAP, Active Directory, OAuth2), and data integration More ❯

between the data warehouse and other systems. Create deployable data pipelines that are tested and robust using a variety of technologies and techniques depending on the available technologies (Nifi, Spark) Build analytics tools that utilise the data pipeline to provide actionable insights into client requirements, operational efficiency, and other key business performance metrics. Complete onsite client visits and provide More ❯

well as programming languages such as Python, R, or similar. Strong experience with machine learning frameworks (e.g., TensorFlow, Scikit-learn) as well as familiarity with data technologies (e.g., Hadoop, Spark). About Vixio: Our mission is to empower businesses to efficiently manage and meet their regulatory obligations with our unique combination of human expertise and Regulatory Technology (RegTech) SaaS More ❯

strong expertise in Python development within financial services in financial market data. You will play a senior, hands-on role in designing and developing scalable data solutions, working with Spark, AWS, and Aurora PostgreSQL to deliver a high-performance, cloud-native data platform. Key Responsibilities Design, build, and optimise data-driven solutions using Python in a financial markets environment … Develop and maintain distributed data pipelines using Spark and AWS-native services Work with Aurora PostgreSQL and other AWS data services for storage, querying, and analytics Integrate and process financial market data feeds (transactions, market/FX data, asset management systems) Contribute to the design and delivery of a new AWS-based data platform, ensuring scalability and performance Collaborate … a Python Developer with a focus on data platform development Strong expertise in financial markets data (transaction feeds, private banking, asset management, trading platforms, FX) Hands-on experience with Apache Spark for distributed data processing Proficiency with Aurora PostgreSQL Strong working knowledge of AWS cloud services, especially for data engineering and platform builds Experience delivering in complex, data More ❯

its native tech stack in designing and building data & AI solutions Experience with data modeling, ETL processes, and data warehousing Knowledge of big data tools and frameworks such as Spark, Hadoop, or Kafka More ❯

Pydantic) for document processing, summarization, and clinical Q&A systems. Develop and optimize predictive models using scikit-learn, PyTorch, TensorFlow, and XGBoost. Design robust data pipelines using tools like Spark and Kafka for real-time and batch processing. Manage ML lifecycle with tools such as Databricks , MLflow , and cloud-native platforms (Azure preferred). Collaborate with engineering teams to More ❯

Pydantic) for document processing, summarization, and clinical Q&A systems. Develop and optimize predictive models using scikit-learn, PyTorch, TensorFlow, and XGBoost. Design robust data pipelines using tools like Spark and Kafka for real-time and batch processing. Manage ML lifecycle with tools such as Databricks , MLflow , and cloud-native platforms (Azure preferred). Collaborate with engineering teams to More ❯

sciences, using modern cloud technologies to deliver scalable, reliable data pipelines that support scientific innovation. What You'll Be Doing: Designing and building data pipelines using tools like Python, Spark, SQL, BigQuery, and Google Cloud Storage Ensuring pipelines meet the specific needs of data-driven scientific applications Writing high-quality, well-documented code with automated testing Monitoring and improving More ❯

and NoSQL databases such as PostgreSQL, DynamoDB, or OpenSearch. -Problem-Solver: Comfortable working in Linux environments and confident debugging logs, scripts, and production issues. -Additional Skills: Exposure to Kafka, Spark, or dbt Core, with an interest in domain-driven data contracts. Meet Citywire We cover - and connect - all sides of the $100 trillion global asset management industry - through our More ❯

technical specialist, design and architecture experience - 7+ years of external or internal customer facing, complex and large scale project management experience - 5+ years of database (eg. SQL, NoSQL, Hadoop, Spark, Kafka, Kinesis) experience - 3+ years of cloud based solution (AWS or equivalent), system, network and operating system experience PREFERRED QUALIFICATIONS - AWS experience preferred, with proficiency in a wide range More ❯

Research/Statistics or other quantitative fields. Experience in NLP, image processing and/or recommendation systems. Hands on experience in data engineering, working with big data framework like Spark/Hadoop. Experience in data science for e-commerce and/or OTA. We welcome both local and international applications for this role. Full visa sponsorship and relocation assistance More ❯

for new and existing diseases, and a pattern of continuous learning and development is mandatory. Key Responsibilities Build data pipelines using modern data engineering tools on Google Cloud: Python, Spark, SQL, BigQuery, Cloud Storage. Ensure data pipelines meet the specific scientific needs of data consuming applications. Responsible for high quality software implementations according to best practices, including automated test More ❯

for new and existing diseases, and a pattern of continuous learning and development is mandatory. Key Responsibilities Build data pipelines using modern data engineering tools on Google Cloud: Python, Spark, SQL, BigQuery, Cloud Storage. Ensure data pipelines meet the specific scientific needs of data consuming applications. Responsible for high quality software implementations according to best practices, including automated test More ❯

for new and existing diseases, and a pattern of continuous learning and development is mandatory. Key Responsibilities Build data pipelines using modern data engineering tools on Google Cloud: Python, Spark, SQL, BigQuery, Cloud Storage. Ensure data pipelines meet the specific scientific needs of data consuming applications. Responsible for high quality software implementations according to best practices, including automated test More ❯

deep learning, GenAI, LLM, etc. as well as hands on experience on AWS services like SageMaker and Bedrock, and programming skills such as Python, R, SQL, Java, Julia, Scala, Spark/Numpy/Pandas/scikit, JavaScript Amazon is an equal opportunities employer. We believe passionately that employing a diverse workforce is central to our success. We make recruiting More ❯

Python, R, or SQL Collaborate with product, engineering, and business teams to solve real-world problems Design experiments, test hypotheses, and communicate insights clearly Work with big data tools (Spark, Hadoop, Snowflake) to wrangle complex datasets Translate messy data into elegant visualisations and actionable recommendations Requirements: 2+ years experience in data science, analytics, or machine learning Strong coding skills More ❯

a plus). Experience with model lifecycle management (MLOps), including monitoring, retraining, and model versioning. Ability to work across data infrastructure, from SQL to large-scale distributed data tools (Spark, etc.). Strong written and verbal communication skills, especially in cross-functional contexts. Bonus Experience (Nice to Have) Exposure to large language models (LLMs) or foundational model adaptation. Previous More ❯