526 to 550 of 569 Apache Spark Jobs in the UK

Strong written and verbal communication skills, including the ability to explain complex methodology to diverse audiences. You might also have Familiarity with Databricks, DBT, Spark/PySpark, or similar large‐scale data platforms. Modified infrastructure‐as‐code (e.g. Terraform) or contributing to production data pipelines. Mentored junior data scientists … engineers. Knowledge of, or passion for, the cryptocurrency and blockchain space. Technologies we use Python SQL Databricks DBT PySpark/Spark Terraform AWS Postgres We encourage applicants across any race, ethnicity, gender/gender expression, age, spirituality, ability, experience and more. If you need any accommodations to make ...

operating large‐scale distributed platforms that power our research, trading and engineering teams across on‐premises and AWS environments, using technologies such as Spark, Trino, Kafka, ClickHouse and Airflow. Key Responsibilities Building, operating and scaling distributed analytics platforms across on‐premises and AWS environments Designing and implementing new platform … hours rota to support critical systems Core Skills and Technologies Experience running distributed data and analytics systems at scale using tools such as Spark, Kafka, Trino or Airflow Strong Linux skills and proficiency in Python for automation and integration Familiarity with infrastructure as code, using Terraform or Ansible Deep ...

engineering teams. You'll deliver highly available, secure and high-performance analytics services across on-premises and AWS environments, using technologies such as Spark, Trino, Kafka, ClickHouse and Airflow. This is a highly collaborative position, working directly with researchers and engineers to evolve platform capabilities, automate operations and explore … support critical systemsWho are we looking for? Core skills and technologiesExperience running distributed data and analytics systems at scale using tools such as Spark, Kafka, Trino or AirflowStrong Linux skills and proficiency in Python for automation and integrationFamiliarity with infrastructure as code, using Terraform or AnsibleDeep understanding ...

communicate effectively Sound knowledge and experience developing and deploying statistical and machine learning models, supervised and unsupervised Python expertise SQL Big Data, including Spark BI tools such as Tableau or Power BI Modern software development experience and toolkit, including Git, code reviews, Feature Stores and CI/CD Design … models. Proficient in data wrangling, compliance with best practices, and effective communication with stakeholders. Highest-signal resume keywords Python Expertise SQL Big Data Including Spark Statistical And Machine Learning Models BI Tools Such As Tableau Or Power BI Hard Skills Statistical Models Machine Learning Models Data Wrangling Data Manipulation ...

retraining, and optimise models for reliability and cost efficiency in AWS. Collaboration & Enablement: Partner with data engineers to integrate ML workflows into wider platforms (Spark, Databricks), and share best practice and mentor other technical professionals. What You’ll Love About This Role Think Big: Build ML and AI solutions … models underperform across data, features and architecture, and make reasoned trade-offs. Real-time & distributed ML: A strong grasp of production inference patterns, plus Spark and distributed data processing. Reproducibility & tooling: Reproducible environments (UV/Docker) and MLflow or equivalent, on AWS with Spark, Databricks and Snowflake. Engineering ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

DynamoDB. Work with large volumes of structured and unstructured data, integrating information from multiple sources. Develop data solutions using technologies including Python, Java, Scala, Spark and SQL. Write scripts, develop APIs and SQL queries, and contribute to the design and deployment of scalable cloud-based data platforms. Work closely … data environment. Strong experience with AWS cloud technologies and data engineering services. Hands-on experience with one or more of Python, Java, Scala, Spark and SQL. Experience working with large-scale structured and unstructured datasets. Understanding of data ingestion, processing, transformation and integration. Strong analytical and problem-solving skills. ...

Proficiency in the Python data science ecosystem, including NumPy, Pandas, scikit-learn, PyTorch and TensorFlow Experience with big data and distributed analytics platforms, including Spark and Hadoop Hands-on experience with Natural Language Processing, graph analytics and advanced analytical methods Experience using cloud analytics platforms, with strong … Industry Keywords Secure Environments Sensitive Data Handling Regulated Environments Operational Decision-Support Systems Intelligence Data Exploitation Tools & Technologies NumPy Pandas Scikit-learn PyTorch TensorFlow Spark Hadoop Analytical Tools Dashboards Cloud Native Tooling #J-18808-Ljbffr ...

iterative delivery—making pragmatic trade‐offs and documenting them clearly. Technology Stack Languages: Predominantly Python and Node.js Distributed/data platforms: HDFS, HBase, Spark, plus increasing use of Kubernetes and cloud services Storage/search: MongoDB, OpenSearch Orchestration: Airflow, Dagster, NiFi Tooling: GitHub, GitHub Actions, Rundeck, Jira, Confluence Deployment … plus) Experience debugging moderately complex systems and improving reliability/performance Strong fundamentals: data structures, testing, version control, Linux basics Nice to have: Spark/PySpark experience Hadoop ecosystem exposure (HDFS/HBase) Workflow orchestration (Airflow/Dagster/NiFi) Search/indexing (OpenSearch, MongoDB) Kubernetes and infrastructure ...

project objectives and timelines. Hands-on Kubernetes experience Hands-on AWS experience Databases and middleware — Postgres, Kafka, Redis Big data stack — data warehousing, MPP, Spark, Flink Micro-service architecture and service mesh UI development experience What You Will Bring 3+ years of software development experience Strong command … Jenkins or CodeDeploy Preferred Hands-on Kubernetes experience Hands-on AWS experience Databases and middleware — Postgres, Kafka, Redis Big data stack — data warehousing, MPP, Spark, Flink Micro-service architecture and service mesh UI development experience Benefits Full time benefits for FT #J-18808-Ljbffr ...

interest or experience in data/network security would be a nice-to-have Datadog/Grafana/Prometheus Data related products (Airflow, Jupyter, Spark, dbt etc) The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems and solve … tracking CircleCI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data modelling SparkSQL for analytics Streamlit for data applications Are you ready for a career with us? We want ...

technologies and innovative approaches, including AI. The environment includes: C#/.NET Core Python SQL Server Microsoft Azure Azure Kubernetes Service (AKS) Kubernetes Redis Apache Kafka Spark HBase/HDFS REST APIs Microservices and distributed architectures We're particularly interested in individuals with: Extensive experience within Credit Risk ...

cross‐functional teams and setting best practice standardsDesigning data models, ETL pipelines, warehousing solutions and distributed systemsBuilding and supporting real‐time streaming platforms using Apache Flink and related technologiesEnsuring compliance with industry standards and government regulationsTracking delivery against agreed designs and managing design deviationsSupporting pre‐sales and bids with … experience in data architecture on complex, secure programmesStrong hands‐on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonusProven experience using Apache Flink for real‐time, high‐throughput data processingExperience leading, mentoring and developing technical teamsConfident engaging with senior technical and non‐technical stakeholdersKnowledge of recognised ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

Build and manage CI/CD pipelines and Infrastructure as Code using Terraform, GitLab CI, and GitHub. Support big data platforms using Hadoop, Hive, Spark, EMR, Athena, and Redshift. Implement monitoring, logging, security, and compliance controls across cloud environments. Drive automation, containerisation, and platform resilience using Kubernetes, Python … with Terraform and CI/CD tools (GitLab CI, Concourse, GitHub). Strong Linux administration experience. Experience with big data technologies including Hadoop, Hive, Spark, and Amazon EMR (or equivalent). Knowledge of monitoring, logging, and security tools. Desirable Skills Kubernetes/EKS or other container platforms. Python scripting ...

driving delivery across complex engagements. Key Responsibilities Lead design and implementation of enterprise-scale data pipelines and lakehouse architectures Provide technical leadership across Databricks, Spark and cloud data platforms Drive delivery of data engineering workstreams within Agile programmes Engage with client stakeholders to translate business objectives into technical solutions … Produce architecture documentation, design artefacts and delivery roadmaps Key Requirements 7+ years experience in data engineering or data platform development Strong experience with Databricks, Spark and distributed data processing Experience with Unity Catalog, Delta Lake and streaming architectures Advanced Python and Microsoft SQL Server development Proven experience leading engineering ...

workflows, applications, transformations, and reusable components. Support the migration of data assets, ETL pipelines, and workloads from the Cloudera Hadoop platform (HDFS, Hive, Impala, Spark) to AWS Cloud. Design, develop, and optimise data ingestion, transformation, and loading processes into Amazon S3 Data Lake and Amazon Redshift. Develop and maintain … based data integration, transformation, validation, and reconciliation processes. Integrate and support data services including AWS Glue Data Catalog, Athena, Redshift Spectrum, EMR Spark Perform data migration, testing, reconciliation, and unit testing activities to ensure data integrity and successful platform migration. Create and maintain Unix/Linux shell scripts supporting ...

have:·Demonstrable hands-on experience in deploying and maintaining machine learning models in production environments·Processing massive amounts of structured and unstructured data using Spark/SQL/Hive·Familiar with offline (batch) and online (live/stream) data pipelines·Writing production code in Python·Understanding the full software … state-of-the-art data analytics products and solutions·Carrying out A/B test experiments·Experience in Big Data technologies (Hadoop, Spark)·Experience in data pre-processing, Machine Learning, and data visualisation·Experience with large relational and NoSQL databases·Communicating findings to non-technical audience·Mentoring and supporting ...

modeling and the creation of type-safe and easily testable batch jobs. The team also orchestrates a compute environment that simplifies the execution of Spark jobs,facilitatingthe analysis of terabytes of data with ease and cost-effectiveness. The runtime environment is based on Hadoop clusters on EC2 spot instances … using our framework and services Identifyingand implementing computational improvements Making improvements and/or fixing bugs bysubmittingpatches toopen-sourceprojects in our stack, such as Spark,Iceberg,Hadoop, Airflow, and many others Acting as anexpert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge-sharing ...

modeling and the creation of type-safe and easily testable batch jobs. The team also orchestrates a compute environment that simplifies the execution of Spark jobs, facilitating the analysis of terabytes of data with ease and cost-effectiveness. The runtime environment is based on Hadoop clusters on EC2 spot … Identifying and implementing computational improvements Making improvements and/or fixing bugs by submitting patches to open-source projects in our stack, such as Spark, Iceberg, Hadoop, Airflow, and many others Acting as an expert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge ...

technical stakeholders, including customers, sales, and product teams Experience with deep learning statistical models such as Transformers B2B product experience desirable Experience with Spark SQL, PySpark, and Databricks desirable Experience with CausalAI modelling frameworks such as DoWhy and EconML desirable Core Competencies Demonstrates expertise in designing and building behavioural … Data Science Techniques Behavioural Insights Soft Skills Creative Problem Solving Collaboration Communication Industry Keywords B2B Product Experience Agentic AI Behavioural Data Science Tools & Technologies Spark SQL PySpark Databricks #J-18808-Ljbffr ...

Hands‐on experience in system design, application development, testing, and operational stability Proficient in coding in one or more languages such as Java or Spark Experience developing, debugging, and maintaining code in a large corporate environment with modern programming and database querying languages Knowledge of the Software Development Life … within a technical discipline (e.g., cloud, artificial intelligence, machine learning, mobile) Ability to solve data‐oriented problems using technologies such as SQL, Relational DB, Spark, NoSQL, optimizing for performance Preferred Qualifications, Capabilities, and Skills Experience with Spark performance tuning of complex calculations on large datasets Experience delivering production ...

within a technical discipline (e.g., cloud, artificial intelligence, machine learning, mobile)Ability to solve data-oriented problems using technologies such as SQL, Relational DB, Spark, NoSQL, optimizing for performancePreferred Qualifications, Capabilities, and Skills: Experience with Spark performance tuning of complex calculations on large datasetsExperience delivering production changes ...