51 to 75 of 77 Remote Apache Spark Jobs

manage multiple priorities and respond calmly to issues. • Self-motivated with a collaborative approach. Preferred: • Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. • Experience supporting data products or client-facing datasets. • Understanding of release testing, UAT or production support processes. • Exposure to Jira/Agile ...

manage multiple priorities and respond calmly to issues. • Self-motivated with a collaborative approach. Preferred: • Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. • Experience supporting data products or client-facing datasets. • Understanding of release testing, UAT or production support processes. • Exposure to Jira/Agile ...

loop, calibration, guardrails, auditability) Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...

using data mining techniques and with statistical rigor Strong technical skills regarding data analysis, statistics, machine learning and programming. Strong working knowledge of Hadoop, Spark, SQL, Python and/or R. Excellent communication skills Strong skills in translating business problems into data problems Strong data visualization skills Strong skills ...

based languages (Java, Kotlin), ideally combined with Python and experience in Spring Boot Solid understanding of data engineering tools and frameworks, like Spark, Flink, Kafka, dbt, Trino, and Airflow. Hands-on experience with cloud environments (AWS, GCP, or Azure), infrastructure-as-code practices, and ideally container orchestration with Kubernetes. … with experience in CI/CD pipelines, monitoring, and observability tools (Grafana or equivalent). Exposure to analytics, reporting, and BI tools such as Apache Superset, Lightdash or OpenSearch Willingness to work across the stack by contributing to API development and, at times, UI components (Vue.js, Zoho, or similar ...

performance. Desirable Skills Experience working with data platforms, data pipelines or data-heavy environments. Exposure to batch or streaming data workloads, such as Kafka, Spark, Airflow, Glue, EMR, Databricks or similar. Data observability experience. Previous public sector experience across multiple engagements. Consultancy experience would be beneficial, especially within environments ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … technologies such as Iceberg, Spark, PostgreSQL, and cloud-native infrastructure on AWS (or equivalent). You write clean, readable, and testable Python code, applying sound abstractions, naming, and review discipline to build systems that scale beyond the first version. You bring strong analytical and problem-solving skills ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats, as well ...

with predictive and diagnostic analytics in CLTV, risk scoring, financial crime, or marketing Technical proficiency in SQL databases and tools like Python, R, or Spark Strong expertise in Tableau specifically Solid background in Financial Services, with experience in Capital Markets, Insurance, or Wealth & Asset Management Bonus Points For Previous ...

team. What We’re Looking For Proficient in one of Python, Scala or Java, with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience … Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

experience What We’re Looking For Strong hands-on experience with Python, Java, or Scala Proficiency in AWS cloud environments and big data tech (Spark, Hadoop, Airflow) Solid understanding of SQL, ETL/ELT approaches, and data modelling techniques Experience building CI/CD pipelines with tools like Jenkins … CircleCI Knowledge of data security protocols and distributed system design Bonus Points For Experience with messaging systems (Kafka, Spark Streaming, Kinesis) Familiarity with schema design and semi-structured data formats Exposure to containerisation, graph databases, or machine learning concepts Proficiency with cloud-native data tools (BigQuery, Redshift, Snowflake) Enthusiasm ...

insights that drive the AI product roadmap forward. Tech Stack Includes : AWS: EC2, Athena, S3, Kinesis, Lambda, Step Functions, RDS, QuickSight Languages & Frameworks: Python, Spark, TensorFlow, LLMs & LLM coding tools Databases & Search: MySQL, PostgreSQL, SOLR Requirements Experience: Minimum 3 years of post-degree commercial experience in a data science ...

data platforms (AWS, Azure, GCP). • Develop & Optimise robust data models, warehouses and 'big data' architectures. • Integrate & Transform data using tools such as Databricks, Spark, Kafka, Talend, Informatica, AWS Glue or Azure Data Factory. • Collaborate & Influence with cross-functional agile teams and client stakeholders. • Automate & Secure CI/… role. Essential qualifications: • You should have hands-on experience with cloud platforms such as AWS, Azure or GCP. • Strong proficiency in Python, SQL, Scala, Spark-SQL, JavaScript or PowerShell. • Proven ability to design and build data pipelines and workflow solutions. • Experience with data tools such as Databricks, Spark ...

platform role Strong hands-on experience with AWS (EC2, S3, RDS, Lambda, Redshift, Glue, Athena) Experience with real-time and batch data processing - Apache Flink, Kafka, Spark, or similar Proficiency in one or more of: Python, Java, Scala Experience with data modelling , ETL processes, and distributed systems Comfortable ...

contribute to best practice implementation and continuous improvement within cross-functional engineering teams. What You’ll Do Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog Build real-time event-driven solutions with tools such as Kafka and Azure Event Hubs Apply DevOps principles … knowledge of DevOps tools and CI/CD processes Bonus Points For Development experience in Scala or Java Familiarity with Cloudera, Hadoop, HIVE, and Spark ecosystem Understanding of data privacy regulations, including GDPR, and experience working with sensitive data Ability to learn and adapt new technologies quickly to meet ...

Purpose of position Data sits at the heart of the company. This role ensures that Awin can fully leverage the data available across the group, enabling the development of robust reporting capabilities that support stronger ...

validation (e.g., schema tests, data quality checks). Manage data documentation and lineage within DBT. Design and develop scalable data pipelines using Databricks and Apache Spark. Build and optimize data workflows for batch and real-time processing. Transform and process large datasets using PySpark, SQL, and Delta Lake. Implement … terraform enterprise , Snowflake terraform provider knowledge. Snowflake : Snow sight , Python SDK for snowflake , Stage , storage integration , Data marketplace , direct shares , listings , catalog linked database , Apache Iceberg , external tables , Catalog integrations , Snowpipe , external volume , Cortex ai ,privatelink. Airflow : Dag creation , operators , connections , deployment on Kubernetes , Astronomer. DBT. ...

Senior Data Engineer 6 Month contract with extension Full-time Inside IR35 Hybrid 1 day every two weeks in the office Nottingham - NG1 4BX Join our client in embarking on an ambitious data transformation journey ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environments As a Senior Data Engineer, you'll join our Data Platform and Operations ...

Job Description Join DWP Digital and lead the future of data engineering in one of the UK's largest and most impactful digital environmentsAs a Senior Data Engineer, you'll join our Data Platform and ...

solving complex challenges and improving data quality as the platform scales. The environment is modern and cloud based, with exposure to technologies such as Spark, Snowflake and BigQuery across AWS, GCP and Azure. You will have the opportunity to support and mentor other engineers, contribute to engineering standards … engineers What We Are Looking For Proven Python and SQL experience within production environments Experience delivering data transformation projects end to end Experience with Spark, Snowflake, BigQuery or similar technologies Solid experience working with AWS, GCP or Azure Comfortable working with complex, imperfect datasets Experience owning solutions from design ...