5 of 5 Permanent AWS Step Functions Jobs in Scotland

pipelines, data lakes, and data warehouse solutions on AWS. The role focuses on developing high-performance data engineering solutions using PySpark, Spark, Python, and AWS services, enabling secure, reliable, and efficient data processing and analytics across enterprise platforms. Key Responsibilities Design, develop, and maintain scalable batch and real-time … data pipelines using PySpark, Spark, Python, and AWS services. Build and optimize data lakes and data warehouse solutions ensuring data quality, security, and accessibility. Develop reusable, production-grade ETL/ELT frameworks and data processing solutions. Implement orchestration workflows using AWS Step Functions, Airflow, and other ...

beyond routine approaches to solve technical problems. Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, and Databricks on AWS, processing and transforming large-scale financial datasets for analytics and reporting. Develop fact and dimension data models for reporting and analytics. Write secure, high … PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security, and lineage best practices. Experience with cloud-based ...

beyond routine approaches to solve technical problems. Design, develop, and maintain scalable data pipelines and processing workflows using Python, PySpark, SQL, and Databricks on AWS, processing and transforming large-scale financial datasets for analytics and reporting. Develop fact and dimension data models for reporting and analytics. Write secure, high … PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security, and lineage best practices. Experience with cloud-based ...

create tomorrow, together, today. About the role and team: Role overview CreateFuture is delivering the migration of an ML estate from Databricks to an AWS SageMaker-based MLOps platform, working alongside AWS. The Senior Data Engineer (ML) sits in the Databricks workstream team (Delivery Manager, Lead ML Ops Engineer … migrated models Implement parity and statistical testing to prove migrated pipelines/models match Databricks outputs Handle data migration/integration between Databricks and AWS: storage, permissions, IAM alignment Work within CI/CD and IaC patterns for pipeline deployment; document runbooks and hand over to Evoke teams Collaborate ...

inputs/outputs, and adherence to resiliency and security expectations; experience coaching senior engineers/leads on compliant usage patterns and controls. Strong AWS background, including serverless orchestration (Step Functions/Lambda), container deployment (ECS/EKS), and managed data services Proven track record designing and evolving … mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation. About The Team Our professionals in our Corporate Functions cover a diverse range of areas from finance and risk to human resources and marketing. Our corporate teams are an essential part ...