5 of 5 AWS Step Functions Jobs in the North West

Required Experience: Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native ...

sequence interdependent migration work, making sound technical decisions before all edge cases are known and adapting as issues surface What we’re looking for AWS & SageMaker (must have) Amazon SageMaker (Studio, Training, Pipelines, Endpoints) — expert level; you can architect and operate the full lifecycle AWS IAM — advanced; writes … least-privilege policies from scratch, not just modifies examples Amazon S3 — advanced; including lifecycle policies, encryption, and bucket policies AWS KMS — working knowledge of key management in an ML context AWS CI/CD tooling (CodePipeline/CodeBuild or equivalent) — advanced; you've automated model promotion across environments ...

Experience Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services ...

sources Contribute to the completion of milestones associated with your project Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka, AWS SQS, Other Cloud Queuing Native ...

Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience across AWS, Azure and/or GCP. Knowledge of messaging and streaming technologies … including Kafka, AWS SQS or equivalent cloud-native services. Strong understanding of SQL and NoSQL databases. Experience with modern data storage technologies including HDFS, Iceberg, Elasticsearch, S3 and Data Lake architectures. Containerisation and orchestration experience using Docker, Kubernetes and/or OpenShift. Knowledge of testing frameworks, engineering standards ...