Data Engineer (DataOps/Big data)

We are hiring For Data Engineer (DataOps/Big data)

Location: Hybrid 2 to 3 Days per week in Glasgow

• Hands-on experience in building, maintaining, and troubleshooting production batch data pipelines in financial services environments.
• Orchestrating batch workflows using Apache Airflow, building reusable pipeline components, handling live production support (investigating failed DAGs, checksum failures, and delivery delays), and applying data control frameworks (reconciliation, data quality, and audit tracking).
• Should have prior Experience in Investment Banking/Financial Services Technology, focusing on large-scale finance datasets, data reconciliation, metadata processing, data lineage, and strict technology change control standards.
• Prior expertise using Scala, Python, Apache Spark (PySpark/Sharedview Spark), SQL/PostgreSQL, and Starburst/Trino and DevOps Tools Docker, GitLab CI/CD, AWS S3, and container orchestration platforms (OpenShift, Kubernetes, or Helm).

Key Skills: Data Orchestration/Apache Spark (PySpark/Sharedview Spark), Scala, Python/Data Querying & Analysis/SQL, PostgreSQL, Starburst/Trino, Metadata Processing