Senior Data Engineer

Data Engineer – SC Cleared

Whitehall Resources are currently looking for a Data Engineer on a remote basis for an initial 5-month contract.

***MUST BE SC CLEARED***

***INSIDE IR35***

Job Spec:

Main responsibilities:

· Data engineering at scale in government or similarly complex domains, including performance tuning and data‐quality management

· Hands‐on expertise in AWS & Spark: Amazon EMR, S3, Lambda; strong PySpark/Python and SQL for large‐scale batch processing.

· CI/CD & DevOps: pipelines and IaC (e.g., Terraform), automated testing, and release governance.

· Version control & collaboration: Git/GitLab, code review, branching strategies, and trunk/PR workflows

· APIs & integration: building/consuming data services to move and expose data safely and reliably.

· Agile ways of working with Jira/Confluence; clear stakeholder communication and concise technical documentation

· Data warehousing & modeling (e.g., Redshift; dimensional modeling; dbt).

· Basic Power BI familiarity to partner with BI developers and validate end‐to‐end data flows.

Required Skills:

• Informatica experience

• Data project experience

• AWS

• Azure

Desirable Skills:

· AWS ecosystem depth (Athena, Redshift, EC2, CloudWatch, IAM) and event‐driven patterns.

· AWS Certified Cloud Practitioner (or higher), Azure AI Fundamentals (awareness of ML/AI services).