Senior Data Engineer

You operate at the frontier of modern data engineering. You understand that AI is not a future consideration — it is a present-day design constraint. You build data infrastructure that is AI-ready by default: pipelines that serve feature stores, architectures that can support RAG and LLM applications, and platforms capable of integrating AI-assisted tooling at every stage of the engineering lifecycle.In a global team spanning Europe, the US, and India, you are a connector — bridging technical depth with business context, and aligning local delivery with global standards.Essential — TechnicalProven data engineering expertise in production, cloud-native environments.Advanced SQL proficiency, including BigQuery-specific development, query profiling, partitioning and clustering optimization, and complex analytical query design.Strong hands-on experience with Google Cloud Platform (GCP), including architecture design, implementation, and delivery of scalable production solutions.Deep, hands-on expertise across: BigQuery, Dataflow, Cloud Composer (Airflow), Pub/Sub, Dataplex, Cloud Storage, Terraform, Cloud BuildMastery of data modelling methodologies: Dimensional/Kimball, 3NF, Data Vault — with real-world application of eachProduction-level Python: OOP design patterns, async processing, unit/integration testing, GCP SDK usageDemonstrated experience designing CI/CD pipelines for data productsTrack record of leading legacy-to-cloud migrationsEssential — Leadership & ProfessionalDemonstrated technical leadership: you have designed solutions, led reviews, and raised the quality bar of a teamProven ability to work in high-ambiguity environments and drive clarity through technical designStrong communication: able to write architecture decision records, run design reviews, and present to non-technical stakeholdersEvidence of mentoring junior engineers and improving team capabilityMinimum 2:2 degree (or international equivalent) in Computer Science or related technical field; demonstrated professional experience considered in lieu for internal applicantsDesiredGCP Professional Data Engineer certificationExperience designing AI/ML data pipelines — feature stores, training data pipelines, Vertex AI integrationHands-on experience with vector databases or embedding pipeline designActive use of AI-assisted development tools (Copilot, Gemini, Cursor) in production deliveryExperience with dbt Core / Dataform in a production, team settingData engineering experience in a regulated financial environment (banking, insurance, credit)Experience designing event-driven architectures with Pub/Sub and DataflowWhat You Can ExpectA defining role on a high-priority, high-visibility data platform programmeGenuine technical leadership — your architecture decisions will stand in productionDirect exposure to GenAI infrastructure, ML platforms, and AI-era data engineeringCollaboration with global engineering teams across three continentsSupport and funding for GCP Professional Data Engineer certification and advanced trainingA clear pathway to Lead Engineer for the right candidateHybrid working from a modern campus environmentThe Company is committed to diversity and equality of opportunity for all and is opposed to any form of less favourable treatment or harassment on the grounds of race, religion or belief, sex, marriage and civil partnership, pregnancy and maternity, age, sexual orientation, gender reassignment or disabilityThis position is based in Dunton, and it is expected the successful candidate will be able to attend the Dunton Campus for typically 4 days a week and remain flexible on the days they are required to attend the office according to business requirements.As part of our pre-employment checks process, successful candidates will be required to undergo a criminal record check. This will be conducted in line with the Rehabilitation of Offenders Act 1974 and applied only to unspent convictions.#LI-MM3 #FordCreditTechnical Architecture & DeliveryLead the end-to-end design and delivery of complex data engineering solutions on GCP — from architecture through production deploymentArchitect scalable, cost-effective data platforms using BigQuery, Dataflow, Cloud Composer, Pub/Sub, Dataplex, and Cloud StorageDesign robust data models using Dimensional (Kimball), 3NF, and Data Vault methodologies — selecting the right approach for each use caseImplement SCD strategies and historical data management patterns for long-lived datasetsLead the migration of legacy data structures to GCP, defining parallel testing and data parity validation strategiesProvision and govern cloud infrastructure using Terraform; champion IaC as a non-negotiable standardDesign and implement CI/CD pipelines for all data solutions — with automated testing, linting, and deployment gatesAI-Era ResponsibilitiesAI-ready architecture: Design every data platform component to be downstream-AI-compatible — appropriate partitioning, feature store integration, and schema design for ML consumptionGenAI data infrastructure: Architect data pipelines for LLM-based applications, including embedding generation pipelines, vector store population, and RAG data retrieval layersFeature store engineering: Build and maintain centralised feature stores on Vertex AI, ensuring reproducibility and low-latency serving for ML modelsAI-assisted development leadership: Champion GitHub Copilot, Gemini Code Assist, and Cursor as engineering productivity tools — set standards for how the team uses them responsiblyAI-powered data quality: Design ML-based anomaly detection into pipeline monitoring — moving beyond threshold alerts to intelligent pattern recognitionLLMOps data layer: Build the data infrastructure that underpins model evaluation, fine-tuning dataset curation, and prompt tracking pipelinesLeadership & CollaborationLead code reviews; hold the bar for quality, testability, and maintainabilityDefine and document reusable engineering patterns — pipeline templates, transformation standards, naming conventionsActively mentor junior engineers through pairing, structured feedback, and technical design sessionsWork closely with global Data Engineering counterparts to align on platform standardsEngage directly with senior business stakeholders to translate complex requirements into technical solutionsContribute to hiring: review take-home tasks, conduct technical interviews, calibrate assessmentsDefine and execute testing strategies for regulated workloads, including parallel-run validation against legacy systemsOperational ExcellenceOwn pipeline reliability: define SLAs, implement alerting, lead incident resolutionDrive DataOps practices: automated testing, data contracts, observability-first designMonitor and optimise GCP costs; propose and implement efficiency improvementsEnsure compliance with data security, encryption, and governance standards in all solutions builtFull timePosting Date: 2026-07-29

Job Details

Company
Appcast
Location
Chelmsford, Essex, UK
Posted