Data Engineer (City of London)

Company

My client are a fast-growing startup revolutionising the hospitality industry by connecting businesses with a flexible, student-powered workforce. The platform enables partners—from local pubs to multinational chains—to optimise staffing in real-time, enhancing efficiency and profitability.

The Role

As a Data Engineer, you'll play a pivotal role in shaping the data infrastructure. Your responsibilities will include:

• Building and Maintaining Data Pipelines: Design and implement scalable ETL processes to integrate data into our mobile and web applications, as well as internal dashboards.
• Data Architecture Design: Develop and optimise data storage, retrieval, and processing systems to ensure efficiency and scalability.
• Collaboration: Work closely with data scientists, software engineers, and internal stakeholders to understand data needs and deliver solutions that drive data-driven decisions.
• Monitoring and Troubleshooting: Oversee pipeline performance, address issues promptly, and maintain comprehensive data documentation.

What You’ll Bring

• Technical Expertise: Proficiency in Python and SQL; experience with data processing frameworks such as Airflow, Spark, or TensorFlow.
• Data Engineering Fundamentals: Strong understanding of data architecture, data modelling, and scalable data solutions.
• Backend Development: Willingness to develop proficiency in backend technologies (e.g., Python with Django) to support data pipeline integrations.
• Cloud Platforms: Familiarity with AWS or Azure, including services like Apache Airflow, Terraform, or SageMaker.
• Data Quality Management: Experience with data versioning and quality assurance practices.
• Automation and CI/CD: Knowledge of build and deployment automation processes.

Nice to Have

• Production Experience: Experience building and maintaining data pipelines in a live environment.
• Data Storage Solutions: Familiarity with data lakes, warehousing, and other data storage patterns.
• Advanced Tools: Experience with tools like Kafka, Jenkins, Athena, or Spark.

This role will require 4 days per week onsite in the office, this is not optional and you must be open to this in order to proceed with the role.