Ingestion Engineer
Job Description As an Ingestion Engineer in the DDF team, your key responsibilities are as follows:
- Technical Proficiency:
- Collaborate in hands-on development using Databricks, Python, PySpark, ADF, and other relevant technologies to ingest data for downstream consumption.
- Assist in engineering and managing data models and pipelines within a cloud environment, utilizing technologies like Databricks, Spark, Delta Lake, and SQL.
- Contribute to the maintenance and enhancement of our progressive tech stack, which includes Python, PySpark, Logic Apps, Azure Functions, ADLS, Django, and ReactJs.
- Support the implementation of DevOps and CI/CD methodologies to foster agile collaboration and contribute to building robust data solutions.
- Develop code that adheres to high-quality standards, promoting a scalable and maintainable data platform.
- Learning and Growth; Contribution to Solutions:
- Collaborate with the team to learn and apply the best practices in data engineering.
- Actively participate in engineering projects, gaining experience in developing high-quality, scalable, and sustainable data solutions.
- Stay updated with emerging technologies and trends in data engineering, contributing to the team's knowledge base by sharing insights and ideas.
- Assist in the development of data solutions within the Pet Nutrition data platform, working on challenging aspects under the guidance of senior team members.
- Contribute to the management of data from various divisions to generate valuable data assets related to pets and pet owners.
- Support the maintenance of a semantic and intelligent data layer to contribute to the comprehensive leadership of the data solution within the environment.
- Collaboration and Communication:
- Collaborate closely with analysts, data scientists, and team members to understand their requirements and assist in translating them into actionable data solutions.
- Maintain effective communication with the Data Ingestion Lead, actively participating in team discussions and sharing ideas to improve platform excellence.