4 Data Lake Jobs in Yorkshire

Lead Data Engineer: We need some strong Data engineer profiles… they need good experience with Pyspark, Python, SQL, ADF and preferably Databricks experience Job description: Building new data pipelines and optimizing data flows using the Azure cloud stack. Building data products from scratch. Support Business Analysts and Data Architects with discovery and best practices. Self-directed and comfortable supporting the data needs of multiple teams, systems and products. What’s required ? Advanced Azure knowledge and experience working and migrating data products from … Azure Synapse Analytics, Azure Analysis Services Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics Experience with Object-oriented/object function scripting languages: Python preferred more »

a comprehensive plan for implementation. Responsibilities: Collaborate with stakeholders to understand business requirements and translate them into effective solution designs for our party data model data platform. Architect and implement cloud-based solutions that leverage popular data platforms to enable efficient storage, processing, and … analysis of large-scale data. Design scalable and resilient architectures that ensure high availability, fault tolerance, and disaster recovery for our data platform. Assess and select appropriate data platforms and technologies based on functional and non-functional requirements, considering factors such as performance, scalability, security, and … platforms such as Amazon Web Services (AWS) (e.g., S3, RDS, Redshift, Athena), Microsoft Azure (e.g., Blob Storage, SQL Database, Data Lake Analytics), or Google Cloud Platform (e.g., Cloud Storage, BigQuery). Deep understanding of cloud computing concepts, including infrastructure as code, serverless computing, and containerization. more »

and platforms. Role purpose/summary Collaborate with stakeholders to understand business requirements and translate them into effective solution designs for our party data model data platform. Architect and implement cloud-based solutions that leverage popular data platforms to enable efficient storage, processing, and … analysis of large-scale data. Design scalable and resilient architectures that ensure high availability, fault tolerance, and disaster recovery for our data platform. Assess and select appropriate data platforms and technologies based on functional and non-functional requirements, considering factors such as performance, scalability, security, and … platforms such as Amazon Web Services (AWS) (e.g., S3, RDS, Redshift, Athena), Microsoft Azure (e.g., Blob Storage, SQL Database, Data Lake Analytics), or Google Cloud Platform (e.g., Cloud Storage, BigQuery). Deep understanding of cloud computing concepts, including infrastructure as code, serverless computing, and containerization. more »

Define Architecture standards and principals • Translation for business requirements into Data Design, technical specifications • Data migration, strategy, mapping/lineage, quality, standards across multiple projects and solutions • Work closely with the development team to design and implement data solutions • Being able to recommend the … best place to source data sets, both internal and external to the organisation • Ability to capture and document standards and designs that can be easily understood for all stakeholders (DBAs through to CIO) • Data migration and the ability to be consulted … with data movements for data performance and optimisation • Microsoft experience (SQL DW/Cosmos DB/Azure Data Lake) • Working collaboratively with stakeholders, colleagues, and strategic partners across a portfolio of projects more »