3 Permanent Data Lake Jobs in Yorkshire

Lead Data Engineer: We need some strong Data engineer profiles… they need good experience with Pyspark, Python, SQL, ADF and preferably Databricks experience Job description: Building new data pipelines and optimizing data flows using the Azure cloud stack. Building data products from scratch. Support Business Analysts and Data Architects with discovery and best practices. Self-directed and comfortable supporting the data needs of multiple teams, systems and products. What’s required ? Advanced Azure knowledge and experience working and migrating data products from … Azure Synapse Analytics, Azure Analysis Services Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics Experience with Object-oriented/object function scripting languages: Python preferred more »

existing processes and systems. Foster productive relationships with key internal stakeholders and external suppliers. Utilize subject matter expertise to integrate, transform, and consolidate data from diverse structured, unstructured, and streaming data systems into appropriate schemas for analytics solutions. Assist stakeholders in comprehending data through … exploration and develop secure, compliant data processing pipelines using a variety of tools and methods. Leverage Azure data services and frameworks to store and generate refined datasets for analysis, with the flexibility to adopt different architecture patterns such as Modern Data Warehouse (MDW), Big … architecture patterns. Experience with Azure Data Factory, Azure Synapse Analytics, Azure Stream Analytics, Azure Event Hubs, Azure Data Lake Storage, and Azure Databricks. Desirable: Microsoft Dynamics and Power Platform accreditations. Microsoft Azure accreditations. Familiarity with Workday configuration. Experience with modelling tools such as more »

a comprehensive plan for implementation. Responsibilities: Collaborate with stakeholders to understand business requirements and translate them into effective solution designs for our party data model data platform. Architect and implement cloud-based solutions that leverage popular data platforms to enable efficient storage, processing, and … analysis of large-scale data. Design scalable and resilient architectures that ensure high availability, fault tolerance, and disaster recovery for our data platform. Assess and select appropriate data platforms and technologies based on functional and non-functional requirements, considering factors such as performance, scalability, security, and … platforms such as Amazon Web Services (AWS) (e.g., S3, RDS, Redshift, Athena), Microsoft Azure (e.g., Blob Storage, SQL Database, Data Lake Analytics), or Google Cloud Platform (e.g., Cloud Storage, BigQuery). Deep understanding of cloud computing concepts, including infrastructure as code, serverless computing, and containerization. more »