the project. There is a massive emphasis on Pyspark and Databricks for this particular role. Technical Skills Required: Azure (ADF, Functions, Blob Storage, Data Lake Storage, Azure Data Bricks) Databricks Spark DeltaLake SQL Python PySpark ADLS Day To Day Responsibilities: Extensive experience in designing, developing, and more »
Airbyte for data ingestion, Prefect for pipeline orchestration, AWS Glue for managed ETL, along with Pandas and PySpark for pipeline logic implementation. We utilize DeltaLake and PostgreSQL for data storage, emphasizing the importance of data integrity and version control in our workflows. Day-to-day, you will more »
City of London, London, United Kingdom Hybrid / WFH Options
Concept Resourcing
success of various IT salesforce projects. Key Responsibilities: Architecture, modelling and leadership skills Strong Azure data skills, including: Azure Data Factory V2, Azure Data Lake Storage V2, Azure Databricks, Azure Function Apps& Logic Apps, Azure Stream Analytics, Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell … Strong PySpark, DeltaLake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc. Strong repositories, CI/CD skills. Strong knowledge of Kimball more »
ETL processes to read, transform, and verifying data using tools such as SQL and PySpark Cloud technologies and architecture including Databricks, Kubernetes, Data/DeltaLake, Azure Machine Learning, Data Factory. Azure is preferable, but AWS and GCP experience is welcome Familiar with all parts of the Machine more »
and constructing robust data pipelines using the best of open-source data engineering and scientific Python toolset. Tech Stack: Airbyte AWS Glue Pandas Pyspark DeltaLake PostgreSQL The team follows agile ways of working and you engage with various stakeholders across the business. The role is Hybrid more »
SE1, Tower of London, Greater London, United Kingdom Hybrid / WFH Options
Avanti Recruitment
working anywhere in the world for up to 30 days! Requirements: Azure Data Bricks Data Strategy Data Transformation Data Cloud Migration Data API Design DeltaLake/Lakehouse Strong communication skills and also being able to translate to a non-technical audience Previous Finance/Fintech/Forex more »