algorithms). Expert in SQL and Python, DBT, and standard data science libraries and frameworks such as pandas, numpy, scikit-learn, TensorFlow/PyTorch, PySpark, etc. Experience with Infrastructure-as-Code, Continuous Integration & Deployment patterns. Experience with PostgreSQL, Google Cloud Platform, and BigQuery. Excellent problem-solving skills and the more »
ML and AI models to get results code use your extensive coding background to remains hands-on and deliver projects on our modern Azure, pyspark-based data infrastructure understand how the IB business works, what our stakeholders and clients really need, and how you can make a difference You more »
Pipeline using AWS services (S3, Lambda, Glue, Athena, QuickSight, etc.) or similar native Azure or GCP services. Strong experience on Snowflake or DataBricks (including PySpark) along with tools like DBT or Snaplogic or Informatica Stack Proactively seek to understand major industry trends and apply them to solutions, new initiatives more »
start interviewing ASAP. Responsibilities: Azure Cloud Data Engineering using Azure Databricks Data Warehousing Data Engineering Very strong with the Microsoft Stack ESSENTIAL knowledge of PySpark clusters Python & C# Scripting experience Experience of message queues (Kafka) Experience of containerization (Docker) FINANCIAL SERVICES EXPERIENCE (Energy/commodities trading) If you have more »
experience in PowerBi would also be useful. You will be a Engineer with past experience with java, data, and infrastructure (devOps). Java, Python, PySpark Mechanisms: MongoDB, Redshift, AWS S3 Environments/Infra: AWS (required), [AWS Lambda, Terraform] (nice to have) Platforms: Creating data pipelines within Databricks or equivalent more »
experience in Data Modeling within a cloud-based data platformStrong experience with SQL ServerAzure data engineering stack, including Azure Synapse and Azure Data LakePython, PySpark and T-SQL In return you will be offered a competitive salary and benefits package, remote working options and an opportunity to work with more »
vision to convert data requirements into logical models and physical solutions. with data warehousing solutions (e.g. Snowflake) and data lake architecture, Azure Databricks/Pyspark & ADF. retail data model standards - ADRM. communication skills, organisational and time management skills. to innovate, support change and problem solve. attitude, with the ability more »
Mart. Utilize Vector Databases, Cosmos DB, Redis, and Elasticsearch for efficient data storage and retrieval. Demonstrate proficiency in programming languages including Python, Spark, Databricks, Pyspark, SQL, and ML Algorithms. Implement Machine Learning models and algorithms using Pyspark, Scikit Learn, and other relevant tools. Manage Azure DevOps, CI/… environments, Azure Data Lake, Azure Data Factory, Microservices architecture. Experience with Vector Databases, Cosmos DB, Redis, Elasticsearch. Strong programming skills in Python, Spark, Databricks, Pyspark, SQL, ML Algorithms, Gen AI. Knowledge of Azure DevOps, CI/CD pipelines, GitHub, Kubernetes (AKS). Experience with ML/OPS tools such more »
Data Engineer - AWS, Python, Spark, Pyspark, SQL - Hybrid - Outside IR35 We are seeking a skilled AWS Data Engineer to support our client in Manchester on a 6 month contract. The successful candidate will spearhead the design, implementation, and upkeep of a data warehouse, ensuring it operates at peak performance more »
Mid-or-Senior level Data Scientist Solid knowledge of Data Engineering principles, including productionisation Technical experience with some or all of the following: Python, PySpark, scikit-learn, pandas, Azure Data Services, Databricks. If this sounds of interest, please apply. more »
not received on time. Communicating outages with the end users of a data pipeline What We Value Comfortable reading and writing code in Python, Pyspark and Java. Basic understanding of Spark and interested in learning the basics of tuning Spark jobs. Data pipeline monitoring team members should be able more »
performance, scalability, and reliability. Technical Skills required: RedShift Glue (inc. Glue Studio, Glue Data Quality, Glue DataBrew) Step Functions Athena Lambda Kinesis Python, Spark, Pyspark, SQL Your contributions as a Data Engineer will directly impact the organization's operations and revenue. In addition to a competitive annual salary, we more »
Senior Databricks Migration Consultant Remote This role demands in-depth knowledge of data engineering, cloud technologies (preferably AWS), and a successful record in enterprise level data migrations into Databricks. You will ensure the efficient and secure transition of our data more »
South East London, London, United Kingdom Hybrid / WFH Options
The Bridge (IT Recruitment) Limited
a long term contract, inside IR35 on a remote basis. The key skills required for this Python Developer role are: Python ETL Azure Databricks Pyspark If you do have the required skills for this remote Python Developer contract, please do apply. more »
engineering leaders/stakeholders in decision making and implementing the models into production. You will need to have hands on skills in Python and PySpark, experience working in a cloud environment and knowledge of development tools like Git or Docker. You can also expect to work with the latest more »
Qualifications PhD/Degree/MSc in a highly numerate subject (e.g. Data Analytics, MORSE, Mathematics, Statistics, Physics) Mandatory Experience Knowledge of Python/Pyspark, SQL or R programming. Experience of applying appropriate analytical tools and methods to provide high quality quantitative and qualitative analysis. Able to identify information more »
Employment Type: Contract
Rate: £20 - £20.47/hour Hybrid Working, Flexible Working
engineering concepts and technologies Experience working in a cloud environment Experience with modern and traditional data warehousing and data processing technologies and concepts (Hadoop, PySpark, Streaming Data) #J-18808-Ljbffr more »
Engineer you will be pivotal in designing, developing, and maintaining data architecture and infrastructure. The ideal candidate should have a strong foundation in Python, PySpark, SQL, and ETL processes, along with proven experience in implementing solutions in a cloud environment. Roles & Responsibilities: Experienced Data Engineer with a background in … and mastering to management and distribution of large datasets. Mandatory Skills: 6+ Years of experience in Design, build, and maintain data pipelines using Python, PySpark and SQL. Develop and maintain ETL processes to move data from various data sources to our data warehouse on AWS. Collaborate with data scientists more »
Lead Azure Data Engineer PySpark (Python) & Synapse Tech for Good/Charity Essential skills required: Azure solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. You will be tested on PySpark Azure Synapse - ESSENTIAL as is used heavily Spark Azure Data … people comprising developers, data engineers, QA and DevOps. Essential skills required: Azure solid experience required of the Azure Data ecosystem Python - ESSENTIAL as PySpark is used heavily. You will be tested on PySpark Azure Synapse - ESSENTIAL as is used heavily Spark Azure Data Lake/Data Bricks/… architecture Familiar with Synapse CI/CD Azure Purview or another governance tool experience. Familiar with building Catalogs and lineage Lead Azure Data Engineer PySpark (Python), Synapse, Data Lake Tech for Good/Charity more »
Shefford, Bedfordshire, South East, United Kingdom Hybrid / WFH Options
Central Bedfordshire Council
pipelines using Azure Data ETL tools such as Data Factory and Databricks. An understanding of legacy SSIS packages will be beneficial Write Python/PySpark scripts and T-SQL for Lakehouse/Database management (tables, views, indexes, stored procedures) Use Azure Data Factory, Databricks, and Synapse for advanced data … running with minimal supervision and proficiency in Azure Synapse Serverless SQL and associated technologies It is essential that you have experience in Python/PySpark and T-SQL scripting, Azure Data Factory, Databricks, and Synapse Analytics, designing and implementing Data warehouses and associated dimensional models, and proficiency in reporting more »
get the most from their data. They are looking for someone with core skills in SQL complimented with Azure experience (Azure Data Factory, Databricks, PySpark etc) This is a very exciting time to join as they shake things up across the industry, so please get in touch asap to … Working with and modelling data warehouses. Skills & Qualifications Strong technical expertise using SQL Server and Azure Data Factory for ETL Solid experience with Databricks, PySpark etc. Understanding of Agile methodologies including use of Git Experience with Python and Spark Benefits £65,000 - £75,000 Bonus To apply for this more »
Synapri collaborates with a large transport company, who is seeking a Senior Data & Analytics Developer for a 12 months contract. You will be involved in the design, development and configuration of applications, components and tools according to the technical plans more »
prem, but the direction of travel is cloud engineering. You'll be executing code in different places across the following tech stack: Azure, Databricks, PySpark and Pandas. You will steer the data engineering function within a wider product team. There'll be lots of connecting and interaction with stakeholders … inclusive environment where you can grow your career and make a real difference. Essential criteria: Enterprise-scale experience with Azure data engineering tools, Databricks, PySpark and Pandas Experience of data modelling and transforming raw data into datasets Experience of building team capability through role modelling, mentoring, and coaching Able more »
