various departments to gather requirements and ensure data solutions reflect real business needs. Key Experience Required: Deep expertise in SQL, Python, and Spark (particularly PySpark) for building and testing end-to-end pipelines in that process both structured and semi-structured datasets. Experience mentoring peers and supporting team growth More ❯
working in cloud-native environments (AWS preferred) Strong proficiency with Python and SQL Extensive hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Familiarity with DevOps practices and infrastructure-as-code (e.g., Terraform, CloudFormation) Solid understanding More ❯
to store and process data. Document workflows, pipelines, and transformation logic for transparency. Key Skills & Experience: Strong hands-on experience in Python (Pandas, NumPy, PySpark). Experience building ETL/ELT processes. Familiarity with cloud platforms (AWS, Azure, GCP) and big data technologies (e.g., Snowflake, Databricks). Understanding of More ❯
machine learning, with a strong portfolio of relevant projects. Proficiency in Python with libraries like TensorFlow, PyTorch, or Scikit-learn for ML, and Pandas, PySpark, or similar for data processing. Experience designing and orchestrating data pipelines with tools like Apache Airflow, Spark, or Kafka. Strong understanding of SQL, NoSQL More ❯
added flexibility for diverse migration and integration projects. Prior experience with tools such as MuleSoft, Boomi, Informatica, Talend, SSIS, or custom scripting languages (Python, PySpark, SQL) for data extraction and transformation. Prior experience with Data warehousing and Data modelling (Star Schema or Snowflake Schema). Skilled in security frameworks More ❯
Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What's in it for More ❯
initiatives Key Requirements of the Database Engineer: Proven experience with Databricks, Azure Data Lake, and Delta Live Tables Strong programming in Python and Spark (PySpark or Scala) Solid knowledge of data modelling, warehousing, and integration concepts Comfortable working in Agile teams, with CI/CD and Azure DevOps experience More ❯
System Integration, Application Development or Data-Warehouse projects, across technologies used in the enterprise space. Software development experience using: Object-oriented languages (e.g., Python, PySpark,) and frameworks Stakeholder Management Expertise in relational and dimensional modelling, including big data technologies. Exposure across all the SDLC process, including testing and deployment. More ❯
similar tools is valuable (Collibra, Informatica Data Quality/MDM/Axon etc.) Data Architecture experience is a bonus Python, Scala, Databricks Spark and Pyspark with Data Engineering skills Ownership and ability to drive implementation/solution design More ❯
good communicator with your team. Key Skills Python in the software engineering level, including unit and integration test experience. Distributed computing knowledge covered by PySpark or Scala, can debug things in SparkUI and knows how to optimise for this purpose. AWS experience Good understanding of data modelling, change data More ❯
Collaborate across technical and non-technical teams Troubleshoot issues and support wider team adoption of the platform What You’ll Bring: Proficiency in Python, PySpark, Spark SQL or Java Experience with cloud tools (Lambda, S3, EKS, IAM) Knowledge of Docker, Terraform, GitHub Actions Understanding of data quality frameworks Strong More ❯
coding languages eg. Python, R, Scala, etc.; (Python preferred) Proficiency in database technologies eg. SQL, ETL, No-SQL, DW, and Big Data technologies e.g. pySpark, Hive, etc. Experienced working with structured and also unstructured data eg. Text, PDFs, jpgs, call recordings, video, etc. Knowledge of machine learning modelling techniques More ❯
key role in building their scalable data capabilities; designing ingestion pipelines, high-performance APIs, and real-time data processing systems. Key Responsibilities Stack: Python, PySpark, Linux, PostgreSQL, SQL Server, Databricks, Azure, AWS Design and implement large-scale data ingestion, transformation, and analysis solutions Model datasets and develop indicators to More ❯
queries for huge datasets. Has a solid understanding of blockchain ecosystem elements like DeFi, Exchanges, Wallets, Smart Contracts, mixers and privacy services. Databricks and PySpark Analysing blockchain data Building and maintaining data pipelines Deploying machine learning models Use of graph analytics and graph neural networks If this sounds like More ❯
skills: Typical Data Engineering Experience required (3+ years): Strong knowledge and experience: Azure Data Factory and Synapse data solution provision Azure Devops PowerBi PythonPySpark (Preference will be given to those who hold relevant certifications) Proficient in SQL. Knowledge of Terraform Ability to develop and deliver complex visualisation, reporting More ❯
in programming languages and data structures such as SAS, Python, R, SQL is key. With Python background, particularly familiarity with pandas/polars/pyspark, pytest; understanding of OOP principles; git version control; knowledge of the following frameworks a plus: pydantic, pandera, sphinx Additionally, experience in any or all More ❯
Engineering Experience required ACTIVE SC is mandatory Essential requirement: Azure Data Factory and Synapse data solution provision Azure DevOps Microsoft Azure PowerBi Python misson Pyspark Dimension Data Model Semantic Data Models, including integration to Power BI Data Engineering Capabilities Business analysis to understand service needs and and documents accurately More ❯
machine learning. Experience with deep learning or generative AI is a plus but not essential. Proficiency in (Spark)SQL and Python . Experience with PySpark is beneficial but not required. Experience designing and implementing robust testing frameworks . Strong analytical skills with keen attention to detail. Excellent communication skills More ❯
sprint planning sessions. Monitor data pipeline executions and investigate test failures or anomalies. Document test results, defects, and quality metrics. Preferred qualifications: Experience with PySpark or notebooks in Databricks. Exposure to Azure DevOps, Unit Testing frameworks, or Great Expectations for data testing. Knowledge of data warehousing or medallion architecture More ❯
data technologies Experience in designing, managing and overseeing task assignment for technical teams. Mentoring data engineers Strong Exposure to SQL, Azure Data Factory, Databricks & PySpark is a must have. Experience in Medallion Silver Layer modelling Experience in Agile project environment Insurance experience – Policy and Claims Understanding of DevOps, continuous More ❯
record of delivering machine learning or AI projects end-to-end Hands-on skills in Python, with frameworks like Scikit-learn, TensorFlow, PyTorch, or PySpark Deep understanding of data science best practices, including MLOps Strong stakeholder communication skills—able to translate complex insights into business impact Experience working in More ❯
record of delivering machine learning or AI projects end-to-end Hands-on skills in Python, with frameworks like Scikit-learn, TensorFlow, PyTorch, or PySpark Deep understanding of data science best practices, including MLOps Strong stakeholder communication skills—able to translate complex insights into business impact Experience working in More ❯
ensuring effective collaboration. Design, develop, and optimise scalable data pipelines and infrastructure using AWS (Glue, Athena, Redshift, Kinesis, Step Functions, Lake Formation). Utilise PySpark for distributed data processing, ETL, SQL querying, and real-time data streaming. Establish and enforce best practices in data engineering, coding standards, and architecture … expertise in AWS Data Services, including Glue, Athena, Redshift, Kinesis, Step Functions, Lake Formation and data lake design. Strong programming skills in Python and PySpark for data processing and automation. Extensive SQL experience (Spark-SQL, MySQL, Presto SQL) and familiarity with NoSQL databases (DynamoDB, MongoDB, etc.). Proficiency in More ❯
across clustering, propensity modelling, regression, and NLP Providing insights on their customers, pricing strategies, and the target audience YOUR SKILLS AND EXPERIENCE Python/PySpark experience is essential to create propensity models and clustering NLP experience is a plus Commercial awareness and insights experience is needed for this role More ❯
both short and long-term projects across clustering, propensity modelling, regression, and NLP Occasionally building dashboards for clients YOUR SKILLS AND EXPERIENCE Python/PySpark experience is essential to create propensity models and clustering NLP experience is a plus Commercial awareness and insights experience is needed for this role More ❯
