translate concepts into easily understood diagrams and visuals for both technical and non-technical people alike. AWS cloud products (Lambda functions, Redshift, S3, AmazonMQ, Kinesis, EMR, RDS (Postgres . Apache Airflow for orchestration. DBT for data transformations. Machine Learning for product insights and recommendations. Experience with microservices using technologies like Docker for local development. Apply engineering best practices to More ❯
data engineer or in a similar role Technical expertise with data models Great numerical and analytical skills Experience with event-driven and streaming data architectures (using technologies such as Apache Spark, Flink or similar) Degree in Computer Science, IT, or similar field; a Master's is a plus or four years' equivalent experience Taptap Values Impact first Team next More ❯
Experience with real-time analytics from telemetry and event-based streaming (e.g., Kafka) Experience managing operational data stores with high availability, performance, and scalability Expertise in data lakes, lakehouses, Apache Iceberg, and data mesh architectures Proven ability to build, deliver, and support modern data platforms at scale Strong knowledge of data governance, data quality, and data cataloguing Experience with More ❯
and managing cloud infrastructure as code Proficiency in programming languages such as Python, Spark, SQL Strong experience with SQL databases Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF) Experience with cloud platforms (Azure preferred) and related data services Excellent problem-solving skills and attention to detail Inclusive and curious, continuously seeks to build knowledge and More ❯
and managing machine learning models and infrastructure. Data Management Knowledge: Understanding of data management principles, including experience with databases (SQL and NoSQL) and familiarity with big data frameworks like Apache Spark or Hadoop. Knowledge of data ingestion, storage, and management is essential. Monitoring and Logging Tools : Experience with monitoring and logging tools to track system performance and model effectiveness More ❯
MySQL Exposure to Docker, Kubernetes, AWS, Helm, Terraform, Vault, Grafana, ELK Stack, New Relic Relevant experience in the maintenance of data APIs and data lake architectures, including experience with Apache Iceberg, Trino/Presto, Clickhouse, Snowflake, BigQuery. Master's degree in Computer Science or Engineering-related field Get to know us better YouGov is a global online research company More ❯
MySQL Exposure to Docker, Kubernetes, AWS, Helm, Terraform, Vault, Grafana, ELK Stack, New Relic Relevant experience in the maintenance of data APIs and data lake architectures, including experience with Apache Iceberg, Trino/Presto, Clickhouse, Snowflake, BigQuery. Master's degree in Computer Science or Engineering-related field #LI-PM1 Get to know us better... YouGov is a global online More ❯
might be more valuable than your direct technical contributions on a project. You care about your craft In addition it would be a bonus if you have Worked with Apache Airflow - we use Airflow extensively to orchestrate and schedule all of our data workflows. A good understanding of the quirks of operating Airflow at scale would be helpful. Experience More ❯
management and associated tools such as Git/Bitbucket. Experience in the use of CI/CD tools such as Jenkins or an understanding of their role. Experience with Apache Spark or Hadoop. Experience in building data pipelines. Experience of designing warehouses, ETL pipelines and data modelling. Good knowledge in designing, building, using, and maintaining REST APIs. Good SQL More ❯
Strong interpersonal and communication skills with an ability to lead a team and keep them motivated. Mandatory Skills : Python, Postgres SQL, Azure Databricks, AWS(S3), Git, Azure DevOps CICD, Apache Airflow More ❯
Strong grasp of API design and integration methods (REST, GraphQL, Webhooks) - Knowledge of OAuth2, JWT, and secure authentication protocols - Experience with ETL/ELT pipelines, workflow orchestration tools (e.g., Apache Airflow) - Solid understanding of both SQL and NoSQL databases - Familiarity with AWS and its integration services - Strong problem-solving, communication, and collaboration skills - Agile team experience Nice to Have More ❯
requirements into data solutions Monitor and improve pipeline performance and reliability Maintain documentation of systems, workflows, and configs Tech environment Python, SQL/PLSQL (MS SQL + Oracle), PySpark Apache Airflow (MWAA), AWS Glue, Athena AWS services (CDK, S3, data lake architectures) Git, JIRA You should apply if you have: Strong Python and SQL skills Proven experience designing data More ❯
of real-time and analytical data pipelines, metadata, and cataloguing (e.g., Atlan) Strong communication, stakeholder management, and documentation skills Preferred (but not essential): AWS or Snowflake certifications Knowledge of Apache Airflow, DBT, GitHub Actions Experience with Iceberg tables and data product thinking Why Apply? Work on high-impact, high-scale client projects Join a technically elite team with a More ❯
Microsoft Office XP/2003/2010 (Outlook, Word, Excel, PowerPoint, Access and Visio) Desirable: Good knowledge and experience of working with the following: PHP 5.2 or higher Tomcat Apache Liferay Systems Analysis skills Oracle DBA skills, including database tuning Unix/Linux administrator skills Interface based design and development Experience of documenting the technical design and solution architecture More ❯
Lutterworth, Cotesbach, Leicestershire, United Kingdom
Whistl
Microsoft Office XP/2003/2010 (Outlook, Word, Excel, PowerPoint, Access and Visio) Desirable: Good knowledge and experience of working with the following: PHP 5.2 or higher Tomcat Apache Liferay Systems Analysis skills Oracle DBA skills, including database tuning Unix/Linux administrator skills Interface based design and development Experience of documenting the technical design and solution architecture More ❯
governance SME; support teams with tooling, guidance, and best practices. About You: Strong technical foundation in data governance architecture and tooling. You've worked with tools such as DataHub, Apache Airflow, AWS, dbt, Snowflake, BigQuery , or similar. Hands-on experience building and maintaining centralized data inventories, business glossaries, and data mapping frameworks. Proficient in automating data classification and lineage More ❯
methodologies. Collaborating with stakeholders to define data strategies, implement data governance policies, and ensure data security and compliance. About you: Strong technical proficiency in data engineering technologies, such as Apache Airflow, ClickHouse, ETL tools, and SQL databases. Deep understanding of data modeling, ETL processes, data integration, and data warehousing concepts. Proficiency in programming languages commonly used in data engineering More ❯
to join a fast-growing team that plays an integral part of the revenue producing arm of a company, then our team is for you. Technologies include Scala, Python, Apache Flink, Spark, Databricks, and AWS (ECS, Lambda, DynamoDB, WAF, among others). Experience in these areas is preferred but not required. Qualifications: You collaborate with team members and project More ❯
pipelines. Implement and optimize automation processes using infrastructure-as-code (Terraform) Build and maintain data pipelines using Airflow. Manage our tech stack including Python, Node.js, PostgreSQL, MongoDB, Kafka, and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and resolving production issues. Contribute to the development of new services as More ❯
IV, IFRS 9, CRD4). Strong leadership and stakeholder engagement skills. 15+ years in software development and cloud engineering, ideally in financial services. Experience with big data frameworks (e.g., Apache Beam, Spark) and data governance tools. About working for us Our ambition is to be the leading UK business for diversity, equity and inclusion supporting our customers, colleagues and More ❯
IV, IFRS 9, CRD4). Strong leadership and stakeholder engagement skills. 15+ years in software development and cloud engineering, ideally in financial services. Experience with big data frameworks (e.g., Apache Beam, Spark) and data governance tools. About working for us Our ambition is to be the leading UK business for diversity, equity and inclusion supporting our customers, colleagues and More ❯
networks into production Experience with Docker Experience with NLP and/or computer vision Exposure to cloud technologies (eg. AWS and Azure) Exposure to Big data technologies Exposure to Apache products eg. Hive, Spark, Hadoop, NiFi Programming experience in other languages This is not an exhaustive list, and we are keen to hear from you even if you don More ❯
Bexhill-on-sea, Sussex, United Kingdom Hybrid / WFH Options
Hastings Direct
drive projects from initiation to completion. Keen interest in emerging Machine Learning techniques. Desirable: Experience with Cloud deployments (e.g. Azure/AWS/GCP) and data processing frameworks (e.g. Apache Spark). The interview process: Our interview process involves the below: Recruiter screening call 1st stage interview - Intro with Hiring Leader 2nd interview - case Study round with hiring leaders More ❯
systems such as: Ansible, Terraform, Jenkins, Bamboo, Concourse etc. Monitoring utilising products such as: Prometheus, Grafana, ELK, filebeat etc. Observability - SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Edge technologies e.g. NGINX, HAProxy etc. Excellent knowledge of YAML or similar languages The following Technical Skills & Experience would be desirable for Data Devops More ❯
of automation IT WOULD BE NICE FOR THE SENIOR SOFTWARE ENGINEER TO HAVE. Cloud based experience Microservice architecture or server-less architecture Big Data/Messaging technologies such as Apache Nifi/MiNiFi/Kafka TO BE CONSIDERED. Please either apply by clicking online or emailing me directly to dominic.barbet@searchability.comFor further information please call me on More ❯