teams to support the orchestration of our ETL pipelines using Airflow and manage our tech stack including Python, Next.js, Airflow, PostgreSQL MongoDB, Kafka and Apache Iceberg. Optimize infrastructure costs and develop strategies for efficient resource utilization. Provide critical support by monitoring services and quickly resolving production issues. Contribute to more »
Glasgow, Scotland, United Kingdom Hybrid / WFH Options
MBN Solutions
technologies, techniques, and architectures to build and maintain robust data pipelines. Technological Proficiency: Experience with technologies such as Azure Data Factory, Pentaho Data Integrator, Apache Hop, etc. ETL/ELT Practices: Strong understanding of modern ETL/ELT practices, frameworks, tooling, and execution environments. Data Delivery: Knowledge of data more »
environment. Familiarity with cloud-native computing concepts and experience with hybrid or private cloud platforms is advantageous. Technical expertise in a Microsoft, Redhat, and Apache data and software engineering environment. A team-oriented individual with a dedication to engineering excellence and the ability to lead and inspire a team more »
City of London, England, United Kingdom Hybrid / WFH Options
Penguin Recruitment
market leading software required to communicate and exchange technical information. Demonstrable academic qualifications appropriate to the role. Be a NABERS Assessor Have experience with Apache HVAC (preferred) Have a genuine interest and enthusiasm in the built environment and progress toward Net Zero Carbon. Demonstrable familiarity with key industry agendas more »
Elasticsearch, Bigquery, PostgresQL FullCircl 3 Lead_Data_Engineer 04.24 · Kubernetes, Docker, Airflow KEY RESPONSIBILITIES · Designing and implementing scalable data pipelines using tools such as Apache Spark, Google PubSub etc. · Optimizing data storage and retrieval systems for maximum performance using both relational and NoSQL databases. · Continuously monitoring and improving the … Data Infrastructure projects, as well as designing and building data intensive applications and services. · Experience with data processing and distributed computing frameworks such as Apache Spark · Expert knowledge in one or more of the following languages - Python, Scala, Java, Kotlin · Deep knowledge of data modelling, data access, and data more »
London, England, United Kingdom Hybrid / WFH Options
Pioneer Search
Senior Scala Developer - Apache Spark - Urgent Requirement Contract Length: 6 Months IR35 status: Inside Location: London - Hybrid working A Senior Scala Developer with experience in Apache Spark is needed for a British consultancy organisation. You will be an integral member of the team providing technical expertise to the more »
Experienced creating data pipelines on a cloud (preferably AWS) environment CI/CD experience Containerization experience (Docker, Kubernetes, etc.) Experience with SQS/SNS, Apache Kafka, RabbitMQ Other interesting/bonus skills – Airflow, Trino, Apache Iceburg, Postgres, MongoDB You *must* be eligible to work in your chosen country more »
Durham, County Durham, North East, United Kingdom Hybrid / WFH Options
Reed Technology
a data science team and managing complex projects. Expertise in machine learning, statistics, data management, and relevant technologies (e.g., Python, R, SQL, AWS SageMaker, Apache Airflow, Dbt, AWS Kinesis). Strong communication skills with the ability to explain complex data concepts to a non-technical audience. Knowledge of industry more »
Bristol, England, United Kingdom Hybrid / WFH Options
Made Tech
and able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create more »
Manchester, England, United Kingdom Hybrid / WFH Options
Made Tech
and able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create more »
Drupal Magento BigCommerce Laravel Proficient in setting up development and staging environments. Proficient in using and altering MySQL databases. Familiarity with server structures, specifically Apache and Nginx. Familiarity with domain management and DNS records. Familiarity with Agile and Waterfall work environments. Nice to haves: Familiarity with digital marketing/ more »
Leeds, England, United Kingdom Hybrid / WFH Options
Candour Solutions
employees Driving change and handling difficult situations Dealing with change on a daily basis Ability to demonstrate the value of changes introduced Web servers - Apache, nginx, IIS, Drupal Databases - Oracle, MYSQL, MSSQL Traffic Management - F5, Haproxy, Keepalived CDNS - CDNetworks, Akamai, Incapsula Scripting - Bash, Bat, Python, Perl Programming Languages - PHP more »
build their AI practice and a team around you. Required Skills: Building cloud and native machine learning architecture with: LLamaIndex, HuggingFace, SentenceTransformers, PyTorch, Python, Apache Spark. Experience with practical application of AI and scaling AI with these tools Experience in Health Care is essential We would love to share more »
Cambridge, Cambridgeshire, United Kingdom Hybrid / WFH Options
ARM
the internal needs of Arm with those of the community. Experience building or using industry standard programmer's interfaces. Expert understanding of Tensorflow, PyTorch, Apache TVM, or comparable inference frameworks. Expert understanding of AI training, quantization and visualization tools. Proven experiences with LINUX. Comfortable presenting technical topics to both more »
ETL/ELT tools.Experience with NoSQL type environments, Data Lakes, Lake-Houses (Cassandra, MongoDB or Neptune).Experience with distributed storage, processing engines such as Apache Hadoop and Apache Spark.Experience with message brokering/stream processing services such as Apache Kafka, Confluent, Azure Stream Analytics.Experience in Test Driven more »
Belfast, Northern Ireland, United Kingdom Hybrid / WFH Options
Version 1
Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics Direct experience in building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks). Experience building data warehouse solutions using ETL/ELT tools such as SQL Server Integration Services (SSIS), Oracle Data Integrator … ODI), Talend, and Wherescape Red. Experience with Azure Event Hub, IOT Hub, Apache Kafka, Nifi for use with streaming data/event-based data Experience with other Open Source big data products eg Hadoop (incl. Hive, Pig, Impala) Experience with Open Source non-relational/NoSQL data repositories (incl. more »
Reading, England, United Kingdom Hybrid / WFH Options
CACTUS IT SOLUTIONS
Exhibit proficiency in Java concurrency for parallel test execution, providing valuable expertise to enhance testing efficiency. • Java Libraries: Demonstrate advanced usage of libraries like Apache Commons, Guava, etc., showcasing the ability to select and implement libraries for optimized efficiency. • Automation Frameworks: Work with Selenium WebDriver, JUnit, TestNG, Cucumber BDD … advanced frameworks, contributing to their enhancement and customization to meet project requirements. • REST API Testing: Utilize advanced API testing tools like Rest Assured and ApacheHTTP Client, showcasing in-depth knowledge of API testing. • Framework Implementation: Contribute to the enhancement and customization of automation frameworks, ensuring they align with more »
Manchester, North West, United Kingdom Hybrid / WFH Options
Searchability NS&D Ltd
existing architectural components including Data Ingest, Data Stores and REST APIs. THE DEVOPS ENGINEER SHOULD HAVE…. You must have an active eDV Clearance. Apache Nifi Flink Java Ansible Docker Kubernetes ELK stack Linux Sys Admin for deployed Clusters (10's of servers) Jenkins Pipeline development Integration/debugging … DEVOPS ENGINEERS - MANCHESTER KEY SKILLS: SOFTWARE DEVELOPER/SOFTWARE ENGINEER/SENIOR SOFTWARE DEVELOPER/SENIOR SOFTWARE ENGINEER/DEVOPS ENGINEER/DEVOPS/APACHE NIFI/FLINK/JAVA/ANSIBLE/DOCKER/KUBERNETES/ELK STACK/TERRAFORM/LINUX/GIT more »
Knutsford, England, United Kingdom Hybrid / WFH Options
Capgemini
Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of data modelling Principles and best practices. more »
Northamptonshire, England, United Kingdom Hybrid / WFH Options
Capgemini
Glue, Starburst, Snowflake, Redshift, Starburst, Hybrid(on-prem to cloud) for building data pipelines and ETL processes. Experience with data pipeline orchestration tools like Apache Airflow . Knowledge of SQL and ability to write complex queries for data transformation and analysis. Understanding of data modelling Principles and best practices. more »
