as well as distributed computing frameworks. Familiarity with data warehousing concepts and dimensional modelling. Strong knowledge of ETL principles and experience with tools like Apache Spark or AWS Glue. Experience with real-time data processing frameworks such as Apache Kafka or AWS Kinesis. Experience with version control systems more »
bristol, south west england, United Kingdom Hybrid / WFH Options
J&C Associates Ltd
so any knowledge of cross domain solutions or air gapped is a plus AWS as initial hosting provider * Containerised apps using Docker and Kubernetes * Apache Jena * Elastic * PostGIS * Kafka * Apache NiFi * AWS Cognito * HTTP REST, GraphQL, SPARQL interfaces * Web apps based on HTML/CSS/Javascript frameworks more »
Experienced creating data pipelines on a cloud (preferably AWS) environment CI/CD experience Containerization experience (Docker, Kubernetes, etc.) Experience with SQS/SNS, Apache Kafka, RabbitMQ Other interesting/bonus skills – Airflow, Trino, Apache Iceburg, Postgres, MongoDB You *must* be eligible to work in your chosen country more »
managers, to understand data requirements and deliver high-quality solutions as well as architecting data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies. Other core duties include optimizing data pipeline performance, ensuring data accuracy, reliability, and timely delivery. Requirements … Services Certifications in relevant technologies, such as Azure Data Engineer or Databricks Certified Developer Experience with real-time data processing and streaming technologies like Apache Kafka or Azure Event Hubs Knowledge of data visualization tools, such as Power BI or Tableau Contributions to open-source projects or active participation more »
data components such as Azure Data Factory, Azure SQL DB, Azure Data Lake, etc. Strong Python and SQL skills for data manipulation Experience with Apache Spark and/or Databricks. Familiarity with BI visualization tools like Power BI Experience in managing end-to-end analytics pipelines (batch and streaming … such as Azure Data Engineer Associate are desirable. Knowledge of data ingestion methods for real-time and batch processing Proficiency in PySpark and debugging Apache Spark workloads. What’s in it for you? Annual bonus scheme – up to 10% Excellent pension scheme Flexible working Enhanced family friendly policies Ongoing more »
Lead Data Engineer (Director) - Individual contributor - Azure, Data Factory, Databricks, Apache Spark - London Based I am hiring for a Lead Data Engineer for a crucial role within one of my Investment Bank clients in London. This role is at Director level as they require a very senior candidate to … Leading data engineering practices Support current applications Introduce AI practices to the team/project Communicate key successes with stakeholders Key Skills: Azure Databricks Apache Spark Datascience, AI, ML Certifications or continued upskilling/contribution to blog posts within Data & AI beneficial but not essential. This is a full … without sponsorship, if you are interested please apply or email me directly - aaron.dhammi@nicollcurtin.com Lead Data Engineer (Director) - Individual contributor - Azure, Data Factory, Databricks, Apache Spark - London Based more »
reading, south east england, United Kingdom Hybrid / WFH Options
CACTUS IT SOLUTIONS
Exhibit proficiency in Java concurrency for parallel test execution, providing valuable expertise to enhance testing efficiency. • Java Libraries: Demonstrate advanced usage of libraries like Apache Commons, Guava, etc., showcasing the ability to select and implement libraries for optimized efficiency. • Automation Frameworks: Work with Selenium WebDriver, JUnit, TestNG, Cucumber BDD … advanced frameworks, contributing to their enhancement and customization to meet project requirements. • REST API Testing: Utilize advanced API testing tools like Rest Assured and ApacheHTTP Client, showcasing in-depth knowledge of API testing. • Framework Implementation: Contribute to the enhancement and customization of automation frameworks, ensuring they align with more »
leeds, west yorkshire, yorkshire and the humber, United Kingdom
Infosys
API/Microservices Containerization (Kubernetes) and Workflow orchestration (Temporal) Strong skills in distributed architecture – local and cross site resiliency and failover Achieving scalability on Apache Nifi based data pipelines to run several concurrent data movement feeds Being able to visualize and integrate the platform and application architecture Own and … product (Kinisi) Strongly preferred skills: OpenSearch, Prometheus, Grafana, log management and observability tools/technologies, Java - API/Microservices, UI/UX, Kubernetes, Temporal, Apache Nifi Soft Skills: Strong skills in communicating architecture, pros and cons of design/architecture and being able to lead customer a solution proactively. more »
required) Experience with distributed message brokers using Kafka (required) Experience with high speed distributed computing frameworks such as AWS EMR, Hadoop, HDFS, S3, MapReduce, Apache Spark, Apache Hive, Kafka Streams, Apache Flink etc. (required) Working knowledge of DevOps tools. Eg Terraform, Ansible, Jenkins, Kubernetes, Helm and CI more »
required) Experience with distributed message brokers using Kafka (required) Experience with high speed distributed computing frameworks such as AWS EMR, Hadoop, HDFS, S3, MapReduce, Apache Spark, Apache Hive, Kafka Streams, Apache Flink etc. (required) Working knowledge of DevOps tools. Eg Terraform, Ansible, Jenkins, Kubernetes, Helm and CI more »
to production, providing subject matter expertise on the .Net stack and contributing to technical design discussions. You'll use a range of technology including Apache Flink with Java for large scale data processing and will be able to assess and recommend new and emerging technologies, using the best tool more »
Install and configure software baselines, Commercial Off-The-Shelf (COTS) products, and application programming interfaces Deploy and maintain internal and external web servers (currently Apache web server) and ensure website availability Maintain, enhance, and modernize the existing CEGS architecture Design, architect, and implement new processes and deployment strategies Troubleshoot … Nimble storage, Dell workstation, Cisco switches, etc.) Database installation and administration (PostgreSQL or equivalent) Windows Systems Administration ActiveMQ installation, configuration, and maintenance Apachehttpd configuration, maintenance, and deployment Familiarity with Amazon Web Services (AWS) Original Posting Date: 2024-05-21 While subject to change based on business needs, Leidos more »
Install and configure software baselines, Commercial Off-The-Shelf (COTS) products, and application programming interfaces Deploy and maintain internal and external web servers (currently Apache web server) and ensure website availability Maintain, enhance, and modernize the existing CEGS architecture Design, architect, and implement new processes and deployment strategies Troubleshoot … Nimble storage, Dell workstation, Cisco switches, etc.) Database installation and administration (PostgreSQL or equivalent) Windows Systems Administration ActiveMQ installation, configuration, and maintenance Apachehttpd configuration, maintenance, and deployment Familiarity with Amazon Web Services (AWS) Original Posting Date: 2024-05-21 While subject to change based on business needs, Leidos more »
Install and configure software baselines, Commercial Off-The-Shelf (COTS) products, and application programming interfaces Deploy and maintain internal and external web servers (currently Apache web server) and ensure website availability Maintain, enhance, and modernize the existing CEGS architecture Design, architect, and implement new processes and deployment strategies Troubleshoot … Nimble storage, Dell workstation, Cisco switches, etc.) Database installation and administration (PostgreSQL or equivalent) Windows Systems Administration ActiveMQ installation, configuration, and maintenance Apachehttpd configuration, maintenance, and deployment Familiarity with Amazon Web Services (AWS) Original Posting Date: 2024-05-21 While subject to change based on business needs, Leidos more »
Install and configure software baselines, Commercial Off-The-Shelf (COTS) products, and application programming interfaces Deploy and maintain internal and external web servers (currently Apache web server) and ensure website availability Maintain, enhance, and modernize the existing CEGS architecture Design, architect, and implement new processes and deployment strategies Troubleshoot … Nimble storage, Dell workstation, Cisco switches, etc.) Database installation and administration (PostgreSQL or equivalent) Windows Systems Administration ActiveMQ installation, configuration, and maintenance Apachehttpd configuration, maintenance, and deployment Familiarity with Amazon Web Services (AWS) Original Posting Date: 2024-05-21 While subject to change based on business needs, Leidos more »
Nottingham, Nottinghamshire, East Midlands, United Kingdom
Microlise
data practices Possess strong knowledge of data tools, data management tools, and various data and information technologies. E.g. DAMA DMBOK, Microsoft SQL Server, Couchbase, Apache Druid, Spark, Kafka, Airflow, etc In-depth understanding of modern data principles, methodologies, and tools Excellent communication and collaboration skills, with the ability to … native computing concepts and experience working with hybrid or private cloud platforms is a plus. Demonstrable technical experience working with a Microsoft, Redhat, and Apache data and software engineering environment. A team-oriented individual with a passion for engineered excellence and the ability to lead and motivate a team more »
to develop unit test cases. Help in backlog grooming. Key skills: Extensive experience in developing Bigdata pipelines in cloud using Bigdata technologies such as Apache Spark Expertise in performing complex data transformation using Spark SQL queries Experience in orchestrating data pipelines using Apache Airflow Proficiency in Git based more »
engineers of varying levels of experience. Flexibility and willingness to adapt to new software and techniques. Nice to Have Experience working with projects in Apache Spark, Databricks of similar. Expert cloud platform knowledge, e.g. Azure What will be your key responsibilities? A technical expert and leader on the Petcare more »
Belfast, Northern Ireland, United Kingdom Hybrid / WFH Options
Version 1
Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics Direct experience in building data pipelines using Azure Data Factory and Apache Spark (preferably Databricks). Experience building data warehouse solutions using ETL/ELT tools such as SQL Server Integration Services (SSIS), Oracle Data Integrator … ODI), Talend, and Wherescape Red. Experience with Azure Event Hub, IOT Hub, Apache Kafka, Nifi for use with streaming data/event-based data Experience with other Open Source big data products eg Hadoop (incl. Hive, Pig, Impala) Experience with Open Source non-relational/NoSQL data repositories (incl. more »
London, Liverpool, Merseyside, United Kingdom Hybrid / WFH Options
Opus Recruitment Solutions
Glue). Hands-on experience with Databricks for data processing and analytics. Proficient in Python programming for data manipulation and automation. Solid understanding of Apache Spark for big data processing. Strong SQL skills for data querying, transformation, and analysis. Excellent problem-solving abilities and attention to detail. Ability to more »
Manchester, North West, United Kingdom Hybrid / WFH Options
Adria Solutions
data tasks. Knowledge of CI/CD approaches for Data Platforms using Bitbucket and Bitbucket Pipelines. Knowledge of AWS data lake approaches using Athena & Apache Iceberg tables. Exposure to visualisation development using Power BI. Knowledge of MS SQL Server, SSIS, Visual Studio, and SSDT projects. Experience in a relevant … maintaining data pipelines on Bitbucket using Bitbucket Pipelines. You will also use your knowledge of AWS data lake approaches to optimize performance by implementing Apache Iceberg tables. This is an exciting chance to gain experience in visualisation development using Power BI and working with MS SQL Server, SSIS, Visual more »
4.Monitoring and Logging: Implement and maintain monitoring, logging, and alerting solutions. Key technologies: AWS, VPN, VPC Peering, EC2, S3, Lambda, Aurora, Docker/Kubernetes. Apache Airflow, AWS networking concepts such as VPN, VPC peering, subnets, security groups, NAT gateways. AWS CloudWatch or equivalent. Kafka or similar data streaming components. more »
experience developing APIs and microservices. Familiarity with data engineering concepts and tools such as data pipelines, ETL processes, SQL, and big data technologies (e.g., Apache Spark, Hadoop). Strong problem-solving skills, analytical thinking, and attention to detail. Excellent communication and collaboration skills, with the ability to work effectively more »
work is largely down to you. It can be entirely Back End. Otherwise, the stack includes Redux Saga, Ag-Grid, Node, TypeScript, gRPC, protobuf, Apache Ignite, Apache Airflow and AWS. As the application suite grows and advances in complexity, there is a decent amount of interaction with the more »
