systems that incorporate various data backends, query languages and ORM frameworks.Experience designing and building ETL pipelines built around libraries and frameworks like Pandas and Apache Spark.Strong API design skills and a familiarity with building web applications.A proponent of great testing, first-class observability and automating everything.Familiarity with security principles more »
stakeholdersWillingness to pick up and learn new technologies and frameworksNice to have:Knowledge of databases, SQLFamiliarity with Boost ASIOFamiliarity with data serialization formats such Apache Arrow/Parquet, Google Protocol Buffers, FlatbuffersExperience with gRPC, http/REST and Websocket protocolsExperience with Google Cloud/AWS and/or containerization more »
expertise in developing and optimising ETL pipelines.Version Control: Experience with Git for code collaboration and change tracking.Data Pipeline Tools: Proficiency with tools such as Apache Airflow.Cloud Platforms: Familiarity with AWS, Azure, Snowflake, and GCP.Visualisation: Tableau or PowerBIDelivery Tools: Familiarity with agile backlogs, code repositories, automated builds, testing, and releases. more »
South East London, England, United Kingdom Hybrid / WFH Options
Client Server
to production, providing subject matter expertise on the .Net stack and contributing to technical design discussions. You'll use a range of technology including Apache Flink with Java for large scale data processing and will be able to assess and recommend new and emerging technologies, using the best tool more »
South East London, England, United Kingdom Hybrid / WFH Options
McGregor Boyall
Processing: Work with Hadoop, Spark, and other platforms for large-scale data processing.Real-Time Data Streaming: Develop and manage pipelines using CDC, Kafka, and Apache Spark.Database Management: Handle SQL databases like Oracle, MySQL, or PostgreSQL.Data Governance: Ensure data quality, security, and compliance with best practices.Ideal Candidate:Significant experience in more »
making, making trade-offs explicit and understandable to othersREQUIREMENTS7+ years' coding experience, including 3 years in a dedicated ML Engineering role2+ years’ experience with Apache SparkExperience working with GB+ scale dataExperience with deployed ML servicesExperience deploying multiple ML projects across different environmentsProductionisation experience in at least one cloud infrastructure more »
Basingstoke, England, United Kingdom Hybrid / WFH Options
Intec Select
cross-functionally across the business to understand the requirements of the products Designing and implementing performance related data ingestion pipelines from multiple sources using Apache Spark Integrating end-to-end data pipelines ensuring a high level of quality is maintained Working with an Agile delivery/DevOps methodology to more »
. • Troubleshooting networks issues (tcpdump/Wireshark). • Scripting capabilities (SH/Bash/Python/Perl). • Configuration of common services (DNS/Apache/NGINX/Postfix/Squid/SSH/iptables). • Understanding of clustering services, enabling High Availability failover. • Experience with enterprise hardware and more »
South East London, England, United Kingdom Hybrid / WFH Options
Hunter Bond
s Degree in Computer Science, Engineering (or other related STEM subject)5+ years experience in data engineering2+ years in a leadership role.Experience working with Apache Spark, Azure Data Factory and other data pipelines tools.Strong programming skills.Impeccable communication skills.Precise attention to detail.Pioneering attitude.If you are a Lead Data Engineer and more »
South East London, England, United Kingdom Hybrid / WFH Options
Modo Energy
teams to support the orchestration of our ETL pipelines using Airflow and manage our tech stack including Python, Next.js, Airflow, PostgreSQL MongoDB, Kafka and Apache Iceberg.Optimize infrastructure costs and develop strategies for efficient resource utilization.Provide critical support by monitoring services and quickly resolving production issues.Contribute to the development and more »
Rickmansworth, Hertfordshire, South East, United Kingdom
Mobilize Financial Services
build, operate and manage a complex production environment. Familiarity with RedHat based Linux versions Experience of Web Application servers architectures, security, protocols and technologies (Apache Web Server, HAProxy, Tomcat) configuration and optimization Understanding of DR/BCP business processes Comfortable liaising with business users as well as technical teams more »
within the EU Fusion programme and connections to international HPC communities, showcasing contributions made to the field. - Experience in workflow management systems such as Apache Airflow. - Familiarity with Research Data Management methodologies, modern database technologies including SQL, NoSQL and Graph Databases, and parallel file access technologies such as MPI more »
Hertfordshire, South East, United Kingdom Hybrid / WFH Options
Hays
Directory Novell Netware systems, Zenworks, e-Directory Linux (SUSE) systems SUN Solaris Unix systems, NIS+ Lotus Domino MS Exchange and mail services Web servers, Apache, Tomcat Experience across the Oracle suite Data warehouse infrastructure, data archive solutions Storage area networks, volume management Anti-virus software Technical architectures and development more »
complete before business approval Oversee release management Maintain technology & security standards Stakeholder management Required Skills: 5+ years of development experience Familiarity with TypeScript, React, Apache NiFi, GraphQL, or Postgres is desirable. Proficiency in Atlassian's Jira and Confluence is desirable. Strong communication and time management skills Due to the more »
like Lambdas, step functions and ECS services.Strong understanding of AWS ecosystems like Lambdas, step functions and ECS services.Experience with data stack technologies, such as Apache Iceberg & Spark. Exposure to Apache Airflow, Prefect, Dagster, DBT.Expertise in data analysis with exposure to data services (such as Glue, Lake Formation, EMR more »
South East London, England, United Kingdom Hybrid / WFH Options
Maclean Moore
given requirement.Ensure to develop unit test cases.Help in backlog grooming.Key skills: Extensive experience in developing Bigdata pipelines in cloud using Bigdata technologies such as Apache SparkExpertise in performing complex data transformation using Spark SQL queriesExperience in orchestrating data pipelines using Apache AirflowProficiency in Git based version control toolsProficiency more »
South East London, England, United Kingdom Hybrid / WFH Options
Client Server
to production, providing subject matter expertise on the .Net stack and contributing to technical design discussions. You'll use a range of technology including Apache Flink with Java for large scale data processing and will be able to assess and recommend new and emerging technologies, using the best tool …/CD and Infrastructure as Code (Terraform)You're familiar with other languages such as Java and are open to learning new things e.g. Apache FlinkYou've worked on systems that require high throughput and low latencyYou enjoy problem solving and have great communication and collaboration skillsWhat's in more »
in cloud development (ideally AWS)Knowledge and ideally hands-on experience with data streaming, event-based architectures and KafkaStrong communication and interpersonal skillsExperience with Apache Spark or Apache Flink would be ideal, but not essentialPlease note, this role is unable to provide sponsorship.If this role sounds of interest more »
and maintaining ETL pipelinesKnow your way around Unix based operating systemExperience working with any major cloud provider (AWS, GCP, Azure)Fluency in EnglishExperience using Apache AirflowExperience using DockerExperience using Apache SparkBenefits:Salary £40-50K per annum dependant on skills and experience25 Days annual leave + Public HolidaysEarly more »
Job DescriptionLead Data Engineer (Director) - Individual contributor - Azure, Data Factory, Databricks, Apache Spark - London Based I am hiring for a Lead Data Engineer for a crucial role within one of my Investment Bank clients in London. This role is at Director level as they require a very senior candidate … Responsibilities: Leading data engineering practicesSupport current applications Introduce AI practices to the team/project Communicate key successes with stakeholders Key Skills: Azure Databricks Apache Spark Datascience, AI, ML Certifications or continued upskilling/contribution to blog posts within Data & AI beneficial but not essential. This is a full … UK without sponsorship, if you are interested please apply or email me directly - aaron.dhammi@nicollcurtin.comLead Data Engineer (Director) - Individual contributor - Azure, Data Factory, Databricks, Apache Spark - London Based more »
Reading, England, United Kingdom Hybrid / WFH Options
CACTUS IT SOLUTIONS
Exhibit proficiency in Java concurrency for parallel test execution, providing valuable expertise to enhance testing efficiency. • Java Libraries: Demonstrate advanced usage of libraries like Apache Commons, Guava, etc., showcasing the ability to select and implement libraries for optimized efficiency. • Automation Frameworks: Work with Selenium WebDriver, JUnit, TestNG, Cucumber BDD … advanced frameworks, contributing to their enhancement and customization to meet project requirements. • REST API Testing: Utilize advanced API testing tools like Rest Assured and ApacheHTTP Client, showcasing in-depth knowledge of API testing. • Framework Implementation: Contribute to the enhancement and customization of automation frameworks, ensuring they align with more »
