76 to 100 of 115 Permanent Apache Jobs

optimizing data pipelines and cloud resources on Google Cloud Platform (BigQuery) for cost-effective, high-performance processing; • Engineering scalable ingestion and transformation processes using Apache Spark, Spark SQL, and Python, orchestrated with Apache Airflow; • Developing advanced SQL queries and performing query optimization on large-scale data platforms such ...

infrastructure. Analyses and reports on test activities, results, issues and risks. Production data analysis to identify root cause of production environment issues. Using Apache Airflow for task orchestration and reviewing output logs. Basic understanding of EMR and reviewing output logs. Basic understanding of data processing using Apache Spark. ...

optimise scalable data pipelines within a Databricks Lakehouse environment. You'll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams …/7 Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER - ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion ...

Code tools such as AWS CDK, CloudFormation, Terraform, or AWS CLI Experience with real-time stream processing and Complex Event Processing (CEP) using Apache Flink or similar technologies is a plus Experience with automated testing frameworks, specifically Playwright for end-to-end and UI testing Strong focus on scalability … monitoring and analytics tools such as Grafana, InfluxDB, or similar Experience with cloud data stores and query engines such as Amazon Redshift, Athena, DynamoDB, Apache Iceberg and S3-based data lakes a plus Understanding of data modeling and transformation using AWS services such as Glue, Lambda, Redshift, Athena ...

Senior Software Engineer Hungerford, United Kingdom Tech stack JavaScript Web API ASP.NET web C TypeScript React Qualifications Our ideal candidate will have hands‐on experience in web and application development using technologies such as ASP.NET ...

Software Developer Permanent | Hybrid (willing to travel to Newcastle) | Python | Apache | Data | SC cleared Hybrid work arrangement. Office attendance is up to 60%. Location is flexible: London, Leeds, Newcastle, Sheffield, Blackpool, Manchester and Birmingham. You must be willing to travel to Newcastle when required. The Role ...

Description Software Developer Permanent | Hybrid (willing to travel to Newcastle) | Python | Apache | Data | SC cleared Hybrid work arrangement. Office attendance is up to 60%. Location is flexible: London, Leeds, Newcastle, Sheffield, Blackpool, Manchester and Birmingham. You must be willing to travel to Newcastle when required. The Role ...

Kingdom. The role demands strong expertise in Solution Architecture, Cloud-Native Architecture Design, and hands-on experience with Snowflake and Data Lakehouse architectures, including Apache Iceberg. Responsibilities involve creating Architecture Design Documents, managing S3-based Data Lakes, and ensuring platform performance tuning and observability. Ideal candidates should have substantial ...

Apache Associates is looking for a Software Engineering Manager to lead and scale a high-performing engineering team responsible for AI-enabled platforms for construction projects. This role involves mentoring engineers, establishing best practices, and driving AI tool adoption. Candidates should have strong management experience, a background in full ...

Description Software Developer Permanent | Hybrid (willing to travel to Newcastle) | Python | Apache | Data | SC cleared Hybrid work arrangement. Office attendance is up to 60%. Location is flexible: London, Leeds, Newcastle, Sheffield, Blackpool, Manchester and Birmingham. xkybehq You must be willing to travel to Newcastle when required. The Role ...

Directory Services (Active Directory, LDAP, etc.) Windows and/or *nix environments Scripting (e.g. PowerShell, Python, Bash) Databases (Oracle, MSSQL, MySQL, PostgreSQL) Web technologies (Apache, IIS, etc.) DevOps and software development practices Application integration Reporting and data analysis ...

kernels. Experience with RLHF pipelines (PPO, DPO, ORPO). Experience training or deploying multimodal or diffusion models. Experience with large‐scale data processing (Apache Arrow, Spark, Ray). How We Work The best products today in the world were built by small, world class teams. We are a high ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

repair and reassembly of systems already in use. In doing so, they work on the most famous aircraft and helicopters; think of the Apache combat helicopter, the F-35 Lightning II (JSF), the Bombardier Dash 8 Q400 and the multipurpose NH90 maritime combat helicopter. Over de functie ...

Software Developer Permanent | Hybrid (willing to travel to Newcastle) | Python | Apache | Data | SC cleared Hybrid work arrangement. Read the overview of this opportunity to understand what skills, including and relevant soft skills and software package proficiencies, are required. Office attendance is up to 60%. Location is flexible: London … must be willing to travel to Newcastle when required. The Role: We are looking for Software Developers with strong Python and data processing with Apache Spark responsible to lead the design, development, and operation of data-driven applications and pipelines in a collaborative Dev and DevOps environment. The role ...

code. Perfect if you want scope for growth without going “post-technical.” What you’ll do Design and build modern data platforms using Databricks, Apache Spark, Snowflake, and cloud-native services (AWS, Azure, or GCP). Develop robust pipelines for real-time and batch data ingestion from diverse … Lead Data Engineer in complex enterprise environments. Can demonstrate strong coding skills in Python (Scala or functional languages a plus). Expertise with Databricks, Apache Spark, and Snowflake is a must (HDFS/HBase also useful). Has experience integrating large, messy datasets into reliable, scalable data products. Strong ...

administration in a physical and virtual enterprise environment. Strong knowledge of Linux scripting languages and installing, configuring, and maintaining OS services such as Bind, Apache, MySQL and other databases, Nginx, etc. A solid understanding configuration management tools such as Puppet is a plus. Experience with Pure SAN or other ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

InnovationResearch, evaluate, and recommend new technologies and tools that enhance Arriva's data and BI capabilitiesStay current with emerging trends including open table formats (Apache Iceberg, Delta Lake), data observability, real-time/streaming architectures, and cloud-native solutionsConduct proofs-of-concept and technical assessments to validate new technologies … understanding of modern data engineering patterns including ETL/ELT, data lake architectures, and lakehouse approachesUnderstanding of emerging technologies such as open table formats (Apache Iceberg, Delta Lake), data observability tools, and real-time/streaming architecturesExperience with data governance frameworks, data quality tools, metadata management, and ensuring compliance ...

evaluate, and recommend new technologies and tools that enhance Arriva's data and BI capabilities* Stay current with emerging trends including open table formats (Apache Iceberg, Delta Lake), data observability, real-time/streaming architectures, and cloud-native solutions* Conduct proofs-of-concept and technical assessments to validate … modern data engineering patterns including ETL/ELT, data lake architectures, and lakehouse approaches* Understanding of emerging technologies such as open table formats (Apache Iceberg, Delta Lake), data observability tools, and real-time/streaming architectures* Experience with data governance frameworks, data quality tools, metadata management, and ensuring compliance ...

evaluate, and recommend new technologies and tools that enhance Arriva's data and BI capabilities* Stay current with emerging trends including open table formats (Apache Iceberg, Delta Lake), data observability, real-time/streaming architectures, and cloud-native solutions* Conduct proofs-of-concept and technical assessments to validate … modern data engineering patterns including ETL/ELT, data lake architectures, and lakehouse approaches* Understanding of emerging technologies such as open table formats (Apache Iceberg, Delta Lake), data observability tools, and real-time/streaming architectures* Experience with data governance frameworks, data quality tools, metadata management, and ensuring compliance ...

code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark – especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects ...

teams and setting best practice standards Designing data models, ETL pipelines, warehousing solutions and distributed systems Building and supporting real-time streaming platforms using Apache Flink and related technologies Ensuring compliance with industry standards and government regulations Tracking delivery against agreed designs and managing design deviations Supporting pre-sales … data architecture on complex, secure programmes Strong hands-on AWS experience (EC2, S3, RDS, Lambda, Redshift); Azure or GCP a bonus Proven experience using Apache Flink for real-time, high-throughput data processing Experience leading, mentoring and developing technical teams Confident engaging with senior technical and non-technical stakeholders ...

partnership with Inspire People, is hiring a Lead Data Engineer with strong SQL and Python expertise, experience across modern cloud data platforms including Databricks, Apache Spark and lakehouse architectures, and a proven ability to design, build and lead delivery of reliable data products. This is a permanent role based … Strong proficiency in SQL and Python - Experience with modern cloud data platforms and lakehouse architectures - Hands-on experience with big data technologies such as Apache Spark or Databricks - Strong knowledge of data warehousing concepts - Experience with build, test and deployment automation - Proven leadership capability leading or mentoring data engineering ...