data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter ,LinkedIn and Facebook . Benefits At Databricks, we strive to provide comprehensive More ❯
Sheffield, Yorkshire, United Kingdom Hybrid / WFH Options
Reach Studios Limited
with web applications utilising the power of Laravel for back-end processes. Front-ends are built in Alpine.js/React/Next.js, rendered by Apache or NGINX servers. Both REST and GraphQL APIs are utilised to integrate and extend the services we provide to our array of clients. What More ❯
data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our More ❯
or product feature use cases. Experience in building and deploying live software services in production. Exposure to some of the following technologies (or equivalent): Apache Spark, AWS Redshift, AWS S3, Cassandra (and other NoSQL systems), AWS Athena, Apache Kafka, Apache Flink, AWS and service oriented architecture. What More ❯
processing large-scale data. Experience with ETL processes for data ingestion and processing. Proficiency in Python and SQL. Experience with big data technologies like Apache Hadoop and Apache Spark. Familiarity with real-time data processing frameworks such as Apache Kafka or Flink. MLOps & Deployment: Experience deploying and More ❯
design, implementation, testing, and support of next-generation features related to Dremio's Query Planner and Reflections technologies Work with open source projects like Apache Calcite and Apache Iceberg Use modular design patterns to deliver an architecture that's elegant, simple, extensible and maintainable Solve complex technical problems … distributed query engines. Hands on experience in query processing or optimization, distributed systems, concurrency control, data replication, code generation, networking, storage systems, heap management, Apache Arrow, SQL Operators, caching techniques, and disk spilling Hands on experience with multi-threaded and asynchronous programming models More ❯
the big 3 cloud ML stacks (AWS, Azure, GCP). Hands-on experience with open-source ETL, and data pipeline orchestration tools such as Apache Airflow and Nifi. Experience with large scale/Big Data technologies, such as Hadoop, Spark, Hive, Impala, PrestoDb, Kafka. Experience with workflow orchestration tools … like Apache Airflow. Experience with containerisation using Docker and deployment on Kubernetes. Experience with NoSQL and graph databases. Unix server administration and shell scripting experience. Experience in building scalable data pipelines for highly unstructured data. Experience in building DWH and data lakes architectures. Experience in working in cross-functional More ❯
Birmingham, Staffordshire, United Kingdom Hybrid / WFH Options
Yelp USA
to the experimentation and development of new ad products at Yelp. Design, build, and maintain efficient data pipelines using large-scale processing tools like Apache Spark to transform ad-related data. Manage high-volume, real-time data streams using Apache Kafka and process them with frameworks like Apache Flink. Estimate timelines for projects, feature enhancements, and bug fixes. Work with large-scale data storage solutions, including Apache Cassandra and various data lake systems. Collaborate with cross-functional teams, including engineers, product managers and data scientists, to understand business requirements and translate them into effective system designs. … a proactive approach to identifying opportunities and recommending scalable, creative solutions. Exposure to some of the following technologies: Python, AWS Redshift, AWS Athena/Apache Presto, Big Data technologies (e.g S3, Hadoop, Hive, Spark, Flink, Kafka etc), NoSQL systems like Cassandra, DBT is nice to have. What you'll More ❯
or product feature use cases. Experience in building and deploying live software services in production. Exposure to some of the following technologies (or equivalent): Apache Spark, AWS Redshift, AWS S3, Cassandra (and other NoSQL systems), AWS Athena, Apache Kafka, Apache Flink, AWS, and service-oriented architecture. What More ❯
Location: Remote-first (UK-based) 💰 Rate: Up to £550 p/d 📆 Contract: 6 - 12 months (Outside IR35) 🛠 Tech Stack: Python, FastAPI, GCP, BigQuery, Apache Spark, Apache Beam, Google Cloud Dataflow We're working with a forward-thinking consultancy that helps top companies build and scale high-performance … You’ll Be Doing: 🔹 Building data pipelines and ETL workflows that process huge datasets 🔹 Designing, optimizing, and maintaining high-throughput reporting solutions 🔹 Working with Apache Spark for large-scale data processing 🔹 Using Apache Beam and Google Cloud Dataflow to manage complex data workflows 🔹 Developing and improving backend APIs … writing clean, efficient, and scalable code ✔ Experience with BigQuery, PostgreSQL, and Elasticsearch ✔ Hands-on experience with Google Cloud, Kubernetes, and Terraform ✔ Deep understanding of Apache Spark for large-scale data processing ✔ Knowledge of Apache Beam & Google Cloud Dataflow for data pipeline orchestration ✔ A team-first mindset with strong More ❯
Skills: Backend Development: Gradle, Liquibase Frontend Development: React, Typescript Cloud & DevOps: Deep knowledge of DevOps principles and GitOps practices, Linux (in a containerized environment), Apache Zookeeper, Apache Nifi Data: Redis, GIS Architecture: API Spec (Swagger OpenAPI) **PLEASE DO NOT APPLY IF YOU ARE NOT PRE-CLEARED TO SC More ❯
with TensorFlow, PyTorch, Scikit-learn, etc. is a strong plus. You have some experience with large scale, distributed data processing frameworks/tools like Apache Beam, Apache Spark, or even our open source API for it - Scio, and cloud platforms like GCP or AWS. You care about agile More ❯
Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13+ billion. Location - London Skill - Apache Hadoop We are looking for open-source contributors to Apache projects, who have an in-depth understanding of the code behind the Apache … bigdata tech stack. Requirement: Experience of platform engineering along with application engineering (hands-on) Experience in design of an open source platform based on Apache framework for Hadoop. Experience in integrating Infra-as-a-Code in their platform (Bespoke implementation from scratch) Experience of design & architect work for the … open source Apache platform in hybrid cloud environment Ability to do debug & fix code in the open source Apache code and should be an individual contributor to open source projects. Job description: The Apache Hadoop project requires up to 3 individuals with experience in designing and building More ❯
plus Puppet, SaltStack), Terraform, CloudFormation; Programming Languages and Frameworks Node.js, React/Material-UI (plus Angular), Python, JavaScript; Big Data Processing and Analysis e.g., Apache Hadoop (CDH), Apache Spark; Operating Systems Red Hat Enterprise Linux, CentOS, Debian, or Ubuntu. More ❯
object-oriented programming (OOP) principles & concepts Familiarity with advanced SQL techniques Familiarity with data visualization tools such as Tableau or Power BI Familiarity with Apache Flink or Apache Storm Understanding of DevOps practices and tools for (CI/CD) pipelines. Awareness of data security best practices and compliance More ❯
Our team values continuous learning, knowledge sharing, and creating inclusive solutions that make a difference. Key Responsibilities Support customers with big data services including Apache Spark, Hive, Presto, and other Hadoop ecosystem components Develop and share technical solutions through various communication channels Contribute to improving support processes and customer … work week schedule, which may include weekends on rotation. BASIC QUALIFICATIONS - Good depth of understanding in Hadoop Administration, support and troubleshooting (Any two applications: Apache Spark, Apache Hive, Presto, Map-Reduce, Zookeeper, HBASE, HDFS and Pig.) - Good understanding of Linux and Networking concepts - Intermediate programming/scripting skills. More ❯
Scala. ? AI Frameworks: Extensive experience with AI frameworks and libraries, including TensorFlow, PyTorch, or similar. ? Data Processing: Expertise in big data technologies such as Apache Spark, Hadoop, and experience with data pipeline tools like Apache Airflow. ? Cloud Platforms: Strong experience with cloud services, particularly AWS, Azure, or Google More ❯
Manchester, Lancashire, United Kingdom Hybrid / WFH Options
Smart DCC
you be doing? Design and implement efficient ETL processes for data extraction, transformation, and loading. Build real-time data processing pipelines using platforms like Apache Kafka or cloud-native tools. Optimize batch processing workflows with tools like Apache Spark and Flink for scalable performance. Infrastructure Automation: Implement Infrastructure … Integrate cloud-based data services with data lakes and warehouses. Build and automate CI/CD pipelines with Jenkins, GitLab CI/CD, or Apache Airflow. Develop automated test suites for data pipelines, ensuring data quality and transformation integrity. Monitoring & Performance Optimization: Monitor data pipelines with tools like Prometheus More ❯
microservices Integrate with and optimize data persistence layers using MongoDB (preferably MongoDB Atlas) , Redis , or DynamoDB Implement distributed caching strategies with Redis, Hazelcast, or Apache Ignite Work closely with DevOps to containerize applications using Docker Must-Have Qualifications 5+ years of hands-on experience in backend development with Java … Production experience with NoSQL databases (MongoDB preferred; DynamoDB, Redis, or similar are a plus) Experience with distributed caching systems such as Redis, Hazelcast, or Apache Ignite Proficiency with Docker and containerized deployments Experience with cloud-based environments (AWS, GCP, or Azure; MongoDB Atlas a strong plus More ❯
microservices Integrate with and optimize data persistence layers using MongoDB (preferably MongoDB Atlas) , Redis , or DynamoDB Implement distributed caching strategies with Redis, Hazelcast, or Apache Ignite Work closely with DevOps to containerize applications using Docker Must-Have Qualifications 5+ years of hands-on experience in backend development with Java … Production experience with NoSQL databases (MongoDB preferred; DynamoDB, Redis, or similar are a plus) Experience with distributed caching systems such as Redis, Hazelcast, or Apache Ignite Proficiency with Docker and containerized deployments Experience with cloud-based environments (AWS, GCP, or Azure; MongoDB Atlas a strong plus More ❯
is made up of a series of components providing the next generation valuation and risk management services. Responsibilities Development of big data technologies like Apache Spark and Azure Databricks Programming complex production systems in Scala, Java, or Python Experience in a platform engineering role on a major cloud provider … development, build, and runtime environments including experience of Kubernetes Salary - 140-150K/year SKILLS Must have Development of big data technologies like Apache Spark and Azure Databricks Programming complex production systems in Scala, Java, or Python Experience in a platform engineering role on a major cloud provider More ❯
