flows through the pipeline. Collaborate with research to define data quality benchmarks . Optimize end-to-end performance across distributed data processing frameworks (e.g., Apache Spark, Ray, Airflow). Work with infrastructure teams to scale pipelines across thousands of GPUs . Work directly with the leadership on the data More ❯
Coalville, Leicestershire, East Midlands, United Kingdom Hybrid / WFH Options
Ibstock PLC
and BI solutions. Ensure data accuracy, integrity, and consistency across the data platform. Knowledge, Skills and Experience: Essentia l Strong expertise in Databricks and Apache Spark for data engineering and analytics. Proficient in SQL and Python/PySpark for data transformation and analysis. Experience in data lakehouse development and More ❯
Falls Church, Virginia, United States Hybrid / WFH Options
Epsilon Inc
the following certifications are desired: Certified Cloud Security Professional (CCSP) , GIAC Security Essentials Certification (GSEC), or CompTIA Cybersecurity Analyst (CySA+) Preferred Qualifications: Familiarity with Apache Spark or comparable distributed data processing frameworks, preferably for large-scale data transformations and analytics. Working knowledge of data governance platforms (e.g., Collibra) and More ❯
Washington, Washington DC, United States Hybrid / WFH Options
Marathon TS Inc
information systems and Federal health care systems is required. Experience must include the following: Java, J2EE, SOA services, Spring, Object Oriented Concepts, Data Structures, Apache, Tomcat, REST Services, Web technologies and internet protocols. Database driven application development such as Oracle, SQL, or schema design; HTML, AJAX, SOA: SOAP/ More ❯
Washington, Washington DC, United States Hybrid / WFH Options
Marathon TS Inc
information systems and Federal health care systems is required. Experience must include the following: Java, J2EE, SOA services, Spring, Object Oriented Concepts, Data Structures, Apache, Tomcat, REST Services, Web technologies and internet protocols. Database driven application development such as Oracle, SQL, or schema design; HTML, AJAX, SOA: SOAP/ More ❯
design, implementation and technical oversight of an internal data ingestion platform. My client are looking for an individual that has the following: Experience with Apache Spark, Kafka, and Airflow for data ingestion and orchestration. Experience in designing and deploying solutions on Azure Kubernetes Service (AKS). Knowledge of the More ❯
Leeds, West Yorkshire, Yorkshire, United Kingdom Hybrid / WFH Options
Evri
flexibility for remote work. Office visits required for key meetings and collaboration sessions. Key Responsibilities: Develop and maintain scalable data pipelines using Databricks and Apache Spark to process logistics and delivery data Design ETL workflows that integrate data from multiple delivery and warehouse systems Development of Data Marts on More ❯
Newport Pagnell, Buckinghamshire, South East, United Kingdom Hybrid / WFH Options
Marshall Resource Partners Ltd
changes effectively. Experience: 2+ years of experience in PHP development. Experience with databases such as PostgreSQL. Experience in developing PHP applications running on either Apache or Nginx. Familiarity with RESTful API development and integration. Understanding of authentication systems like OAuth 2.0. Experience with source control systems such as Git. More ❯
Purview or equivalent for data governance and lineage tracking Experience with data integration, MDM, governance, and data quality tools . Hands-on experience with Apache Spark, Python, SQL, and Scala for data processing. Strong understanding of Azure networking, security, and IAM , including Azure Private Link, VNETs, Managed Identities, and More ❯
Luton, Bedfordshire, United Kingdom Hybrid / WFH Options
leonardo company
in Electronic Intelligence or Electronic Warfare Your work may be exploited into: The protection systems for current and future RAF aircraft, including Typhoon and Apache The technology development platforms being developed by the UK's Team Tempest to support the development of a new 6th generation aircraft within the More ❯
Fort George G Meade, Maryland, United States Hybrid / WFH Options
August Schell
configure monitoring for data pipelines to ensure high availability and throughput, low latency, and alerting • Knowledge of stream processing pipelines and analytics. • Experience with Apache NiFi, multi-cluster or containerized environment experience preferred. • Knowledge of cybersecurity concepts, including threats, vulnerabilities, security operations, encryption, boundary defense, auditing, authentication, and supply More ❯
Colorado Springs, Colorado, United States Hybrid / WFH Options
Aerospace Corporation
Logrhythm, ACAS/Nessus/SCAP, mandatory/role-based access control concepts, SE Linux extensions to RHE, Oracle/MS SQL database security, Apache/IIS Web server security Experience in Linux systems administration, including configuration, troubleshooting, performance tuning, and security hardening for an enterprise environment How You More ❯
Fort Belvoir, Virginia, United States Hybrid / WFH Options
CGI
knowledge of RMF Excellent knowledge of and proficiency with: o VULNERATOR o USCYBERCOM CTO Compliance Program o Wireless vulnerability assessment o Web Services (IIS, Apache, Proxy) o Database (SQL Server, Oracle) o Email Services (Exchange) o Vulnerability Scans (NESSUS, SCCM) o Knowledge of Phishing exercises o USB Detect o More ❯
Data Streaming , Unity Catalogue etc.). Prove experience designing high volume, live data streaming solutions using Azure DLT (Delta Live Tables). Expert with Apache Spark and PySpark (ability to review quality of code and debug issues). Experience with Qlik Replicate to move data from on-prem to More ❯
data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our More ❯
Sheffield, Yorkshire, United Kingdom Hybrid / WFH Options
Reach Studios Limited
with web applications utilising the power of Laravel for back-end processes. Front-ends are built in Alpine.js/React/Next.js, rendered by Apache or NGINX servers. Both REST and GraphQL APIs are utilised to integrate and extend the services we provide to our array of clients. What More ❯
or product feature use cases. Experience in building and deploying live software services in production. Exposure to some of the following technologies (or equivalent): Apache Spark, AWS Redshift, AWS S3, Cassandra (and other NoSQL systems), AWS Athena, Apache Kafka, Apache Flink, AWS and service oriented architecture. What More ❯
processing large-scale data. Experience with ETL processes for data ingestion and processing. Proficiency in Python and SQL. Experience with big data technologies like Apache Hadoop and Apache Spark. Familiarity with real-time data processing frameworks such as Apache Kafka or Flink. MLOps & Deployment: Experience deploying and More ❯
Birmingham, Staffordshire, United Kingdom Hybrid / WFH Options
Yelp USA
to the experimentation and development of new ad products at Yelp. Design, build, and maintain efficient data pipelines using large-scale processing tools like Apache Spark to transform ad-related data. Manage high-volume, real-time data streams using Apache Kafka and process them with frameworks like Apache Flink. Estimate timelines for projects, feature enhancements, and bug fixes. Work with large-scale data storage solutions, including Apache Cassandra and various data lake systems. Collaborate with cross-functional teams, including engineers, product managers and data scientists, to understand business requirements and translate them into effective system designs. … a proactive approach to identifying opportunities and recommending scalable, creative solutions. Exposure to some of the following technologies: Python, AWS Redshift, AWS Athena/Apache Presto, Big Data technologies (e.g S3, Hadoop, Hive, Spark, Flink, Kafka etc), NoSQL systems like Cassandra, DBT is nice to have. What you'll More ❯
or product feature use cases. Experience in building and deploying live software services in production. Exposure to some of the following technologies (or equivalent): Apache Spark, AWS Redshift, AWS S3, Cassandra (and other NoSQL systems), AWS Athena, Apache Kafka, Apache Flink, AWS, and service-oriented architecture. What More ❯
Location: Remote-first (UK-based) 💰 Rate: Up to £550 p/d 📆 Contract: 6 - 12 months (Outside IR35) 🛠 Tech Stack: Python, FastAPI, GCP, BigQuery, Apache Spark, Apache Beam, Google Cloud Dataflow We're working with a forward-thinking consultancy that helps top companies build and scale high-performance … You’ll Be Doing: 🔹 Building data pipelines and ETL workflows that process huge datasets 🔹 Designing, optimizing, and maintaining high-throughput reporting solutions 🔹 Working with Apache Spark for large-scale data processing 🔹 Using Apache Beam and Google Cloud Dataflow to manage complex data workflows 🔹 Developing and improving backend APIs … writing clean, efficient, and scalable code ✔ Experience with BigQuery, PostgreSQL, and Elasticsearch ✔ Hands-on experience with Google Cloud, Kubernetes, and Terraform ✔ Deep understanding of Apache Spark for large-scale data processing ✔ Knowledge of Apache Beam & Google Cloud Dataflow for data pipeline orchestration ✔ A team-first mindset with strong More ❯
Chantilly, Virginia, United States Hybrid / WFH Options
Aerospace Corporation
with big data frameworks (Hadoop, Spark, Flink etc.) Experience with ML lifecycle management tools (MLflow, Kubeflow, etc.) Familiarity with data pipelining and streaming technologies (Apache Kafka, Apache Nifi, etc.) Demonstrated contributions to open-source software repositories (github, kaggle, etc.) Experience deploying ML models on cloud platforms (AWS, Azure More ❯
Our team values continuous learning, knowledge sharing, and creating inclusive solutions that make a difference. Key Responsibilities Support customers with big data services including Apache Spark, Hive, Presto, and other Hadoop ecosystem components Develop and share technical solutions through various communication channels Contribute to improving support processes and customer … work week schedule, which may include weekends on rotation. BASIC QUALIFICATIONS - Good depth of understanding in Hadoop Administration, support and troubleshooting (Any two applications: Apache Spark, Apache Hive, Presto, Map-Reduce, Zookeeper, HBASE, HDFS and Pig.) - Good understanding of Linux and Networking concepts - Intermediate programming/scripting skills. More ❯