1 to 25 of 425 Apache Jobs

robust way possible! Diverse training opportunities and social benefits (e.g. UK pension schema) What do you offer? Strong hands-on experience working with modern Big Data technologies such as Apache Spark, Trino, Apache Kafka, Apache Hadoop, Apache HBase, Apache Nifi, Apache Airflow, Opensearch Proficiency in cloud-native technologies such as containerization and Kubernetes Strong More ❯

real-time systems, and business management information systems) based upon documented requirements for the Data Transport System (DTS) • DTS products include but are not limited to: Cloud storage areas: Apache Accumulo (Apache Zookeeper, ApacheHadoop), Oracle DMBS Real time streaming: Storm Distributed in-memory data Cache/Storage: Redis, Graph Compute engine/Query Interface apache Tinkerpop/… Gremlin. Rules Engine: JBoss Drools, Apache Camel, Spring Framework: used extensively to standardize/simplify configuration logic control, data access, security, Web-tier, etc. Candidates will: o Analyze user requirements to derive software design and performance requirements o Debug existing software and correct defects o Design and code new software or modify existing software to add new features o … DoD) and other federal partners • The DTS portfolio encompasses transport streams, messages and files with content size ranging from bytes to Terabytes • Candidates should have experience writing analytics using Apache Hadoop, HDFS, and MapReduce • Experience processing large data sets or high-volume data ingest is a plus • Experience monitoring, maintaining and troubleshooting Apache Accumulo, Apache Hadoop, and More ❯

and Snowflake Marketplace. Familiarity with Snowpark for Python/Java-based transformations. Understanding of role-based access control, data masking, and time travel features. Databricks: Hands-on experience with Apache Spark and Databricks Runtime. Proficiency in Delta Lake for ACID-compliant data lakes. Experience with Structured Streaming and Auto Loader. Familiarity with MLflow, Feature Store, and Model Registry. Use More ❯

on experience with cloud data platforms such as Snowflake, Redshift, Athena, or BigQuery, including optimization techniques and custom parsers/transpilers. Practical knowledge of distributed and analytical engines (e.g., Apache Spark, Trino, PostgreSQL, DuckDB) with skills in query engines, performance tuning, and integration in local and production environments. Experience building developer tooling such as CLI tools, SDKs, and database More ❯

and solution design. Drive excellence in data governance, cloud security, and continuous improvement . What You Bring Solid track record of designing and scaling modern data architectures. Mastery of Apache Spark , PySpark , and Databricks (Delta Lake, Unity Catalog, MLflow, Workflows). Strong command of Python and SQL for building reusable, modular pipelines. Expertise in AWS (S3, Lambda, Glue, API More ❯

Responsibilities: Develop and Maintain Data Integration Solutions: o Design and implement data integration workflows using AWS Glue/EMR,AWS MWAA(Airflow), Lambda, Redshift o Demonstrate proficiency in Pyspark, Apache Spark and Python for data processing large datasets o Ensure data is accurately and efficiently extracted, transformed, and loaded into target systems. Ensure Data Quality and Integrity: o Validate More ❯

in AWS. Strong expertise with AWS services, including Glue, Redshift, Data Catalog, and large-scale data storage solutions such as data lakes. Proficiency in ETL/ELT tools (e.g. Apache Spark, Airflow, dbt). Skilled in data processing languages such as Python, Java, and SQL. Strong knowledge of data warehousing, data lakes, and data lakehouse architectures. Excellent analytical and More ❯

data quality, or other areas directly relevant to data engineering responsibilities and tasks Proven project experience developing and maintaining data warehouses in big data solutions (Snowflake) Expert knowledge in Apache technologies such as Kafka, Airflow, and Spark to build scalable and efficient data pipelines Ability to design, build, and deploy data solutions that capture, explore, transform, and utilize data More ❯

Advanced SQL skills for data transformation and performance optimization. Hands-on experience with data pipeline tools (Airflow, dbt, Kafka, or equivalent). Strong knowledge of big data processing frameworks (Apache Spark, Databricks, Flink, etc.). Cloud & Infrastructure: Experience with cloud computing platforms (AWS, Azure, Google Cloud). Familiarity with modern data architectures (data lakes, lakehouses, warehouses). Exposure to More ❯

experience in software engineering practices (e.g., version control, CI/CD pipelines, containerization) Experience building and optimizing data pipelines, ETL processes, and real-time streaming solutions using tools like Apache Airflow, Kafka, Spark, or equivalent Knowledge of applied AI principles, particularly in implementing AI systems for operational decision support and analyzing unstructured data (e.g., text, imagery) Ability to architect More ❯

of SQL and Python You have strong hands-on experience of building scalable data pipelines in cloud based environments using tools such as DBT, AWS Glue, AWS Lake Formation, Apache Spark and Amazon Redshift You have a good knowledge of data modelling, ELT design patterns, data governance and security best practices You're collaborative and pragmatic with great communication More ❯

end development with React. Experience with .NET/C#. Knowledge of containerization tools like Docker and Kubernetes. Exposure to SQL/NoSQL databases (e.g., CosmosDB, DocumentDB). Experience with Apache Airflow, Snowflake, or microservices architecture. Awareness of cloud security best practices. If interested, please apply for an immediate discussion Skills Python,Asset Management,Cloud Deployment About UST UST is More ❯

us even more if you have some of these skills: Active Security+ certification Active CE certification Ability to obtain Privileged User Account (PUA) certification by the Government GraphQL Service Apache NiFi Experience with systems at scale Kafka or other message passing systems Experience of a production/enterprise system Appian low-code application development platform Experience with any of More ❯

systems. Nice to haves: Experience with NoSQL databases (MongoDB, Cassandra, Redis). Familiarity with message brokers (Kafka, SQS/SNS, RabbitMQ). Knowledge of real-time streaming (Kafka Streams, Apache Flink, etc.). Exposure to big-data or machine-learning frameworks (TensorFlow, PyTorch, Hugging Face, LangChain). Experience working with AI-driven development tools such as Cursor, Copilot, or More ❯

of data modelling and data warehousing concepts Familiarity with version control systems, particularly Git Desirable Skills: Experience with infrastructure as code tools such as Terraform or CloudFormation Exposure to Apache Spark for distributed data processing Familiarity with workflow orchestration tools such as Airflow or AWS Step Functions Understanding of containerisation using Docker Experience with CI/CD pipelines and More ❯

Linux-based, concurrent, high-throughput, low-latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelor or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about DRW's processing activities More ❯

in SQL and experience with relational and NoSQL databases. Familiarity with cloud platforms (e.g., AWS or Azure) and related services for data engineering. Experience with data orchestration tools like Apache Airflow or similar is a plus Certifications in Confluent Kafka and/or Qlik a plus Experience with containerization and orchestration (e.g., Docker, Kubernetes) a plus Knowledge of data More ❯

the first time, Dev10 equips you with the tools to succeed. You'll train in technologies including Relational and Non-Relational Databases, APIs, Python, Pandas, Excel, Dash, Kafka, Airflow, Apache Spark, and Machine Learning. Now hiring for our Data Engineer cohort starting Monday, December 1, 2025. To apply, please submit a formal application through our website: Dev10 Apply Here More ❯

the first time, Dev10 equips you with the tools to succeed. You'll train in technologies including Relational and Non-Relational Databases, APIs, Python, Pandas, Excel, Dash, Kafka, Airflow, Apache Spark, and Machine Learning. Now hiring for our Data Engineer cohort starting Monday, December 1, 2025. To apply, please submit a formal application through our website: Dev10 Apply Here More ❯

the first time, Dev10 equips you with the tools to succeed. You'll train in technologies including Relational and Non-Relational Databases, APIs, Python, Pandas, Excel, Dash, Kafka, Airflow, Apache Spark, and Machine Learning. Now hiring for our Data Engineer cohort starting Monday, December 1, 2025. To apply, please submit a formal application through our website: Dev10 Apply Here More ❯

the first time, Dev10 equips you with the tools to succeed. You'll train in technologies including Relational and Non-Relational Databases, APIs, Python, Pandas, Excel, Dash, Kafka, Airflow, Apache Spark, and Machine Learning. Now hiring for our Data Engineer cohort starting Monday, December 1, 2025. To apply, please submit a formal application through our website: Dev10 Apply Here More ❯

standards. Develop and deliver documentation for each project including ETL mappings, code use guide, code location and access instructions. Design and optimize Data Pipelines using tools such as Spark, Apache Iceberg, Trino, OpenSearch, EMR cloud services, NiFi and Kubernetes containers Ensure the pedigree and provenance of the data is maintained such that the access to data is protected Clean More ❯

data engineering tasks. Experience building and maintaining web scraping pipelines. Strong SQL skills, with expertise in performance tuning. Strong proficiency with dbt for data transformations. Hands-on experience with Apache Airflow or Prefect. Proficiency with GitHub, GitHub Actions, and CI/CD pipelines. Nice to have: Experience with GCP (BigQuery, Dataflow, Composer, Pub/Sub) or AWS. Familiarity with More ❯

If You Have: Experience with NoSQL DBs, including Mongo, ElasticSearch, Redis, or Graph DB Experience with data wrangling, including Discovery, Mining, Cleaning, Exploration, Modeling, Structuring, Enriching, and Validating, with Apache NiFi or related tools Experience with CI/CD, including Jenkins, Junit testing, or related Experience with DevOps, including Packer, Terraform, or Ansible Experience with containerization, including Docker or More ❯

with DevOps technologies Experience with CI/CD technologies, including Ansible, Jenkins, and GIT Experience with Agile development or Scrum Knowledge of non-relational databases, including MongoDB Knowledge of Apache NiFi and Kafka Ability to work in a fast-paced and collaborative team environment Possession of excellent verbal and written communication skills TS/SCI clearance with a polygraph More ❯