diverse sources, transform it into usable formats, and load it into data warehouses, data lakes or lakehouses. Big Data Technologies: Utilize big data technologies such as Spark, Kafka, and Flink for distributed data processing and analytics. Cloud Platforms: Deploy and manage data solutions on cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP), leveraging cloud-native services More ❯
for data transformation and performance optimization. Hands-on experience with data pipeline tools (Airflow, dbt, Kafka, or equivalent). Strong knowledge of big data processing frameworks (Apache Spark, Databricks, Flink, etc.). Cloud & Infrastructure: Experience with cloud computing platforms (AWS, Azure, Google Cloud). Familiarity with modern data architectures (data lakes, lakehouses, warehouses). Exposure to containerization and orchestration More ❯
designing and supporting multi-tenant SaaS data platforms with strategies for data partitioning, tenant isolation, and cost management Exposure to real-time data processing technologies such as Kafka, Kinesis, Flink, or Spark Streaming, alongside batch processing capabilities Strong knowledge of SaaS compliance practices and security frameworks Core Competencies Excellent problem-solving abilities with the capacity to translate requirements into More ❯
DAST pipeline integration, STIG automation (Ansible), SBOM generation (CycloneDX/Syft/Grype), and supply-chain security (SLSA/SCVS). Real-time/streaming analytics (Kafka Streams/Flink/Spark), geospatial, or time-series processing for mission applications. GDIT IS YOUR PLACE At GDIT, the mission is our purpose, and our people are at the center of More ❯
Familiarity with MLOps, model serving, and AI workload optimization. Knowledge of A/B testing frameworks and feature flag systems. Experience with real-time data platforms (e.g., Apache Kafka, Flink). Background in microservices and distributed systems architecture. Open-source contributions. Understanding of security, compliance, and risk management in distributed systems. Exposure to chaos engineering and resilience testing. Minimum More ❯
external tools and data sources Understanding of secure, real-time data access methodologies. Streaming Data Processing: Hands-on experience with real-time streaming data processing technologies (e.g., Apache Kafka, Flink, Spark Streaming, Kinesis). Proficiency with cloud-native data platforms (e.g., AWS Kinesis/MSK/S3, Azure Event Hubs/Data Lake, GCP Pub/Sub/BigQuery More ❯
Experience with Apache Spark Hands-on experience with AWS data services (e.g., S3, Glue, Lambda, MSK) Capable in batch or streaming data processing using technologies such as Spark, Kafka, Flink, and DLT Proficient in CI/CD pipelines, automated testing, code quality enforcement, and environment management for production-grade data systems Expert in orchestration and transformation frameworks such as More ❯
Liverpool, Merseyside, North West, United Kingdom Hybrid / WFH Options
Forward Role
Excellent stakeholder management and documentation skills Team leadership experience with ability to mentor and develop engineering talent Nice to haves: Knowledge of data streaming platforms such as Kafka or Flink Exposure to graph databases or vector database technologies Professional certifications in Azure or AWS cloud platforms If you're ready to take the lead on transformative data engineering projects More ❯
Java, C, C++ for distributed systems, with proficiency in networking, multi-threading and implementation of REST APIs. Experience with the Spring framework, messaging frameworks (Kafka, RabbitMQ), streaming analytics (ApacheFlink, Spark), management of containerized applications (Kubernetes). Experience with Enabling tools (Git, Maven, Jira), DevOps (Bamboo, Jenkins, GitLab Cl/Pipelines), Continuous Monitoring (ELK Stack (ElasticSearch, Logstash and Kibana More ❯
Configure and manage data analytic frameworks and pipelines using databases and tools such as (but not limited to) NoSQL, SQL, NiFi, Kafka, HDInsight, MongoDB, Cassandra, Neo4j, GraphDB, OrientDB, Spark, Flink, Hadoop, Kafka, Hive, and others. • Apply distributed systems concepts and principles such as consistency and availability, liveness and safety, durability, reliability, fault-tolerance, consensus algorithms. • Administrate cloud computing and More ❯
leading data and ML platform infrastructure, balancing maintenance with exciting greenfield projects. develop and maintain our real-time model serving infrastructure, utilising technologies such as Kafka, Python, Docker, ApacheFlink, Airflow, and Databricks. Actively assist in model development and debugging using tools like PyTorch, Scikit-learn, MLFlow, and Pandas, working with models from gradient boosting classifiers to custom GPT More ❯
Israel. Willingness and ability to travel abroad. Bonus Points: Knowledge and hands-on experience of Office 365 - A big advantage. Experience in Kafka, and preferably some exposure to ApacheFlink, is a plus. Why Join Semperis? You'll be part of a global team on the front lines of cybersecurity innovation. At Semperis, we celebrate curiosity, integrity, and people More ❯
Israel. Willingness and ability to travel abroad. Bonus Points: Knowledge and hands-on experience of Office 365 - A big advantage. Experience in Kafka, and preferably some exposure to ApacheFlink, is a plus. Why Join Semperis? You'll be part of a global team on the front lines of cybersecurity innovation. At Semperis, we celebrate curiosity, integrity, and people More ❯
practical experience building and running such systems in production. Strong coding skills in Java/Python and familiarity with CI/CD. Hands-on with some of: Kafka/Flink, Spark, Delta/Iceberg, Kubernetes, NoSQL/columnar stores. Proven ability to work independently, make sound tradeoffs, and deliver quality outcomes with minimal supervision. Solid debugging, performance analysis, and More ❯
to Date with Technology: Keep yourself and the team updated on the latest Python technologies, frameworks, and tools like Apache Spark, Databricks, Apache Pulsar, Apache Airflow, Temporal, and ApacheFlink, sharing knowledge and suggesting improvements. Documentation: Contribute to clear and concise documentation for software, processes, and systems to ensure team alignment and knowledge sharing. Your Qualifications Experience: Professional experience … pipelines, and machine learning workflows. Workflow Orchestration: Familiarity with tools like Apache Airflow or Temporal for managing workflows and scheduling jobs in distributed systems. Stream Processing: Experience with ApacheFlink or other stream processing frameworks is a plus. Desired Skills Asynchronous Programming: Familiarity with asynchronous programming tools like Celery or asyncio. Frontend Knowledge: Exposure to frontend frameworks like React More ❯
delivering under tight deadlines without compromising quality. Your Qualifications 12+ years of software engineering experience, ideally in platform, infrastructure, or data-centric product development. Expertise in Apache Kafka, ApacheFlink, and/or Apache Pulsar. Deep understanding of event-driven architectures, data lakes, and streaming pipelines. Strong experience integrating AI/ML models into production systems, including prompt engineering More ❯
delivering under tight deadlines without compromising quality. Your Qualifications 12+ years of software engineering experience, ideally in platform, infrastructure, or data-centric product development. Expertise in Apache Kafka, ApacheFlink, and/or Apache Pulsar. Deep understanding of event-driven architectures, data lakes, and streaming pipelines. Strong experience integrating AI/ML models into production systems, including prompt engineering More ❯
with diverse stakeholders to identify priorities for the Fixed income data acquisition platform and the respective ownership of the roadmap We'd love to see: - Experience with Kafka, ApacheFlink, or Trino - Experience using SQL or Python to explore data - Experience in tick level databases and time series analytics - Systems thinkers - an interest or a natural preference to think More ❯
with a view to becoming an expert BS degree in Computer Science or meaningful relevant work experience Preferred Qualifications Experience with large scale data platform infrastructure such as Spark, Flink, HDFS, AWS/S3, Parquet, Kubernetes is a plus More ❯
Frameworks: Spring Boot, Flask Familiarity with multiple of the following: Analytical development: Machine Learning, Stream, Batch Virtualization Environments: Kubernetes, Containers (Docker, Containerd), Cloud (AWS/HCI) Streaming Frameworks: Kafka, Flink MongoDB, ArangoDB, Redis, PostgreSQL Desired skills: ElasticSearch Additional information: Work performed in contractor facility in Annapolis Junction Both low and high side access Potential telework Individual Capabilities Required: Seven More ❯
Columbia, Maryland, United States Hybrid / WFH Options
HII Mission Technologies
effectively navigating and contributing within the Risk Management Framework (RMF) process Proficiency in system design and meticulous documentation Experience in streaming and/or batch analytics (e.g. Kafka, Spark, Flink, Storm, MapReduce, Hadoop) Experience in distributed databases, NoSQL databases, full text-search engines (e.g. Elasticsearch, MongoDB, Solr) Experience in designing enterprise APIs. Experience in RESTful web services Experience in More ❯
of the platform Your Qualifications 12+ years of software engineering experience in enterprise-scale, data-centric, or platform environments Deep expertise in distributed data technologies such as Apache Kafka, Flink, and/or Pulsar Strong background in event-driven architectures, streaming pipelines, and data lakes Hands-on experience with AI/ML production systems, including prompt-based LLM integrations More ❯
lifecycle management and model versioning Hands-on experience with Databricks Model Serving for production ML deployments Proficiency with GenAI frameworks/tools and technologies such as Apache Airflow, Spark, Flink, Kafka/Kinesis, Snowflake, and Databricks. Demonstrable experience in parameter-efficient fine-tuning, model quantization, and quantization-aware fine-tuning of LLM models Hands-on knowledge of Chain-of More ❯
design and problem-solving skills We'd love to see : Experience building platforms that serve a large number of tenants Familiarity with large-scale data processing frameworks (e.g., Spark, Flink, Kafka) Passion for data quality, performance analysis, anomaly detection Familiarity with testing infrastructure, CI/CD workflows, or developer productivity tooling Experience with owning end-to-end delivery of More ❯
Liverpool, Merseyside, England, United Kingdom Hybrid / WFH Options
red recruitment
to manage secure, high-performance database environments. Excellent communication and cross-functional collaboration skills. A passion for continuous learning and innovation. Desirable: AzureSynapse/Sharedo/Databricks Python/Flink/Kafka technology If you are interested in this Senior Database Developer position and have the relevant skills and experience required, please apply now! Red Recruitment (Agency More ❯