1 to 25 of 49 Apache NiFi Jobs in the UK

secure, scalable data solutions across the full lifecycle, including acquisition, integration, storage, processing and analysis. Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL. Design data models, schemas and data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate … take ownership of customer-facing deliverables. Strong Python and SQL skills, with experience building, testing and optimising data pipelines. Hands-on experience with Apache NiFi for data ingestion, routing, transformation and orchestration. Experience of data modelling and schema design, including defining data contracts or Interface Control Documents (ICDs ...

Responsibilities Design, build and maintain secure and scalable data engineering solutions. Develop and maintain data ingestion and transformation pipelines using Python, SQL and Apache NiFi . Design data models, schemas and data contracts/Interface Control Documents (ICDs) . Integrate data from REST APIs and other source systems. … maintainable. Essential Experience Proven commercial experience delivering Data Engineering solutions in project or production environments. Strong Python and SQL skills. Hands‐on experience with Apache NiFi and data pipeline orchestration. Data modelling and schema design. Parquet and S3‐compatible object storage . REST API integration. OpenShift or Kubernetes ...

What You'll Do Design, build and maintain secure, scalable data platforms across ingestion, storage, processing and analytics workloads. Develop robust data pipelines using Apache NiFi, Python and SQL, integrating complex data from APIs and enterprise systems. Define and implement data models, schemas, data contracts and Interface Control … engineering solutions within complex project or production environments. Strong hands-on experience with Python and SQL, building and optimising scalable data pipelines. Expertise with Apache NiFi and data orchestration across multiple sources and systems. Experience designing data models, schemas, data contracts and integration frameworks. Strong understanding of REST ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … techniques to handle large-scale, real-time, and unstructured data. Responsibilities include: Design and develop data pipelines, including ingestion, orchestration, and ETL processing (e.g., NiFi). Ensure data consistency, quality, and security across all processes. Create and maintain database schemas and data models. Integrate and enrich data from diverse ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … techniques to handle large-scale, real-time, and unstructured data. Responsibilities include: Design and develop data pipelines, including ingestion, orchestration, and ETL processing (e.g., NiFi). Ensure data consistency, quality, and security across all processes. Create and maintain database schemas and data models. Integrate and enrich data from diverse ...

secure, scalable data solutions across the full lifecycle, including acquisition, integration, storage, processing and analysis. Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL. Design data models, schemas and data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate … outputs. Experience required Proven experience delivering data engineering solutions in a project or production environment. Strong Python and SQL skills. Hands-on experience with Apache NiFi and data pipeline orchestration. Experience with data modelling and schema design. Experience with Parquet and S3-compatible object storage. Experience with REST ...

secure, scalable data solutions across the full lifecycle, including acquisition, integration, storage, processing and analysis. Develop and maintain data ingestion and transformation pipelines using Apache NiFi, Python and SQL. Design data models, schemas and data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate … outputs. Experience required Proven experience delivering data engineering solutions in a project or production environment. Strong Python and SQL skills. Hands-on experience with Apache NiFi and data pipeline orchestration. Experience with data modelling and schema design. Experience with Parquet and S3-compatible object storage. Experience with REST ...

bespoke integration approaches for source-system onboarding. Define conceptual, logical and physical data models, including data flows, data domains and data ownership considerations. Specify Apache NiFi architecture patterns, flow standards, interface contracts and reusable ingestion components. Define governance, metadata, lineage, auditability, security monitoring and data quality requirements. Produce … lake architectures on AWS. Strong knowledge of AWS security, IAM, networking, storage and data platform services relevant to secure data engineering. Extensive experience with Apache NiFi architecture, flow design, governance and operational considerations. Experience designing integrations using REST APIs, SOAP services, file transfers, batch ingestion, legacy adapters ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka ...

Experience of working in an Agile delivery team Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka ...

related field* Experience influencing technical decisions and supporting capability growth.* Experience with CI/CD practices* Experience with big data tools e.g. Spark, Hadoop, NiFi* Exposure to infrastructure scaling e.g. Kubernetes* Exposure to Ansible and Terraform* Understanding of ML algorithms and deployment* Experience with SQL Server high-availability technologies ...

NoSQL)* Experience with cloud technologies* Experience with cloud platforms* Experience with CI/CD practices* Experience with big data tools e.g. Spark, Hadoop, NiFi* Experience managing direct reports, providing regular guidance and performance feedback* Ability to create delivery plans with estimated timelines for personal and team workloads* Experience supporting ...

processes. Technical Skills Programming: Proficiency in Python, Java, Scala, or similar languages. Big Data Technologies: Hands-on experience with big data tools e.g. (Databricks, Apache Spark, Hadoop). Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Expertise in relational databases (e.g. postgres … server) Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment. Experience 2+ ...

have Programming: Proficiency in Python, Java, Scala, or similar languages for data processing. Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop. Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Database Management: Expertise in relational databases … e.g. Postgres, SQL Server). Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines ...

TypeScript), Bash Frontend: Vue CI/CD & Automation: Jenkins, GitLab CI, Ansible, Cucumber Cloud & Platforms: AWS, OpenShift, Linux Infrastructure as Code: Terraform Data & Streaming: Apache NiFi Monitoring & Logging: Grafana, ELK Stack, SonarQube Version Control: GitLab Automation & Infrastructure as Code (IaC) Automate infrastructure provisioning, configuration, and CI/ ...

Docker and containerised environments * Linux and Microsoft Windows * SQL and database technologies * Prometheus, Grafana and ELK * ArgoCD or Flux * Kafka or NiFi * Nexus, Artifactory or Harbor * IAM, OAuth, OIDC and secrets management * Automated testing frameworks * Networking including DNS, TCP/IP, routing, switching, firewalls and VPNs ...

short-term engagement in Wiltshire. The role requires active MOD DV clearance, strong Python and SQL skills, and hands-on experience with Apache NiFi, data modelling and OpenShift/Kubernetes to deliver end-to-end data pipelines and resilient data contracts. #J-18808-Ljbffr ...

cloud migration. Relational and NoSQL databases, data integration and search technologies. Web and application frameworks such as React, Express and Next.js. Tools including Gitlab, NiFi, Elasticsearch, Kibana, Jira and Confluence. AI coding assistants, agentic coding and ML tooling. While experience in certain domains is highly valuable, this role ...

Experience of working with MOD Accreditors; and Knowledge of Onsite/project tech stack includes but is not limited to Azure, AWS, Docker, Kubernetes, Apache (NiFi, Kafka), NodeJS, Typescript, MongoDB, AI, Machine Learning etc. Just as important are curiosity, strong problem solving ability, and the motivation to continuously ...

cleared) to design and deliver secure, scalable data platforms for critical operations in the SW England region. You will craft data pipelines with NiFi, Python and SQL, define data models and ICDs, and ensure governance across complex environments. The role requires strong stakeholder engagement and hands-on experience with ...

problem‐solving skills Excellent communication and stakeholder skills Desirable GitLab/CI/CD REST APIs IIS Jira/Confluence Keycloak, Grafana, Elasticsearch or NiFi #J-18808-Ljbffr ...

problem-solving skills - Excellent communication and stakeholder skills Desirable - GitLab/CI/CD - REST APIs - IIS - Jira/Confluence - Keycloak, Grafana, Elasticsearch or NiFi ...

cloud migration. Relational and NoSQL databases, data integration and search technologies. Web and application frameworks such as React, Express and Next.js Tools including Gitlab, NiFi, Elasticsearch, Kibana, Jira and Confluence AI coding assistants, agentic coding and ML tooling Experience in some of these areas is advantageous, but this ...

limited to: C#/.NET, SPython FastAPI/Django, TypeScript, Angular, Vue.js, PostgreSQL, MongoDB, Redis/Valkey, RabbitMQ, Kafka, Azure, S3-compatible storage, Talend, NiFi, Docker, Podman, Kubernetes, Power Platform, and Office document automation tools such as Aspose. ð Join Assystem and play a key role in delivering enterprise ...