16 of 16 Apache Jobs in Scotland

vector databases Streaming technologies (Kafka, Kinesis, Flink, etc.) Containers and related services (Docker, Kubernetes, container Registries, etc) Data modelling approaches (Kimball, Imnon) Orchestration tools - Apache Airflow, Prefect or cloud-native tools Backend software development (Java, APIs, Scalability, Logging and Monitoring etc.) MLFlow and other MLOps/Machine Learning Engineering ...

cloud platforms such as AWS, Azure or GCP is desirable. Basic understanding of relational databases and SQL queries. Awareness of tools such as Airflow, Apache NiFi or Talend is helpful. Foundations in data warehousing, modelling and schema design, with an interest in building dashboards using Power BI, QlikView ...

inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworks Familiarity with AWS-native data services such as Glue, Lake Formation, Redshift, or Athena Exposure to data governance practices ...

external customers. Cloud data platform experience (AWS, Azure or GCP) and streaming/real-time data. Wider data product experience across platforms such as Apache Spark, Databricks and Snowflake. Exposure to AI/ML data pipelines and emerging agent interfaces such as MCP. Hands-on master data management delivery ...

scripts, with a particular focus on building and optimizing multi‐threaded, multi‐process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI‐powered solutions using Large Language Models (LLMs), retrieval‐augmented ...

scripts, with a particular focus on building and optimizing multi-threaded, multi-process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI-powered solutions using Large Language Models (LLMs), retrieval‐augmented ...

analytical and operational domains.Establish ELT/ETL and streaming architectures on Snowflake Snowpipe, Snowpipe Streaming, Dynamic Tables, Streams and Tasks, external tables, and Apache Iceberg tables along with orchestration standards (dbt, Airflow, Snowflake Tasks) and CI/CD/DataOps best practices.Implement security-by-design in Snowflake: RBAC … Azure, or GCP) including Snowflake deployment, networking (Private Link), and cloud storage integration (S3, ADLS, GCS); familiarity with lakehouse stacks (Databricks, Delta Lake, Apache Iceberg) is a plus.Mastery of Snowflake security and governance: RBAC, tag-based governance, dynamic data masking, row access policies, column-level security, Tri-Secret encryption ...

C#, .NET, Python, etc.) found in software development. Knowledge in Operating Systems (Linux, MS Windows) Databases (MS SQL, MySQL, Postgres), Application/Web Servers (Apache, IIS, Wildfly) Knowledge/experience in Networking, including switching, routing, firewalls and vulnerabilities Knowledgable about AI (artificial intelligence) Security Testing Tools such ...

processes. Technical Skills Programming: Proficiency in Python, Java, Scala, or similar languages. Big Data Technologies: Hands-on experience with big data tools e.g. (Databricks, Apache Spark, Hadoop). Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Expertise in relational databases (e.g. postgres … server) Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment. Experience 2+ years ...

have Programming: Proficiency in Python, Java, Scala, or similar languages for data processing. Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop. Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Database Management: Expertise in relational databases … e.g. Postgres, SQL Server). Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines ...

scalable systems together. Key Experience Experience working with real-time streaming applications using Kafka Streams, including KStream and KTable concepts. Experience working with Apache Flink solutions, including stateful processing, checkpointing, and fault-tolerant approaches. Experience working with high-throughput, low-latency data pipelines that process and transform large volumes ...

lifecycle management. Experience working within Agile development teams. Active UK SC security clearance. Highly Desirable AWS – experience building or supporting cloud-native data solutions. Apache Kafka – experience with event streaming, messaging or real-time data processing. Kubernetes – experience with container orchestration and distributed application deployment. Experience developing Java-based … data processing or event-driven applications. Additional Advantageous Experience Apache Spark, Flink, Hadoop or other Big Data technologies. Working with high-volume or complex enterprise datasets. Experience delivering solutions within government, defence or other secure environments. Why Consider This Opportunity? Long-term contract running until December 2027 . Work ...

Experience building applications on public cloud platforms like AWS, GCP, and Azure, with a preference for AWS. Proficiency in real-time streaming applications using Apache Spark Streaming, Apache Kafka, Amazon Kinesis, and emerging cloud serverless managed services. #J-18808-Ljbffr ...

teams to maintain reliable data processes and ensure compliance with technology, security, and change standards. Responsibilities Develop, maintain, and troubleshoot data orchestration workflows using Apache Airflow for extract generation, data publishing, feed delivery, and operational batch processes. Build and support reusable engineering components using Python, Scala, and Sharedview Spark … with technology, security, and change standards. Qualifications Strong skills and experience in data engineering using Spark, Scala, and Python. Experience developing, maintaining, and troubleshooting Apache Airflow workflows for data orchestration and operational batch processes. Hands-on experience building reusable engineering components to process, transform, and validate large-scale financial ...

invites applications for a Lead Data Engineer based in Glasgow. You will design and develop real-time data streaming solutions using Kafka Streams and Apache Flink, building scalable, low-latency data platforms to support critical business processes. You will influence architecture, standards, and collaboration with data scientists for model ...

APIs using Java, Scala, and Python. Design, develop, and operate real-time data pipelines and stream-processing applications using Java, Scala, Kafka Streams, and Apache Flink, as well as batch-processing workflows using Python and Apache Airflow. Drive end-to-end technical ownership of backend services from initial ...