26 to 50 of 133 Apache Jobs in the UK excluding London

Azure Cosmos DB, including SQL API and Mongo API. Excellent knowledge of Cosmos DB Change Data Feed (CDF) and Spark integration. Strong PySpark and Apache Spark distributed computing experience. Experience with Cosmos DB SDKs, including .NET, Java, Python or Node.js. Expertise in NoSQL data modelling, partitioning, indexing and query ...

with monitoring/logging tools (Prometheus, Grafana, ELK) and root cause analysis Knowledge of storage (NFS/EFS), databases (PostgreSQL), and web servers (Nginx, Apache, Tomcat) Experience with SSO and security (SAML, OIDC, Azure AD, Okta, Keycloak) Scripting skills in Python and/or Bash Profile Client-focused with ...

equivalent apprenticeship/work experience in a subject such as Mathematics, Statistics, Computer Science, or Geography DESIRABLE Exposure to Big Data Analytics (BigQuery, Apache Spark), Parallel Computing (Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Graph Computing, or Microservices Relevant industry sector knowledge across retail, property ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise Observability – Experience ...

concepts, experience with LLMs, prompt engineering, or integrating AI capabilities into applications • Data Technologies – Experience with Kafka, PySpark, Parquet and Avro data formats, Apache Airflow for workflow management, or Streamlit for building interactive data applications • Advanced Data Management – Strong SQL design, query optimization, and database architecture expertise • Observability – Experience ...

Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks Experience assessing and remediating partially built or vendor-abandoned data platform implementations ...

scripts, with a particular focus on building and optimizing multi-threaded, multi-process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI-powered solutions using Large Language Models (LLMs), retrieval‐augmented ...

Proficiency in data quality assurance, data governance, and query optimization. Nice to have: Familiarity with the AWS data ecosystem (S3, Glue, EMR/PySpark, Apache Iceberg). Non-Technical Skills & Competencies Proven sense of ownership, commitment, and personal accountability. Meticulous attention to detail with a proactive problem-solving mindset. ...

Experience with non-relational database technologies (for example, MongoDB) and associated query patterns Familiarity with large-scale data processing and analytics platforms (for example, Apache Spark or Hadoop) Experience implementing, or partnering closely on, data quality, metadata, and governance controls to improve reliability and auditability Understanding of modern distributed ...

Analytics, Artificial Intelligence or a related field, or equivalent experience, would be effective, although practical experience is equally valuable. “Nice to Have” Azure, Databricks, Apache Spark or comparable cloud and data platforms. Data pipelines, ETL and workflow‐orchestration technologies. Machine‐learning methods such as forecasting, prediction or classification. Experience ...

experience with BigQuery . Experience building and supporting production data pipelines. Experience developing transformation layers using dbt . Experience with orchestration tools such as Apache Airflow or Cloud Composer . Strong SQL and Python development skills. Experience designing scalable data models and data warehouse solutions. Strong stakeholder engagement ...

help design and deliver cutting-edge data solutions for customers operating in some of the UK's most complex and challenging environments. Using Databricks, Apache Spark and cloud-native technologies, you’ll develop scalable Lakehouse architectures and production-grade data pipelines that enable organisations to trust, share and exploit … opportunity to grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data ...

colleagues Must-have skills: RabbitMQ Java and microservices MongoDB and MySQL Kubernetes Grafana and Prometheus GitLab Job scheduling tooling (eg JS7 Job Scheduler, TWS, Apache Airflow) Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects) Nice to have: Databases and data platforms, ideally Oracle or Cloudera ...

have skills: RabbitMQ Java/Microservices MongoDB/MySQL Kubernetes Grafana/Prometheus Gitlab Job Scheduling tooling (e.g. Berlin/JS7 Job Scheduler, TWS, Apache Airflow). Reporting and visualisation tooling (from Pentaho BA, Power BI, SAP Business Objects). Nice to have: Experience with databases/data platforms ...

happen; and if that’s you, roll up your sleeves and get stuck in! Our Signifyd stack Java, Cassandra, ElasticSearch, Redshift, Dynamo, MySQL, Apache Spark, Play! framework, gRPC, Linux, Docker, Kubernetes, AWS, GCP What support can you expect: We’ll provide you with a solid investment in your individual ...

analytical and operational domains.Establish ELT/ETL and streaming architectures on Snowflake Snowpipe, Snowpipe Streaming, Dynamic Tables, Streams and Tasks, external tables, and Apache Iceberg tables along with orchestration standards (dbt, Airflow, Snowflake Tasks) and CI/CD/DataOps best practices.Implement security-by-design in Snowflake: RBAC … Azure, or GCP) including Snowflake deployment, networking (Private Link), and cloud storage integration (S3, ADLS, GCS); familiarity with lakehouse stacks (Databricks, Delta Lake, Apache Iceberg) is a plus.Mastery of Snowflake security and governance: RBAC, tag-based governance, dynamic data masking, row access policies, column-level security, Tri-Secret encryption ...

should have specialist experience in one or more of the following technologies**Azure Databricks*** Design and build high-performance data pipelines: Utilize Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services.* Experience of Databricks ML and Azure … develop predictive models and drive business insights.* Proven expertise in Databricks, Apache Spark, and data pipeline development and strong understanding of data warehousing concepts and practices.* Experience with Microsoft Azure cloud platform, including Azure Data Lake Storage, Databricks and Azure Data Factory.* Azure Data Engineer Associate and Databricks Certified ...

configuration management and an interest in migrating to Ansible and cloud-native tooling. Experience managing Kubernetes platforms and deploying containerised applications. Knowledge of GitLab, Apache NiFi, HashiCorp Vault, Trend Micro Deep Security Manager, Ivanti device control, and Ansible Tower/AWX. Windows Infrastructure Experience with Windows Server administration, deployment ...

cloud platforms (AWS, Azure, GCP). Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow). Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

have: Around 3-5 years' commercial Python experience across data engineering, or data-driven software development Hands-on experience with technologies such as Kubernetes, Apache Airflow and Pandas Strong knowledge of data engineering, software engineering and production delivery Experience building and delivering data-driven applications or solutions A solid ...

etc.) Exposure to Angular, AlpineJS, Bootstrap Database tools: Toad, DBeaver, SQL Developer IDEs: VS Code, Eclipse, etc. Typescript TLS, PKI – certs etc. Keyfactor, Sectigo Apache config Containers (Docker, Podman, etc) Testing tools: Selenium, JUnit Data formats: XML, JSON, YAML AWS, Azure, Cloud etc. Login Authentication (SSO, MFA) CI/ ...

support comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, UnityCatalog, and Apache Atlas tomaintaindata integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining and optimising data … latency, such as managing 1 billion+ records or terabyte-sized databases. Knowledge of Delta TablesorApache Iceberg:Understanding and experience with Delta Tables or Apache Iceberg for managing large-scale data lakes efficiently. Knowledge of Data Governance Tools:Experience with data governance tools like Azure Purview, UnityCatalog, or Apache ...

model: Hybrid 3 days on site each week Rate: up to £530 per day --- Responsibilities Design, implement, and maintain high-throughput data pipelines using Apache Spark and cloud-native tools Develop ETL/ELT workflows for managing structured and unstructured data in AWS cloud environments Collaborate with AI/… analytics enablement --- Requirements (Must Have Skills) 5+ years of data engineering experience building high-performing and large-scale data solutions Hands-on expertise in Apache Spark development using Java or Scala (Python experience considered) Strong proficiency in AWS cloud services (S3, EMR, Glue, IAM, etc.) and cloud-native architecture ...

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great ...

effectively with stakeholders and champion better ways of working. Nice to Have Experience with Docker, Java/Scala, Kafka, Pulsar, Azure Event Hub, Collibra, Apache Iceberg, Apache Arrow, PySpark or AI engineering technologies such as Snowflake AI, LangChain, LlamaIndex or Pinecone would be advantageous. This is a great ...