51 to 75 of 416 Apache Jobs in the UK

Oracle databases. Experience designing and supporting ETL/ELT pipelines and data quality frameworks (rules, thresholds, exception handling). Working knowledge of Autosys and Apache Airflow (monitoring schedules, reruns, failure triage). Experience with CI/CD and release controls (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift ...

skills. Ability to think creatively and propose innovative solutions. To qualify for this role, you must have: Proven experience and knowledge with PySpark and Apache Spark including the fundamentals of how it works. Experience with cloud platforms (e.g., AWS, Azure, GCP) and data lake architectures. Strong experience and programming ...

similar data-focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands-on experience with Apache Spark, Databricks, dbt and cloud-based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with ...

data pipeline development, ETL/ELT processes, and data warehousing solutions. Proficient with cloud platforms (AWS, GCP, Azure) and big data tools like Apache Spark, Python, etc. Experience with relational and NoSQL databases. Solid understanding of data modeling, data architecture, engineering frameworksProven ability to lead and scale engineering teams ...

with data transformation tools such as dbt, or Dataflow. Understanding of data modeling techniques. Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset. Understanding of software engineering best practices and their application to data processing systems. Knowledge of Python, Java or Flink is a plus ...

with data transformation tools such as dbt, or Dataflow. Understanding of data modeling techniques. Experience with using visualisation platforms such as Looker, Tableau, or Apache Superset. Understanding of software engineering best practices and their application to data processing systems. Knowledge of Python, Java or Flink is a plus ...

similar data‐focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands‐on experience with Apache Spark, Databricks, dbt and cloud‐based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with ...

Databricks REST APIs, Prometheus, or Azure Monitor. Maintain best practices for data engineering performance, testing, and documentation. Technical Skills Required Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala), and Delta Lake. Deep understanding of Spark execution plans, shuffle optimization, and data partitioning strategies. Experience tuning job configurations (e.g. ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

Microsoft Azure, with practical exposure to the Microsoft Fabric ecosystem (OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python ...

Computer Science or equivalent practical experience Experience developing finance applications is a plus Working experience with Terraform is a plus Experience creating workflows for Apache Airflow is a plus #LI-JB6 What's Roku's approach to hybrid working? Roku fosters an inclusive and collaborative environment where teams generally ...

knowledge of optimisation techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

United Kingdom A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical and engineering skills to deliver trusted, well … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

scripts, with a particular focus on building and optimizing multi-threaded, multi-process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI-powered solutions using Large Language Models (LLMs), retrieval-augmented ...

Skills MUST HAVE: Strong hands-on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud ...

Skills MUST HAVE: Strong hands‐on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong cloud ...

skills; experience with Oracle databases . Experience designing and supporting ETL/ELT pipelines and data quality frameworks . Working knowledge of Autosys& Apache Airflow Experience with CI/CD tools (Git, Harness, UrbanCode Deploy (UCD), Red Hat OpenShift) Familiarity with AWS S3 for large-scale data storage. Experience ...

hands-on familiarity with vessel-tracking data, its structure, quirks, and limitations. SQL— strong proficiency, including query optimization on large datasets. Nice-to-Haves Apache Airflow (or equivalent orchestration) for scheduling and managing pipelines. AWS experience (Azure or GCP equally welcome). Machine Learning experience, particularly applied to geospatial ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

decisions, documentation and continuous improvement initiatives. Key Skills & Experience Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. ...

Software Engineer on large software applications. Proficient in technologies including Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, and Apache Spark. Experience working with one or more of the following database systems: DynamoDB, DocumentDB, MongoDB. Demonstrated expertise in unit testing and tools such as JUnit ...

with monitoring/logging tools (Prometheus, Grafana, ELK) and root cause analysis Knowledge of storage (NFS/EFS), databases (PostgreSQL), and web servers (Nginx, Apache, Tomcat) Experience with SSO and security (SAML, OIDC, Azure AD, Okta, Keycloak) Scripting skills in Python and/or Bash Profile Client-focused with ...

build tools (Maven/Gradle). Desirable skills: Scala: Hands-on experience or a strong functional programming background using Scala. Big Data: Exposure to Apache Spark, Kafka, or Hadoop ecosystems. Cloud: Experience with AWS, Azure, or Google Cloud Platform. Experience in banking or financial services industry. Soft Skills Strong ...

scale data pipelines, with a strong grasp of data structuring for performance and efficiencyHands-on experience with Airflow for orchestrating complex data workflows, and Apache Iceberg for table format management at scaleExtensive AWS experience including EKS, Docker, and Kubernetes; comfortable owning infrastructure deployment and scaling decisionsCI/CD fluency ...

hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...