1 to 25 of 29 Apache Jobs in Yorkshire

approach, that converges our product capabilities and opening new possibilities using machine learning and artificial intelligence. This approach encompasses a multitude of vendors (Microsoft, AWS, ESRI, Snowflake, IBM and Apache Software Foundation) and services (Azure, Microsoft Fabric, Apache Airflow, dbt Cloud, watsonx, Kafka, Kubernetes (EKS and AKS), Red Hat Open Shift, plus others.) thus ensuring we can leverage … core object-oriented concepts with good knowledge of Java, Web services, JavaScript, Json, XML, JDBC, and other related technologies. Experience in data and orchestration tools including some of dbt, Apache Airflow, Azure Data Factory. Experience with unit testing in a variety of languages and tools. Perhaps experience of Junit, NUnit, XUnit and Jest, along with experience of associated tools More ❯

production issues. Optimize applications for performance and responsiveness. Stay Up to Date with Technology: Keep yourself and the team updated on the latest Python technologies, frameworks, and tools like Apache Spark , Databricks , Apache Pulsar , Apache Airflow , Temporal , and Apache Flink , sharing knowledge and suggesting improvements. Documentation: Contribute to clear and concise documentation for software, processes, and … cloud platforms like AWS , GCP , or Azure . DevOps Tools: Familiarity with containerization ( Docker ) and infrastructure automation tools like Terraform or Ansible . Real-time Data Streaming: Experience with Apache Pulsar or similar systems for real-time messaging and stream processing is a plus. Data Engineering: Experience with Apache Spark , Databricks , or similar big data platforms for processing … large datasets, building data pipelines, and machine learning workflows. Workflow Orchestration: Familiarity with tools like Apache Airflow or Temporal for managing workflows and scheduling jobs in distributed systems. Stream Processing: Experience with Apache Flink or other stream processing frameworks is a plus. Desired Skills: Asynchronous Programming: Familiarity with asynchronous programming tools like Celery or asyncio . Frontend Knowledge More ❯

such as AWS, Azure, or GCP · Docker · Documenting code, architectures, and experiments · Linux systems and bash terminals Preferred Qualifications o Databases and/or data warehousing technologies, such as Apache Hive, Iceberg etc. o Data transformation via SQL and DBT. o Orchestration platforms such as Apache Airflow, Argo Workflows, etc. o Data catalogs and metadata management tools o More ❯

processes, including prompt logic, model usage, and data governance considerations, promoting transparency and responsible AI use. 7. Automate ETL pipeline orchestration and data processing workflows: Leverage orchestration tools like Apache Airflow, Perfect to schedule, automate, and manage ETL jobs, reducing manual intervention and improving operational reliability. 8. Implement monitoring, alerting, and troubleshooting for data workflows: Set up real-time More ❯

Proven desire to expand your cloud/platform engineering capabilities Experience working with Big Data Experience of data storage technologies: Delta Lake, Iceberg, Hudi Proven knowledge and understanding of Apache Spark, Databricks or Hadoop Ability to take business requirements and translate these into tech specifications Competence in evaluating and selecting development tools and technologies Sound like the role you More ❯

best practices for data security and compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse and ETL/ELT development in AWS More ❯

best practices for data security and compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse and ETL/ELT development in AWS More ❯

or in a similar role. Strong proficiency in SQL and experience with database management systems (e.g., MySQL, PostgreSQL, MongoDB). Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Luigi). Familiarity with cloud platforms and services and a particular knowledge of Google Cloud would be preferable. Proficiency in programming languages such as Python, Java, or Scala. More ❯

Senior Data Engineer/Data Analytics Engineer - Full Remote Working from anywhere in the UK SQL+Python+ETL+ Apache Airflow Our client is a global leading and fast growing high tech company: -Over 6,500 employees across 20+ offices; -300 Million+ active users on some of the platforms they developed; -Cutting edge AR and VR technologies, 3D printing, etc. They are … will develop scalable data pipelines, ensure data quality, and support business decision-making with high-quality datasets. -Work across technology stack: SQL, Python, ETL, Big Query, Spark, Hadoop, Git, Apache Airflow, Data Architecture, Data Warehousing -Design and develop scalable ETL pipelines to automate data processes and optimize delivery -Implement and manage data warehousing solutions, ensuring data integrity through rigorous … testing and validation -Lead, plan and execute workflow migration and data orchestration using Apache Airflow -Focus on data engineering and data analytics Requirements: -5+ years of experience in SQL -5+ years of development in Python -MUST have strong experience in Apache Airflow -Experience with ETL tools, data architecture, and data warehousing solutions -Strong communication skills This More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or informal More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or informal More ❯

Engineering (open to professionals from various data eng. backgrounds - data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience - formal or informal More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or informal More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or informal More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or informal More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or informal More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

or Informatica Cloud Solid understanding of data extraction, transformation, and loading (ETL) processes Proficiency in SSH and managing/troubleshooting virtual machine environments Familiarity with orchestration tools such as Apache Airflow Experience with AWS services (e.g., S3, EC2, RDS) Understanding of cloud-based environments (Google Cloud Platform experience a plus) Ability to collaborate across infrastructure, data engineering, and business More ❯

Azure etc.) What You'll Need Must-haves: Comprehensive experience in a DevOps or SRE role, ideally in a multi-project environment Deep experience with web stacks: Nginx/Apache, PHP-FPM, MySQL, Redis, Varnish, Elasticsearch Proven expertise in managing and optimising Cloudflare across DNS, security, performance, and access Experience with Magento 2 infrastructure and deployment CI/CD More ❯

common life sciences data acquisition software, such as Scientific Data Management Systems (SDMS) or Laboratory Information Management Systems (LIMS). Hands-on experience with data pipeline orchestration tools (e.g., Apache Airflow) and data parsing. Familiarity with cloud service models, SaaS infrastructure, and related SDLC. Familiarity with containerization and container orchestration tools (e.g., Docker, Kubernetes). ZONTAL is an Equal More ❯