26 to 50 of 479 Permanent Apache Jobs

stack development for cloud solutions (Azure or AWS) - Azure preferred Skills and Knowledge The full stack engineers for Commercial Technology will be proficient in C# (.NET Core), Visual Studio, Apache Web Services, XML, and Restful-API/Kafka & event-driven integration capabilities. Must have development experiences in Azure Integration Services (Logic Apps, Azure Functions, Azure Event Hub, Azure Redis More ❯

stack development for cloud solutions (Azure or AWS) - Azure preferred Skills and Knowledge The full stack engineers for Commercial Technology will be proficient in C# (.NET Core), Visual Studio, Apache Web Services, XML, and Restful-API/Kafka & event-driven integration capabilities. Must have development experiences in Azure Integration Services (Logic Apps, Azure Functions, Azure Event Hub, Azure Redis More ❯

In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and scalability. If you have a passion for data engineering and a … significant impact, we encourage you to apply! Job Responsibilities ETL/ELT Pipeline Development: Design, develop, and optimize efficient and scalable ETL/ELT pipelines using Python, PySpark, and Apache Airflow. Implement batch and real-time data processing solutions using Apache Spark. Ensure data quality, governance, and security throughout the data lifecycle. Cloud Data Engineering: Manage and optimize … effectiveness. Implement and maintain CI/CD pipelines for data workflows to ensure smooth and reliable deployments. Big Data & Analytics: Develop and optimize large-scale data processing pipelines using Apache Spark and PySpark. Implement data partitioning, caching, and performance tuning techniques to enhance Spark-based workloads. Work with diverse data formats (structured and unstructured) to support advanced analytics and More ❯

modern technology stacks to build and optimize a powerful data platform and search engine. With an opportunity to explore vector search, machine learning, and large-scale data processing using Apache Lucene, Solr, or Elasticsearch. What you’ll be doing: Design, build, and optimize a high-performance data platform and search solution. Develop robust search capabilities using Apache Lucene … and search technologies. Role Profile You have strong experience in Java development and exposure to Python. Have experience with large-scale data processing and search technologies. An expert in Apache Lucene, Solr, Elasticsearch, if not you have the appetite to learn more. Hands on experience with SQL and NoSQL databases under your belt. Hold a degree in Computer Science More ❯

systems, with a focus on data quality and reliability. Design and manage data storage solutions, including databases, warehouses, and lakes. Leverage cloud-native services and distributed processing tools (e.g., Apache Flink, AWS Batch) to support large-scale data workloads. Operations & Tooling Monitor, troubleshoot, and optimize data pipelines to ensure performance and cost efficiency. Implement data governance, access controls, and … ELT pipelines and data architectures. Hands-on expertise with cloud platforms (e.g., AWS) and cloud-native data services. Comfortable with big data tools and distributed processing frameworks such as Apache Flink or AWS Batch. Strong understanding of data governance, security, and best practices for data quality. Effective communicator with the ability to work across technical and non-technical teams. … Additional Strengths Experience with orchestration tools like Apache Airflow. Knowledge of real-time data processing and event-driven architectures. Familiarity with observability tools and anomaly detection for production systems. Exposure to data visualization platforms such as Tableau or Looker. Relevant cloud or data engineering certifications. What we offer: A collaborative and transparent company culture founded on Integrity, Innovation and More ❯

Role, You'll: • Design, build, and maintain secure, scalable data pipelines and services • Ingest, transform, and model structured and unstructured data for analytics and ML • Work with technologies like Apache Spark, Apache Iceberg, Trino, NiFi, OpenSearch, and AWS EMR • Ensure data integrity, lineage, and security across the entire lifecycle • Collaborate with DevOps to deploy containerized data solutions using More ❯

engineers + external partners) across complex data and cloud engineering projects Designing and delivering distributed solutions on an AWS-centric stack, with open-source flexibility Working with Databricks, Apache Iceberg, and Kubernetes in a cloud-agnostic environment Guiding architecture and implementation of large-scale data pipelines for structured and unstructured data Steering direction on software stack, best practices, and … especially AWS), and orchestration technologies Proven delivery of big data solutions-not necessarily at FAANG scale, but managing high-volume, complex data (structured/unstructured) Experience working with Databricks, Apache Iceberg, or similar modern data platforms Experience of building software environments from the ground up, setting best practice and standards Experience leading and mentoring teams Worked in a startup More ❯

ad hoc to scalable, production-grade systems - Collaborate closely with senior stakeholders and shape the future of the data platform - Python | SQL | Snowflake | AWS (S3, Docker, Terraform) | Airflow | DBT | Apache Spark & Iceberg | PostgresDB The ideal candidate: - 5+ years as a senior/principal data engineer - Experience leading or mentoring engineers - Strong technical grounding - Start-up/scale-up mindset More ❯

patterns in pipeline architecture and design. Confident using Git-based version control systems, including Azure DevOps or similar. Skilled in managing and scheduling data workflows using orchestration platforms like Apache AirFlow. Involved in building and optimizing data warehouses on modern analytics platforms like Snowflake, Redshift, or Databricks. Familiar with visual or low-code data integrations tools, including platforms such More ❯

pipelines Hands-on experience with Agile (Scrum) methodologies Database experience with Oracle and/or MongoDB Experience using the Atlassian suite : Bitbucket, Jira, and Confluence Desirable Skills Knowledge of Apache NiFi Front-end development with React (JavaScript/TypeScript) Working knowledge of Elasticsearch and Kibana Experience developing for cloud environments, particularly AWS (EC2, EKS, Fargate, IAM, S3, Lambda) Understanding More ❯

Python Strong experience developing on Linux Version control using Git Agile development (SCRUM) Working with both relational databases (Oracle) and NoSQL (MongoDB) Experience with GitLab CI/CD Pipelines , Apache NiFi , and Atlassian tools (Jira, Bitbucket, Confluence) Front-end skills: JavaScript/TypeScript, React Search and analytics tools: Elasticsearch, Kibana Nice to Have: Experience developing for AWS Cloud (EC2 More ❯

Python Strong experience developing on Linux Version control using Git Agile development (SCRUM) Working with both relational databases (Oracle) and NoSQL (MongoDB) Experience with GitLab CI/CD Pipelines , Apache NiFi , and Atlassian tools (Jira, Bitbucket, Confluence) Front-end skills: JavaScript/TypeScript, React Search and analytics tools: Elasticsearch, Kibana Nice to Have: Experience developing for AWS Cloud (EC2 More ❯

data into a data platform using Fivetran. Experience of developing BI dashboards using Power BI. Knowledge of security concepts relevant to Azure. Experience of workflow management tools such as Apache Airflow. Interested in the role? Complete the online application. We look forward to getting to know you. Discover more about LGT Wealth Management A message from our CEO Ben More ❯

with multiple languages • Technologies: Scala, Java, Python, Spark, Linux and shell scripting, TDD (JUnit), build tools (Maven/Gradle/Ant) • Experience in working with process scheduling platforms like Apache Airflow. • Should be ready to work in GS proprietary technology like Slang/SECDB • An understanding of compute resources and the ability to interpret performance metrics (e.g., CPU, memory More ❯

data ecosystem (e.g., Pandas, NumPy) and deep expertise in SQL for building robust data extraction, transformation, and analysis pipelines. Hands-on experience with big data processing frameworks such as Apache Spark, Databricks, or Snowflake, with a focus on scalability and performance optimization Familiarity with graph databases (e.g., Neo4j, Memgraph) or search platforms (e.g., Elasticsearch, OpenSearch) to support complex data More ❯

Proficiency in one or more programming languages including Java, Python, Scala or Golang. Experience with columnar, analytical cloud data warehouses (e.g., BigQuery, Snowflake, Redshift) and data processing frameworks like Apache Spark is essential. Experience with cloud platforms like AWS, Azure, or Google Cloud. Strong proficiency in designing, developing, and deploying microservices architecture, with a deep understanding of inter-service More ❯

serverless services and enables powerful querying and analytics through Amazon Athena. In this role, you'll work on a system that combines streaming ingestion (Firehose), data lake technologies (Parquet, Apache Iceberg), scalable storage (S3), event-driven processing (Lambda, EventBridge), fast access databases (DynamoDB), and robust APIs (Spring Boot microservices on EC2). Your role will involve designing, implementing, and More ❯

field. Technical Skills Required Hands-on software development experience with Python and experience with modern software development and release engineering practices (e.g. TDD, CI/CD). Experience with Apache Spark or any other distributed data programming frameworks. Comfortable writing efficient SQL and debugging on cloud warehouses like Databricks SQL or Snowflake. Experience with cloud infrastructure like AWS or More ❯

Azure Storage, Medallion Architecture, and working with data formats such as JSON, CSV, and Parquet. • Strong understanding of IT concepts, including security, IAM, Key Vault, and networking. • Exposure to Apache Airflow and DBT is a bonus. • Familiarity with agile principles and practices. • Experience with Azure DevOps pipelines. The "Nice to Haves": • Certification in Azure or related technologies. • Experience with More ❯

OCI Images), GitHub Actions, Gradle, Jenkins (legacy, moving towards GitHub Actions), Maven, SonarCloud Data : Elasticsearch, MongoDB, MySQL, Neo4J IaC : Ansible, Terraform Languages : Java, Python, TypeScript Monitoring : Grafana, Prometheus Misc : Apache (legacy, moving towards AWS CloudFront/API Gateway), Git (GitHub), Linux (Ubuntu), RabbitMQ We are looking to start or make more use of the following AWS services : CloudTrail Secrets More ❯

MySQL Exposure to Docker, Kubernetes, AWS, Helm, Terraform, Vault, Grafana, ELK Stack, New Relic Relevant experience in the maintenance of data APIs and data lake architectures, including experience with Apache Iceberg, Trino/Presto, Clickhouse, Snowflake, BigQuery. Master's degree in Computer Science or Engineering-related field Get to know us better YouGov is a global online research company More ❯

as required; comfortable building multi page Web Applications from scratch. Expertise with Application Server integration; JBoss 7, SpringBoot or later preferred. Proficient in developing microservices with SpringBoot Knowledge of Apache Web Server preferred. Database Skills with working knowledge of Structured Query Language (e.g. SQL/NoSQL commands and queries). 2+ years Working with Oracle, MySQL, MS SQL and … as required; comfortable building multi page Web Applications from scratch. Expertise with Application Server integration; JBoss 7, SpringBoot or later preferred. Proficient in developing microservices with SpringBoot Knowledge of Apache Web Server preferred. Database Skills with working knowledge of Structured Query Language (e.g. SQL/NoSQL commands and queries). 2+ years Working with Oracle, MySQL, MS SQL and More ❯

Manage deployments with Helm and configuration in YAML. Develop shell scripts and automation for deployment and operational workflows. Work with Data Engineering to integrate and manage data workflows using Apache Airflow and DAG-based models. Perform comprehensive testing, debugging, and optimization of backend components. Required Skills Bachelor's degree in Computer Science, Software Engineering, or a related field (or … and YAML for defining deployment configurations and managing releases. Proficiency in shell scripting for automating deployment and maintenance tasks. Understanding of DAG (Directed Acyclic Graph) models and experience with Apache Airflow for managing complex data processing workflows. Familiarity with database systems (SQL and NoSQL) and proficiency in writing efficient queries. Solid understanding of software development best practices, including version More ❯

Manage deployments with Helm and configuration in YAML. Develop shell scripts and automation for deployment and operational workflows. Work with Data Engineering to integrate and manage data workflows using Apache Airflow and DAG-based models. Perform comprehensive testing, debugging, and optimization of backend components. Required Skills Bachelor's degree in Computer Science, Software Engineering, or a related field (or … and YAML for defining deployment configurations and managing releases. Proficiency in shell scripting for automating deployment and maintenance tasks. Understanding of DAG (Directed Acyclic Graph) models and experience with Apache Airflow for managing complex data processing workflows. Familiarity with database systems (SQL and NoSQL) and proficiency in writing efficient queries. Solid understanding of software development best practices, including version More ❯

scripting) 3 + years of experience working with AWS Cloud, including S3, Redshift, Glue, Lambda, Step Functions, etc. 5 + years of experience with workflow orchestration tools such as Apache Airflow, Control-M, Maestro (Dynamic Workload console), or others Due to the nature of this position, as part of our background check process, candidates must be able to pass More ❯