151 to 175 of 345 Apache Jobs

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

Complex modelling and performance-aware DML and DDL across Snowflake/Clickhouse• Kubernetes and Docker• Kafka• Strong understanding of Git, code review, CI, TDD• Apache Airflow - desirable, but not essentialResponsibilities• Design, build and own bespoke low latency pipelines for our most demanding data sources.• On-call rota, owning detection ...

consumers Actively uses AI-powered tools to move faster and focus on architecture, system design, and scalability tradeoffs Nice to have: experience with Databricks, Apache Iceberg, Kafka, or similar lakehouse and streaming technologies Условия Hybrid work schedule with office attendance twice a week and flexible working hours Virtual Shares ...

latency software systems Experience with pipeline orchestration frameworks (e.g. Airflow, Dagster) Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databases Have a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experience For more information about ...

throughput, low-latency software systemsExperience with pipeline orchestration frameworks (e.g. Airflow, Dagster)Experience with streaming platforms (e.g. Kafka), data lake platforms (e.g. Delta Lake, Apache Iceberg), and relational databasesHave a Bachelors or advanced degree in Computer Science, Mathematics, Statistics, Physics, Engineering, or equivalent work experienceFor more information about ...

databasesExperience with one of Azure/AWS/GCPETL tools like Informatica, Alteryx and Business Glossary tools like Collibra, Azure Purview or Apache Atlas.TDD/BDD, Experience working in an Agile development environmentIdentity management and synchronisation will be an add on for the resourcesExperience with Data Dictionaries, Business glossaries ...

rather than “telling”, preferring to inspire stakeholders through built examples. Technical Expertise: Deep experience in core data & AI technologies, including languages (Python, SQL), frameworks (Apache Spark) and platforms (Databricks, etc.) Collaboration: You can orchestrate diverse teams to achieve a common goal and explain complex tech to non-technical leaders. ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits ...

data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache SparkTM, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook. Our Commitment to Diversity and Inclusion At Databricks ...

infrastructure. Well versed and proven experience in programming languages such as Python, Java, or C++. Experience with data pipeline and workflow management tools (e.g., Apache Airflow, Kubeflow). Strong problem-solving skills and ability to work in a fast-paced environment. Excellent communication and collaboration skills. Longstanding experience ...

pipelines, inference serving, and GenAI/RAG architectures grounded in proprietary content. Lakehouse and modern data platform design: strong knowledge of open table formats (Apache Iceberg, Delta Lake), zone architecture (landing, cleansed, curated), streaming and batch pipeline patterns, and data quality frameworks. Project engagement and design assurance: able ...

event-driven technologies, such as:Testing & Automation: PyTest, Cucumber, Behave, DBT, GreatExpectations, GCP DVT, Monte Carlo, Soda, Deequ, RestAssured etcData Engineering & Orchestration: BigQuery, Spanner, Apache Kafka, Airflow, Spark, Cloud Composer, DAGs, Apache Beam, Pub/Sub, Dataflow, DataStage, Teradata, Snowflake, ETL, SQL etcData Governance & Visualisation: Looker, Ataccama, Dataplex ...

with platform, analytics and engineering teams on data modelling, table design and storage optimization . Implement and work with modern lakehouse technologies such as Apache Iceberg/Delta patterns . Technical Skills & Experience Strong commercial experience in Data Engineering/Data Platform Engineering within an enterprise environment. Strong … . Strong hands-on experience with AWS , including: AWS Lambda S3/Data Lake EKS EMR PySpark MWAA IAM Lakehouse technologies Strong experience with Apache Iceberg . Experience with Snowflake , including query optimization and performance tuning. Experience building batch data pipelines and/or streaming solutions. Experience with Kafka ...

implement highly scalable and efficient data pipelines using advanced Python (Pandas, web scraping and machine learning libraries), SQL (including advanced optimisation techniques) and Apache Airflow.Lead the extraction, transformation and loading (ETL) of large and complex datasets, including time series data, from various sources (web APIs, databases, real-time streams … preferred), including data modelling and design skills.Extensive experience designing and building large-scale, efficient data pipelines for complex and time series data.Proven experience with Apache Airflow for data orchestration and scheduling at scale.Expertise in AWS cloud services (S3, Redshift, EC2, Lambda, etc.) is a must.Definition and design ...

platforms. This includes setting up monitoring and event management capabilities to alert for outage or service degradation. Experience with SQL Server and IIS/Apache administration is an asset Experience managing SaaS service health, vendor communications, and feature rollouts. Proficiency in PowerShell or equivalent scripting for automation tasks. Solid ...

enhancement programme. Investment Data Engineer, key skills: Strong commercial experience in Data Engineering, particularly using Snowflake Proven experience developing and maintaining workflows using Apache Airflow and DBT (Data Build Tool) Experience integrating and supporting Bloomberg data feeds, services or associated investment data solutions Strong SQL development and data modelling ...

enhancement programme. Investment Data Engineer, key skills: Strong commercial experience in Data Engineering, particularly using Snowflake Proven experience developing and maintaining workflows using Apache Airflow and DBT (Data Build Tool) Experience integrating and supporting Bloomberg data feeds, services or associated investment data solutions Strong SQL development and data modelling ...

Automation: GitLab CI, Terraform, Ansible, Helm Charts Scripting & APIs: Python, PowerShell, REST APIs Containerisation & Platforms: Kubernetes cluster build and application delivery Application Technologies: Apache NiFi, Elastic ECK, Artifactory Secrets Management: Hashicorp Vault Operating Systems: RedHat Linux and Windows Experience & Background Experience across the full software development lifecycle Strong understanding ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward Financial ...

configuration management) for fleet managementSolid Python skills — you'll be writing and maintaining production daemons, not just scriptsDeep Linux systems knowledge (Ubuntu) — comfortable with Apache/Nginx, PHP-FPM, Varnish, systemd, filesystem mounts, and networking fundamentalsUnderstanding of distributed systems concepts: consensus, leader election, distributed locking, eventual consistency ...

crypto exchanges, trading venues, protocols, or market infrastructure. Experience developing collateral, position, accounting, market data, or reference data systems. Experience with Python. Experience with Apache, Kafka or other event streaming technologies. Experience using AI-assisted software development tools. Understanding of traditional financial instruments and financial market infrastructure. Our benefits ...

manage multiple projects and stakeholders Functional and object-oriented programming skills with expertise in Scala, Java, and Python, and open-source frameworks such as Apache Spark Familiarity with software engineering and project management tools such as Git, Jira, and Agile Experience leading adoption of agentic AI-enabled engineering practices ...

with external clients across a variety of industry markets. Minimum of 5+ years of customer-facing experience would be preferred[Preferred] Experience working with Apache Spark to process large-scale distributed datasetsCommunication: Proven ability to communicate and teach complex technical concepts to both technical and non-technical audiences.Core Traits ...

driving our values through ML[Preferred] 2+ years customer-facing experience in a pre-sales or post-sales role[Preferred] Experience working with Apache Spark to process large-scale distributed datasetsCan meet expectations for technical training and role-specific outcomes within 3 months of hireCan travel ...

procedural and object orientated software development languages (e.g. Ada, C, C++ & Java)Knowledge of scripting languages (e.g. Python)Knowledge of COTS integration technologies (e.g. Apache)Working with large-scale and complex integration programmesKnowledge of NoMagic Cameo/IBM DOORS, Atlassian toolset, TestRail or equivalent toolsATM knowledge would be advantageous ...