11 of 11 Parquet Jobs in the South West

ideally Microsoft Azure. Experience using technologies such as Spark, Databricks and data lake architectures. Knowledge of data storage and transformation formats such as Apache Parquet and Delta Lake. Experience with containerisation and orchestration technologies, including Docker and Kubernetes. Experience using Git/GitHub and working within collaborative development environments. ...

data orchestration across multiple sources and systems. Experience designing data models, schemas, data contracts and integration frameworks. Strong understanding of REST API integration, Parquet data formats and S3-compatible object storage. Experience deploying or supporting data platforms within OpenShift or Kubernetes environments. Knowledge of GitLab CI/ ...

manage environments Implementing logging, monitoring and alerting across production systems Working with technologies such as Pandas, Pydantic, Airflow, AWS Glue, Docker, JSON and Parquet Working alongside data scientists, engineers and multidisciplinary delivery teams Supporting and mentoring other engineers and sharing technical knowledge WHAT WE'RE LOOKING ...

building lean processes Experience with distributed data processing Experience with monitoring, testing, and data quality Experience building robust pipelines from diverse sources e.g. parquet, SQL & no-SQL databases, API endpoints It would be helpful to have experience/expertise/knowledge in the following: Python AWS Kubernetes Spark & distributed ...

modern data stack, building data ingestion pipelines and working with ETL and orchestration tools (e.g., Airflow, Luigi, Argo, dbt), big data technologies (Spark, Kafka, Parquet), and web frameworks for model serving (e.g. Flask or FastAPI). Data Science: Familiarity or experience with classical NLP techniques (BERT, topic modelling, summarisation ...

PostgreSQL, and cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing ...

relational database knowledge, including data extraction and quality management Experience working with Data Warehouse and ETL processes, alongside formats such as CSV, JSON and Parquet Excellent stakeholder management skills with the ability to influence decision-making at all levels of the business Successful candidates must be able to obtain ...

data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage. Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose. Develop … routing, transformation and orchestration. Experience of data modelling and schema design, including defining data contracts or Interface Control Documents (ICDs). Experience working with Parquet and S3-compatible object storage. Experience integrating REST APIs and troubleshooting data exchanges between systems. Experience deploying or supporting data solutions within OpenShift ...

Description We re looking for a Data Engineer with a strong background in bioinformatics and genetics to help build the data pipelines supporting Our Future Health s growing Clinical Research Recruitment Service. This is an ...

data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage. Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose. Develop … Strong Python and SQL skills. Hands-on experience with Apache NiFi and data pipeline orchestration. Experience with data modelling and schema design. Experience with Parquet and S3-compatible object storage. Experience with REST API integration. Experience with OpenShift or Kubernetes. Experience with GitLab CI/CD practices and pipelines. ...

data contracts/Interface Control Documents (ICDs) to support reliable integration between systems. Integrate data from REST APIs and other source systems, working with Parquet and S3-compatible object storage. Implement data validation, assurance, metadata management and traceability to ensure data is accurate, complete and fit for purpose. Develop … Strong Python and SQL skills. Hands-on experience with Apache NiFi and data pipeline orchestration. Experience with data modelling and schema design. Experience with Parquet and S3-compatible object storage. Experience with REST API integration. Experience with OpenShift or Kubernetes. Experience with GitLab CI/CD practices and pipelines. ...