1 to 25 of 35 Parquet Jobs

sFTP protocols. ETL/ELT Pipelines : Design and optimize data pipelines using Azure Data Factory and Databricks Medallion Architecture : Implement Bronze, Silver, and Gold layers using formats like Delta , Parquet , and JSON for data transformation. Data Modeling : Develop and optimize data models using star schema and slowly changing dimensions for analytics and operations. Data Governance : Ensure robust data security … Azure Data Engineer. Technical Expertise : Proficiency with Azure Data Factory , Databricks and Azure Storage . Strong skills in SQL , Python , and data modeling techniques. Familiarity with data formats like Parquet and JSON. Experience with AI/ML model management on Azure Databricks . Education : Bachelor's degree in IT, Computer Science, or a related field. Microsoft Certified: Azure Data More ❯

science, software engineering, databases, and distributed/parallel processing frameworks to prepare big data for the use of data analysts and data scientists. If you have experience with Apache Parquet, Apache Spark, AWS Glue, AWS Athena, Databricks and want your work to contribute to systems that collect healthcare data used by hundreds of thousands of daily users, we want More ❯

or equivalent experience, coupled with relevant professional certifications . Advanced SQL knowledge for database querying. Proficiency with big data tools (Hadoop, Spark) and familiarity with big data file formats (Parquet, Avro). Skilled in data pipeline and workflow management tools (Apache Airflow, NiFi). Strong background in programming (Python, Scala, Java) for data pipeline and algorithm development. Skilled in More ❯

tools (QuickSight, Power BI, Tableau, Looker, etc.) Interest or experience in building internal data communities or enablement programs Working with diverse data sources (APIs, CRMs, SFTP, databases) and formats (Parquet, JSON, XML, CSV) Exposure to machine learning models or AI agents Why Join Us Help shape the future of data in an organization that treats data as a product More ❯

All our office locations considered: Newbury & Liverpool (UK); Šibenik, Croatia (considered) We're on the hunt for builders . No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems More ❯

with SQL databases (PostgreSQL, Oracle, SQL Server) Knowledge of big data technologies (Hadoop, Spark, Kafka) Familiarity with cloud platforms and containerization (Docker, Kubernetes) Understanding of data formats (JSON, XML, Parquet, Avro) Professional Experience Bachelor's degree in Computer Science, Engineering, or related field 5+ years of experience in data engineering or related roles Experience working in classified or high More ❯

Minimum of 3 years' experience with Python. Demonstrated experience building & orchestrating automated, production-level data pipelines and solutions (ETL/ELT). Experience with file-based data storage, including Parquet or Iceberg. Experience with data catalogs (ex. Hive, AWS Glue). General understanding of key AWS services (e.g. EC2, S3, EKS, IAM, lambda). Experience building and/or More ❯

Data modelling (building optimised and efficient data marts and warehouses in the cloud) Work with Infrastructure as code (Terraform) and containerising applications (Docker) Work with AWS, S3, SQS, Iceberg, Parquet, Glue and EMR for our Data Lake Experience developing CI/CD pipelines More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy More ❯

AWS serverless services and enables powerful querying and analytics through Amazon Athena. In this role, you'll work on a system that combines streaming ingestion (Firehose), data lake technologies (Parquet, Apache Iceberg), scalable storage (S3), event-driven processing (Lambda, EventBridge), fast access databases (DynamoDB), and robust APIs (Spring Boot microservices on EC2). Your role will involve designing, implementing … processing pipeline and platform services. Key Responsibilities: Design, build, and maintain serverless data processing pipelines using AWS Lambda, Firehose, S3, and Athena. Optimize data storage and querying performance using Parquet and Iceberg formats. Manage and scale event-driven workflows using EventBridge and Lambda. Work with DynamoDB for fast, scalable key-value storage. Develop and maintain Java Spring Boot microservices … Java backend development experience. 3+ years of Python development. Strong hands-on experience with AWS services: Lambda, S3, K8S. Deep understanding of data lake architectures and formats such as Parquet and Iceberg. Proficiency in Spring Boot and working experience with microservices. Experience with high-scale, event-driven systems and serverless patterns. Nice to Have: Solid understanding of distributed systems More ❯

hands-on AWS experience – S3, Redshift, Glue essential. Proven experience building ETL/ELT pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau, Looker, etc.). Strong scripting and Linux/ More ❯

hands-on AWS experience – S3, Redshift, Glue essential. Proven experience building ETL/ELT pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau, Looker, etc.). Strong scripting and Linux/ More ❯

processing and automation Solid understanding of ETL/ELT workflows, data modelling, and structuring datasets for analytics Experience working with large, complex datasets and APIs across formats (CSV, JSON, Parquet, etc) Familiarity with workflow automation tools (eg, Power Automate) and/or Power Apps is desirable Excellent interpersonal and communication skills with the ability to work cross-functionally and More ❯

processing and automation Solid understanding of ETL/ELT workflows, data modelling, and structuring datasets for analytics Experience working with large, complex datasets and APIs across formats (CSV, JSON, Parquet, etc) Familiarity with workflow automation tools (eg, Power Automate) and/or Power Apps is desirable Excellent interpersonal and communication skills with the ability to work cross-functionally and More ❯

processing and automation Solid understanding of ETL/ELT workflows, data modelling, and structuring datasets for analytics Experience working with large, complex datasets and APIs across formats (CSV, JSON, Parquet, etc) Familiarity with workflow automation tools (eg, Power Automate) and/or Power Apps is desirable Excellent interpersonal and communication skills with the ability to work cross-functionally and More ❯

tools such as Spark, NiFi, Kafka, Flink, or at multi-petabyte scale Experience in designing and maintaining ETL or ELT data pipelines utilizing storage, serialization formats, schemas, such as Parquet and Avro Experience administrating and maintaining data science workspaces and tool benches for Data Scientists and Analysts Secret clearance HS diploma or GED Nice If You Have: Experience deploying More ❯

data warehousing (e.g. Hadoop, Spark, Redshift, Snowflake, GCP BigQuery) Expertise in building data architectures that support batch and streaming paradigms Experience with standards such as JSON, XML, YAML, Avro, Parquet Strong communication skills Open to learning new technologies, methodologies, and skills As the successful Data Engineering Manager you will be responsible for: Building and maintaining data pipelines Identifying and More ❯

including GitLab CI Experience with metrics and monitoring tools such as Prometheus and Grafana Experience with structured and semi-structured data formats, including JSON, XML, CSV, TCLD, ASDF, and Parquet, for data parsing and transformation tasks Experience with Git for source code management Desired Skills Experience with the Atlassian suite of tools including Confluence and Jira Experience with Big More ❯

presenting results/metrics across a range of technologies Linux (RedHat/Rocky, Alpine and distroless containers) Python (pandas/pyspark), bash, SQL, Terraform, data/file formats (json, parquet, etc.) Spark, Kafka, ELK, MinIO/Ceph, Nifi, Airflow Gitlab, Jira, Confluence AWS DESIRED QUALIFICATIONS: Familiarity with DoD and/or cyber security operations desired CLEARANCE: Secret minimum More ❯

Experience with alternative data technologies (e.g. duckdb, polars, daft). Familiarity with eventing technologies (Event Hubs, Kafka etc ). Deep understanding of file formats and their behaviour such as parquet, delta and iceberg. What we offer We want to give you a great work environment; contribute back to both your personal and professional development; and give you great benefits More ❯

software, libraries, and packages involving stream/batch data processing and analytic frameworks Experience with data parsing/transformation technologies and file formats including JSON, XMl, CSV, TCLD, and Parquet General Cloud and HPC knowledge regarding computer, networking, memory, and storage components Experience with Linux administration including software integration, configuration management and routine O&M operations related to provisioning More ❯

with REST APIs Experience with Java Experience with full lifecycle agile software development projects Desired skills: Experience with Python. Experience building data products in Apache Avro and/or Parquet On-the-job experience with Java software development. Experience deploying the complete DevOps Lifecycle including integration of build pipelines, automated deployments, and compliance scanning using test driven development. Responsibilities More ❯

With solid software engineering fundamentals, fluent in Java and Python (Rust is a plus). Knowledgeable about data lake systems like Athena, and big data storage formats such as Parquet, HDF5, ORC, focusing on data ingestion. Driven by working in an intellectually engaging environment with top industry minds, where constructive debates are encouraged. Excited about working in a start More ❯

About the role Taktile is a high-growth, post product-market-fit start-up, on a fast trajectory to becoming market leader in the field of automated decisioning. We are looking for a Full-stack Engineer to join the Decide More ❯

collection efforts. Strategically apply AI/ML to extract, format, and expose in indexed search tools relevant content such as raw text, multimedia (audio, image, video, document), tabular (CSV, Parquet, Avro) or nested (JSON, JSONL, XML), and other structured/unstructured data types. Data is expected to be of varying formats, schemas, and structures. Provide Data Engineering support to More ❯

Bonus Points For Workflow orchestration tools like Airflow. Working knowledge of Kafka and Kafka Connect. Experience with Delta Lake and lakehouse architectures. Proficiency in data serialization formats: JSON, XML, PARQUET, YAML. Cloud-based data services experience. Ready to build the future of data? If you're a collaborative, forward-thinking engineer who wants to work on meaningful, complex problems More ❯