1 to 25 of 27 Permanent Parquet Jobs

sFTP protocols. ETL/ELT Pipelines : Design and optimize data pipelines using Azure Data Factory and Databricks Medallion Architecture : Implement Bronze, Silver, and Gold layers using formats like Delta , Parquet , and JSON for data transformation. Data Modeling : Develop and optimize data models using star schema and slowly changing dimensions for analytics and operations. Data Governance : Ensure robust data security … Azure Data Engineer. Technical Expertise : Proficiency with Azure Data Factory , Databricks and Azure Storage . Strong skills in SQL , Python , and data modeling techniques. Familiarity with data formats like Parquet and JSON. Experience with AI/ML model management on Azure Databricks . Education : Bachelor's degree in IT, Computer Science, or a related field. Microsoft Certified: Azure Data More ❯

tools (QuickSight, Power BI, Tableau, Looker, etc.) Interest or experience in building internal data communities or enablement programs Working with diverse data sources (APIs, CRMs, SFTP, databases) and formats (Parquet, JSON, XML, CSV) Exposure to machine learning models or AI agents Why Join Us Help shape the future of data in an organization that treats data as a product More ❯

All our office locations considered: Newbury & Liverpool (UK); Šibenik, Croatia (considered) We're on the hunt for builders . No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems More ❯

Data modelling (building optimised and efficient data marts and warehouses in the cloud) Work with Infrastructure as code (Terraform) and containerising applications (Docker) Work with AWS, S3, SQS, Iceberg, Parquet, Glue and EMR for our Data Lake Experience developing CI/CD pipelines More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy More ❯

Log Analytics, Serverless Architecture, ARM Templates. Strong proficiency in Spark, SQL, and Python/scala/Java. Experience in building Lakehouse architecture using open-source table formats like delta, parquet and tools like jupyter notebook. Strong notions of security best practices (e.g., using Azure Key Vault, IAM, RBAC, Monitor etc.). Proficient in integrating, transforming, and consolidating data from More ❯

using S3Input and S3Output components. Handle large-scale data sets and optimize ETL processes for performance and scalability. Integrate Talend workflows with data lakes, employing file formats such as Parquet, ORC, or Avro for efficient storage and retrieval. Work with NoSQL databases for specific use cases, ensuring seamless integration with Talend workflows. Leverage Apache Spark for data processing and … is highly preferred. Experience with cloud platforms, especially AWS S3, and knowledge of data lake architectures. Familiarity with big data tools such as Apache Spark and file formats like Parquet, ORC, and Avro. Experience with NoSQL databases and their integration with Talend. Solid understanding of CI/CD pipelines and YAML configuration in DevOps contexts. Familiarity with Amazon Glue More ❯

AWS serverless services and enables powerful querying and analytics through Amazon Athena. In this role, you'll work on a system that combines streaming ingestion (Firehose), data lake technologies (Parquet, Apache Iceberg), scalable storage (S3), event-driven processing (Lambda, EventBridge), fast access databases (DynamoDB), and robust APIs (Spring Boot microservices on EC2). Your role will involve designing, implementing … processing pipeline and platform services. Key Responsibilities: Design, build, and maintain serverless data processing pipelines using AWS Lambda, Firehose, S3, and Athena. Optimize data storage and querying performance using Parquet and Iceberg formats. Manage and scale event-driven workflows using EventBridge and Lambda. Work with DynamoDB for fast, scalable key-value storage. Develop and maintain Java Spring Boot microservices … Java backend development experience. 3+ years of Python development. Strong hands-on experience with AWS services: Lambda, S3, K8S. Deep understanding of data lake architectures and formats such as Parquet and Iceberg. Proficiency in Spring Boot and working experience with microservices. Experience with high-scale, event-driven systems and serverless patterns. Nice to Have: Solid understanding of distributed systems More ❯

hands-on AWS experience – S3, Redshift, Glue essential. Proven experience building ETL/ELT pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau, Looker, etc.). Strong scripting and Linux/ More ❯

hands-on AWS experience – S3, Redshift, Glue essential. Proven experience building ETL/ELT pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau, Looker, etc.). Strong scripting and Linux/ More ❯

processing and automation Solid understanding of ETL/ELT workflows, data modelling, and structuring datasets for analytics Experience working with large, complex datasets and APIs across formats (CSV, JSON, Parquet, etc) Familiarity with workflow automation tools (eg, Power Automate) and/or Power Apps is desirable Excellent interpersonal and communication skills with the ability to work cross-functionally and More ❯

processing and automation Solid understanding of ETL/ELT workflows, data modelling, and structuring datasets for analytics Experience working with large, complex datasets and APIs across formats (CSV, JSON, Parquet, etc) Familiarity with workflow automation tools (eg, Power Automate) and/or Power Apps is desirable Excellent interpersonal and communication skills with the ability to work cross-functionally and More ❯

data warehousing (e.g. Hadoop, Spark, Redshift, Snowflake, GCP BigQuery) Expertise in building data architectures that support batch and streaming paradigms Experience with standards such as JSON, XML, YAML, Avro, Parquet Strong communication skills Open to learning new technologies, methodologies, and skills As the successful Data Engineering Manager you will be responsible for: Building and maintaining data pipelines Identifying and More ❯

Experience with alternative data technologies (e.g. duckdb, polars, daft). Familiarity with eventing technologies (Event Hubs, Kafka etc ). Deep understanding of file formats and their behaviour such as parquet, delta and iceberg. What we offer We want to give you a great work environment; contribute back to both your personal and professional development; and give you great benefits More ❯

SQL, and orchestration tools like Apache Airflow. Hands-on experience with data serialization formats such as protobuf, Arrow, FlatBuffers, or Cap'n Proto. Familiarity with data storage formats like Parquet or Avro. Experience with modern analytic storage technologies such as Apache Iceberg or DuckDB. Binary message parsing experience. Strong understanding of classified data handling, secure networking, and compliance in More ❯

our datalake platform Kubernetes for data services and task orchestration Terraform for infrastructure Streamlit for data applications Airflow purely for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing DBT for data modelling SparkSQL for analytics Why else you'll love it here Wondering More ❯

of logic, functions, performance and delivery Extensive database knowledge and ability to manage relational data servers such as: MySQL, Microsoft SQL server, Postgres Legacy formats, e.g. CSV, JSON, XML, Parquet T-SQL (relational, queries, joins, procedures, performance) Familiarity with Python and the Pandas library or similar Familiarity with RESTful and SOAP APIs Ability to build and execute ETL processes More ❯

with REST APIs Experience with Java Experience with full lifecycle agile software development projects Desired skills: Experience with Python. Experience building data products in Apache Avro and/or Parquet On-the-job experience with Java software development. Experience deploying the complete DevOps Lifecycle including integration of build pipelines, automated deployments, and compliance scanning using test driven development. Responsibilities More ❯

Survey Testing Community, with common standards such as metrics and use of test tools Here is a snapshot of the technologies that we use Scala, Apache Spark, Databricks, Apache Parquet, YAML, Azure Cloud Platform, Azure DevOps (Test plans, Backlogs, Pipelines), GIT, GeoJSON What we're looking for Highly skilled in creating, maintaining and peer reviewing test automation code, preferably More ❯

With solid software engineering fundamentals, fluent in Java and Python (Rust is a plus). Knowledgeable about data lake systems like Athena, and big data storage formats such as Parquet, HDF5, ORC, focusing on data ingestion. Driven by working in an intellectually engaging environment with top industry minds, where constructive debates are encouraged. Excited about working in a start More ❯

About the role Taktile is a high-growth, post product-market-fit start-up, on a fast trajectory to becoming market leader in the field of automated decisioning. We are looking for a Full-stack Engineer to join the Decide More ❯

various exchange and processing techniques (ETL, ESB, API). Lead the way in delivering Agile methodologies for successful and timely project delivery. Leverage strong database skills (SQL, NoSQL, and Parquet) for efficient data storage and management. What we're looking for from you: Proficiency in Data Science techniques, including statistical models and ML algorithms. Expertise in NLP, with a … keen understanding of LLM and RAG technologies. Strong development capabilities, particularly in Python. Experience with data exchange, processing, and storage frameworks (ETL, ESB, API, SQL, NoSQL, and Parquet). Comfort with Agile development methodologies. Excellent teamwork and communication skills, with a talent for translating technical concepts into actionable insights for non-specialists. Ability to influence company decision-makers and More ❯

and RAG-based solutions Proficiency in Python and modern AI/ML libraries (eg HuggingFace, LangChain, TensorFlow, PyTorch) Experience with data exchange and storage frameworks (eg APIs, SQL, NoSQL, Parquet) Track record of delivering technical solutions in Agile environments Excellent communication skills and a collaborative mindset Beneficial, but not essential: Experience with containerisation (Docker) Awareness of secure data handling More ❯

and RAG-based solutions Proficiency in Python and modern AI/ML libraries (e.g. HuggingFace, LangChain, TensorFlow, PyTorch) Experience with data exchange and storage frameworks (e.g. APIs, SQL, NoSQL, Parquet) Track record of delivering technical solutions in Agile environments Excellent communication skills and a collaborative mindset Beneficial, but not essential: Experience with containerisation (Docker) Awareness of secure data handling More ❯

Key Skills and Experience required Extensive background in delivering production-grade, data-centric applications for quantitative trading and analytics, with demonstrated expertise in: The python data engineering stack (Polars, Parquet, FastAPI, Jupyter, Airflow, Streamlit, Ray) High-performance data stores and query engines (Starburst, Snowflake) Real-time streaming analytics technologies (Kafka, Flink) Cloud container technologies (AWS, Azure, GCP, Docker, Kubernetes More ❯

as SQL Server, PostgreSQL, Teradata and others. • Proficiency in technologies in the Apache Hadoop ecosystem, especially Hive, Impala and Ranger • Experience working with open file and table formats such Parquet, AVRO, ORC, Iceberg and Delta Lake • Extensive knowledge of automation and software development tools and methodologies. • Excellent working knowledge of Linux. Good working networking knowledge. • Ability to gain customer More ❯