DevOps experience in CI/CD. Experience using Tensorflow, Kubernetes, MLFlow, Kafka, and Airflow. Experience using Python is a must (tools like AWS and Spark are beneficial) Excellent communication skills and team and colleague engagement. A keen interest in problem-solving and using scalable machine learning to solve the more »
improvements Key Skills 3+ years of Python experience Highly statistical and Analytical Exposure to Google Cloud Platform ( BigQuery, GCS, Datalab, Dataproc, Cloud ML (desirable) Spark & Hadoop experience Strong communication skills Good problem solving skills Qualifications Bachelor's degree or equivalent experience in a quantative field (Statistics, Mathematics, Computer Science … classification techniques, and algorithms Fluency in a programming language (Python, C,C++, Java, SQL) Familiarity with Big Data frameworks and visualization tools (Cassandra, Hadoop, Spark, Tableau) This is a permanent position, and offers flexibility with Hybrid working, 2-3 days per week in the office, depending on workload more »
data from a wide variety of corporate structured and unstructured sources, 3rd party providers and publicly available data. Leverage cloud data processing frameworks like ApacheSpark, Databricks, or similar for handling, processing and analysing complex, high-volume data both structured and unstructured. Ability to script repetitive data tasks more »
reviews, and performance of technical proofs of concept. · Experience working in cloud environments with AWS, Azure, or GCP. · Platform implementation experience with technologies like Apache Hadoop, Kafka, Storm, Spark, Cassandra, Elasticsearch, and others. · Conduct technical architecture reviews, code reviews, and performance of technical proofs of concept. · Ability to more »
DevOps experience in CI/CD. Experience using Tensorflow, Kubernetes, MLFlow, Kafka, and Airflow. Experience using Python is a must (tools like AWS and Spark are beneficial) Excellent communication skills and team and colleague engagement. A keen interest in problem-solving and using scalable machine learning to solve the more »
s Office. There will be a particular emphasis in this role on developing within a Microsoft SQL Server development environment and/or an ApacheSpark big data processing environment - creating algorithms and pipelines to ingest and transform data into information systems and solutions capable of answering clinical more »
Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as ApacheSpark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands more »
tools such as Informatica MDM, Informatica AXON, Informatica EDC, and Collibra MySQL, SQL Server, Oracle, Snowflake, PostgreSQL and NoSQL databases Programming languages such as Spark or Python Amazon Web Services, Microsoft Azure or Google Cloud and distributed processing technologies such as Hadoop Benefits : Base Salary more »
Terraform/Docker/Kubernetes. Write software using either Java/Scala/Python . The following are nice to have, but not required - ApacheSpark jobs and pipelines. Experience with any functional programming language. Database design concepts. Writing and analysing SQL queries. Application overVIOOH Our recruitment team more »
quality testing frameworks. Proficiency in Python and familiarity with modern software engineering practices, including 12factor, CI/CD, and Agile methodologies. Deep understanding of Spark (PySpark), Python (Pandas), orchestration software (e.g. Airflow, Prefect) and databases, data lakes and data warehouses. Experience with cloud technologies, particularly AWS Cloud services, with more »
deployment. Have a good understanding of the Data Science Lifecycle and be adept at stakeholder and project management Demonstrate technical skills in Python and Spark, with experience using AWS tooling and Databricks would be ideal Show broad applied knowledge across standard machine learning techniques, with experience in building LLM more »
equivalents) such as EMR, Glue, RedShift, Kinesis, Lambda, Data Factory, and Databricks. Design and deploy production data pipelines using languages like Java, Python, Scala, Spark, and SQL. Integrate data engineering components into production systems, focusing on data cleaning, data wrangling, data visualization, and reporting. Handle substantial volumes of structured … science, and product development, with experience in both stream and batch processing. Experience in designing and deploying production data pipelines with Java, Python, Scala, Spark, and SQL. Capability in scripting, data extraction via APIs, and composing SQL queries. Familiarity with data mining, machine learning, and natural language processing is more »
or Rust. Experience in building and enhancing compute, storage, and data platforms with exposure to open source products like Kubernetes, Knative, Ceph, Rook, Cassandra, Spark, Nate etc. Hands-on exp. with IaC tools and automation, such as Terraform, Ansible, or Helm. Active engagement or contributions to the open-source more »
ll be at the forefront of innovation, working with their Modern Data Stack which includes a scale-out Engine (Trino), general-purpose distributed compute (Spark), and ad-hoc containerized compute. They utilize an S3 data lake and are soon incorporating dbt and Airbyte. Their cloud-agnostic architecture runs on … to expand to Azure, GCP and on-prem. TECH STACK: Programming languages: Python, Flask, React Databases & Storage: Redis, Postgres, AWS Athena SQL, Athena & EMR Spark, ECS Fargate, Azure Synapse/Data Lake Analytics, HDInsight Tools & Platforms: Docker, Temporal, Plotly KEY RESPONSIBILITIES Design and implement mid-to-low level data … levels Strong focus on identifying and progressing on critical paths, both independently and as part of a team PREFERRED BUT NOT REQUIRED: Experience with Spark or dbt Experience in deploying production systems on multiple cloud platforms Familiar with MLOps principles and practices in a production setting COMPENSATION/BENEFITS more »
such as Data Factory, Event Hubs, Data Lake, Synapse, and Azure SQL Server. Create and optimize data processing workflows in Databricks using PySpark and Spark SQL. Ensure ETL coding standards are met, including self-documenting code and reliable testing. Apply best practice data encryption techniques and standards to ensure … experience with Azure data products including Data Factory, Event Hubs, Data Lake, Synapse, and Azure SQL Server. Proficient in developing with Databricks, PySpark, and Spark SQL. Strong understanding of ETL coding standards, including standardized, self-documenting code and reliable testing. Knowledge of data encryption techniques and standards. Familiarity with more »
Greater London, England, United Kingdom Hybrid / WFH Options
Hunter Bond
My client are looking for a talented and motivated Big Data Architect (Azure, Databricks, Spark) to be based in their London office. You'll be responsible for providing technical leadership in architecting and designing end-to-end solutions for the organisation's datalake initiatives, as they provide increasing numbers … improvements in design, processes, and implementation to improve operational management, scalability, and extensibility. The following skills/experience is essential: Strong implementation experience using Spark and Databricks Strong Cloud experience (ideally Azure) Previously heavily involved in an implementation programme Data Warehouse Strong stakeholder management experience Excellent IT background, ideally more »
analysis. Your expertise will be instrumental in ensuring the security and efficiency of the data handling and reporting processes. Key Responsibilities: Data Processing: Utilize ApacheSpark, AWS RDS, and Hadoop to process large datasets efficiently and securely. Reporting: Generate comprehensive and insightful reports using Tableau. Business Rules Management … adherence to best practices and maintaining high-security standards. Requirements: Security Clearance: Must hold a current and valid Security Clearance. Technical Skills: Proficient with ApacheSpark, AWS RDS, and Hadoop. Experienced in using Tableau for data visualization and reporting. Familiarity with Red Hat Decision Manager for business rules more »
ADF, SSIS) Data governance (Purview, Unity Catalogue) Databricks Delta Lake Storage Azure Dev OPS DESIRED SKILLS Advanced Analytics Data Technologies Databricks, Delta Lake, Synapse Spark SQL, Pyspark Azure Data Explorer Logic Apps, Key Vault Semi structured data processing Integration Runtime Coding experience: Python, C#, Java for Data analysis purpose more »
or similar, with 6+ years of professional experience. A good understanding of modern lakehouse architectures and corresponding technologies, such as Dremio, Snowflake, Iceberg, (Py)Spark/Glue/EMR, dbt and Airflow/Dagster. Experience with Cloud providers. Familiarity with AWS S3, ECS and EC2/Fargate would be more »
Solid experience with programming languages like Python and proficiency in SQL Experience working with cloud platforms (e.g., Azure, GCP) and big data technologies like Spark Working knowledge of ETL and ELT frameworks and tools like dbt Working knowledge of Version Control and CICD pipelines Hiring, developing, and retaining cyber more »
C++, Java, or similar. Experience using SQL, Python (Pandas, NumPy, Scikit-Learn, etc.)/R or equivalent. Experience with Data Storage, Ingestion, and Transformation (Spark, Kafka or similar tools). Experience developing cloud infrastructure services, preferably with Kubernetes. You are passionate about solving data problems at Scale. Expertise with more »
machine learning or more general statistical analysis Strong software development skills with proficiency in Python or C++ Experience with analytics frameworks such as Pandas, ApacheSpark, Dask, or Flink Experience with machine learning frameworks such as TensorFlow, JAX, PyTorch, Spark MLlib, Keras, or scikit-learn Experience in … cloud-based infrastructures such as AWS or GCP Exposure to orchestration platforms such as Apache Airflow or Kubeflow Proven attention to detail, critical thinking, and the ability to work independently within a cross-functional team What benefits do we offer ? Cond Nast Learning Hub where you ll find you more »
C++, Java, or similar. Experience using SQL, Python (Pandas, NumPy, Scikit-Learn, etc.)/R or equivalent. Experience with Data Storage, Ingestion, and Transformation (Spark, Kafka or similar tools). Experience developing cloud infrastructure services, preferably with Kubernetes. You are passionate about solving data problems at Scale. Expertise with more »
tasks, both for oneself and others, ensuring progress aligns with project goals. Nice to have: Previous experience in startup environments. Proficiency or experience with Apache Spark. Familiarity or background in working with Azure. Experience orchestrating workflows, particularly within distributed system environments. Knowledge of MLOps principles and practices, especially in more »
and partners Preferred Requirements Experience or strong interest in blockchain and other Web 3.0 technologies Experience with OLAP technologies, such as, Presto/Trino, Spark, Hadoop, Athena, or BigQuery is a plus Experience in Golang or any other strongly-typed programming language Experience mentoring and supporting fellow engineers Our more »
