101 to 125 of 194 Remote/Hybrid Apache Spark Jobs

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python . Experience with data engineering and pipeline technologies such as Apache Airflow and Spark . Experience working with cloud platforms, particularly AWS and Azure . Knowledge of big data technologies and distributed data processing. ...

delivery. Able to demonstrate evidence of applying technology to solve business problems. Essential SkillsStrong working knowledge of modern cloud data platforms (Azure Databricks/Spark using Python and associated frameworks, or Snowflake equivalent).Familiarity with how AI/ML and agentic systems are integrated with enterprise data platforms. Strong … Node) using Webpack, Typescript, and GitHub. Experience with Docker and/or Kubernetes and containerized deployments (AKS).Experience with workflow orchestration tools such as Apache Airflow or n8n.Experience of code quality, test, CI/CD and package managers such as Jest, Lint, Brigade, and Helm. Experience with Kafka ...

processing at edge and implementing secure configuration distributionBuilding streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA's and repeatable deployment Implementing metadata, lineage and catalogue capabilities and implementing policy ...

processing at edge and implementing secure configuration distributionBuilding streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA's and repeatable deploymentImplementing metadata, lineage and catalogue capabilities and implementing policy-based ...

/CD paradigms and systems such as: Ansible, Terraform, Jenkins, Bamboo, Concourse etc. Observability – SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Excellent knowledge of YAML or similar languages The following Technical Skills & Experience would be desirable: Jupyter Hub Awareness RabbitMQ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

such as Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. ...

technologies that our Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing. Discover more about ...

technology to growth, retention and differentiation. Transparent technology investment tied to business priorities. The technology The environment includes technologies such as Scala, Python, Kubernetes, Spark, Hive, AWS, Docker, Terraform, Vault, TypeScript, React, Go and Rust. You do not need experience with every technology. Technical depth, architectural judgement ...

related field, or equivalent practical experience. Experience with AWS or GCP, Kubernetes, SQL warehouse and batch-processing technologies such as Trino, Presto or Spark, Kafka or other streaming systems, and Go or Python would be useful. Node.js experience is a plus. We value transferable evidence over an exact match ...

develop homogenous data repositories to deliver data led decision making. Ingest data from various sources, including on premise SQL databases, REST API’s, Apache Kafka streams, etc. Define and apply cleansing rules to the data to ensure meets quality expectations. Define and model data for its appropriate use including … etc.) Other languages, such as C#, PowerShell IaC, e.g. Terraform, ARM, Bicep, etc. Test driven development (TDD) Streaming technologies, such as Azure Stream Analytics, Spark Structured Streaming, etc. Power BI Engineering experience Certified SCRUM Developer (CSD) Data Governance tool experience e.g. Purview Machine learning and Artificial intelligence ...

meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes Big Data Analytics platforms Cloud-based solutions experience AWS preferred Software Engineering Expertise with: RDBMS … Knowledge of: MLOps Feature Engineering Exposure to data analysis and ML tools such as: scikit-learn pandas NumPy XGBoost LightGBM Familiarity with orchestration tools: Apache Oozie Apache Airflow Why you’ll love it here We want to make Prima a happy and empowering place to work. ...

Python. Experience with Kubernetes and Docker. Understanding of CI/CD pipelines (e.g. GitLab). Exposure to data platforms, MLOps or machine learning environments. Spark or Scala. AWS services (e.g. S3). Elasticsearch or graph databases. OIDC/OAuth. Node.js or React. #J-18808-Ljbffr ...

Qlik). Knowledge of Python or R for advanced analytics and automation is highly desirable. Familiarity with cloud/big data platforms (Snowflake, Databricks, Spark) is preferred. Key Requirements Minimum 5+ years of hands-on experience in Data Analytics or Data Science (experience within Customer Success, Operations, or Digital ...

should have the following skills: Experience implementing data processes for Data Warehouse and production systems. Strong experience with Big Data technologies (e.g., Hadoop/Spark). Programming in modern languages (e.g., Scala and Python). Strong delivery engineering tooling: Git/GitHub, Jenkins, Ansible, and JIRA for requirement/ ...

manage multiple priorities and respond calmly to issues. Self-motivated with a collaborative approach. Preferred: Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. Experience supporting data products or client-facing datasets. Understanding of release testing, UAT or production support processes. Exposure to Jira/Agile ...

model training, evaluation and monitoring Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...

solutions, and lead their implementation. Good written and verbal communication skills. Any Beneficial/Nice to have experience: Experience with Big Data technology (HDFS, Spark, Impala) Understanding of Cloud‐native development principles. Experience with Container Orchestration tools (Docker, Kubernetes, OpenShift, etc.). Proficiency in using Jira for tracking tasks ...

modern NLP stack and multi-modal representation learning is a plus We’d love to see that you've worked with big data systems (Spark, S3, and Airflow) and can program (Java, Scala, or Python) Good understanding of system architecture. Have experience in big data technologies and streaming architecture ...

Azure, AWS or GCP * Define lakehouse standards, data modelling approaches and governance frameworks * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies * Establish security, access control, data quality and compliance standards * Optimise performance, scalability and cloud cost management * Lead and mentor a small team … Required experience * Strong experience as a senior or lead data engineer building and scaling production data platforms * Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows * Python and SQL proficiency * Infrastructure as code experience using Terraform, Bicep or CloudFormation * Cloud data platform experience, ideally Azure ...