101 to 125 of 184 Remote Apache Spark Jobs

delivery. Able to demonstrate evidence of applying technology to solve business problems. Essential SkillsStrong working knowledge of modern cloud data platforms (Azure Databricks/Spark using Python and associated frameworks, or Snowflake equivalent).Familiarity with how AI/ML and agentic systems are integrated with enterprise data platforms. Strong … Node) using Webpack, Typescript, and GitHub. Experience with Docker and/or Kubernetes and containerized deployments (AKS).Experience with workflow orchestration tools such as Apache Airflow or n8n.Experience of code quality, test, CI/CD and package managers such as Jest, Lint, Brigade, and Helm. Experience with Kafka ...

processing at edge and implementing secure configuration distributionBuilding streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA's and repeatable deployment Implementing metadata, lineage and catalogue capabilities and implementing policy ...

processing at edge and implementing secure configuration distributionBuilding streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA's and repeatable deploymentImplementing metadata, lineage and catalogue capabilities and implementing policy-based ...

/CD paradigms and systems such as: Ansible, Terraform, Jenkins, Bamboo, Concourse etc. Observability – SRE Big Data solutions (ecosystems) and technologies such as: Apache Spark and the Hadoop Ecosystem Excellent knowledge of YAML or similar languages The following Technical Skills & Experience would be desirable: Jupyter Hub Awareness RabbitMQ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

effectively with technical teams including Data Scientists and MLOps Engineers. Desirable Skills Experience working with Data Science and/or MLOps environments. Experience with Apache Spark. Experience with Scala. Experience working with S3 and cloud storage technologies. Experience with graph databases. Experience with Elasticsearch. Experience with vector databases/ ...

such as Government, Public Sector, Security, Justice, Defence, Healthcare or other similarly challenging environments . Experience with large-scale data technologies such as SQL, Spark or Hadoop would also be beneficial. About You You will be a technically credible leader who is still comfortable getting into the detail. ...

technologies that our Data Scientists use (we don't expect applicants to have experience with all of these):Python and Databricks for Data Science, Spark, MLFlow and Airflow for ML Workflows, Google Cloud Platform for our analytics infrastructure, dbt and BigQuery for data modelling and warehousing. Discover more about ...

related field, or equivalent practical experience. Experience with AWS or GCP, Kubernetes, SQL warehouse and batch-processing technologies such as Trino, Presto or Spark, Kafka or other streaming systems, and Go or Python would be useful. Node.js experience is a plus. We value transferable evidence over an exact match ...

develop homogenous data repositories to deliver data led decision making. Ingest data from various sources, including on premise SQL databases, REST API’s, Apache Kafka streams, etc. Define and apply cleansing rules to the data to ensure meets quality expectations. Define and model data for its appropriate use including … etc.) Other languages, such as C#, PowerShell IaC, e.g. Terraform, ARM, Bicep, etc. Test driven development (TDD) Streaming technologies, such as Azure Stream Analytics, Spark Structured Streaming, etc. Power BI Engineering experience Certified SCRUM Developer (CSD) Data Governance tool experience e.g. Purview Machine learning and Artificial intelligence ...

meetups Continuously learn and improve engineering practices What we’re looking for batch processing distributed data processing Experience with technologies such as: Kafka Flink Spark Databricks experience is a plus Experience implementing: Data Lakes Big Data Analytics platforms Cloud-based solutions experience AWS preferred Software Engineering Expertise with: RDBMS … Knowledge of: MLOps Feature Engineering Exposure to data analysis and ML tools such as: scikit-learn pandas NumPy XGBoost LightGBM Familiarity with orchestration tools: Apache Oozie Apache Airflow Why you’ll love it here We want to make Prima a happy and empowering place to work. ...

Python. Experience with Kubernetes and Docker. Understanding of CI/CD pipelines (e.g. GitLab). Exposure to data platforms, MLOps or machine learning environments. Spark or Scala. AWS services (e.g. S3). Elasticsearch or graph databases. OIDC/OAuth. Node.js or React. #J-18808-Ljbffr ...

Qlik). Knowledge of Python or R for advanced analytics and automation is highly desirable. Familiarity with cloud/big data platforms (Snowflake, Databricks, Spark) is preferred. Key Requirements Minimum 5+ years of hands-on experience in Data Analytics or Data Science (experience within Customer Success, Operations, or Digital ...

manage multiple priorities and respond calmly to issues. Self-motivated with a collaborative approach. Preferred: Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. Experience supporting data products or client-facing datasets. Understanding of release testing, UAT or production support processes. Exposure to Jira/Agile ...

model training, evaluation and monitoring Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...

solutions, and lead their implementation. Good written and verbal communication skills. Any Beneficial/Nice to have experience Experience with Big Data technology (HDFS, Spark, Impala) Understanding of Cloud‐native development principles. Experience with Container Orchestration tools (Docker, Kubernetes, OpenShift, etc.). Proficiency in using Jira for tracking tasks ...

DatabricksFamiliarity in Cloud-based application development (we use AWS & Azure)Familiarity with one or more ML frameworks and technologies: scikit-learn, xgboost, TensorFlow, PyTorch, Spark, SageMaker, Vertex AI, Kubeflow, Seldon, TritonStrong communication skills, able to express ideas clearly and collaborate across teamsAdditional InformationBring all of you to workWe create ...

translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with an ethos of commitment ...

core component of the role is applying, modifying and designing algorithms and mathematical models to solve business problems using big data architectures (Hadoop, Spark, Cloud). In this role, your primary modelling focus will be neural network‐based forecasting, with a strong emphasis on rigorous evaluation against robust baselines ...

fastest growing scale-upsExperience in the following would be beneficial but not essential: Working with big data technology, ideally Spark but others will also be useful such as Hadoop or ElasticsearchExperience with DevOps tooling and approaches (automation and tooling is something we are passionate about, we even have ...

cohesive user experience Technologies We Use Different backend languages, including Java, Rust, Python and Go Distributed systems technologies such as Kafka, Cassandra, Elasticsearch and Spark Docker and Kubernetes for containerisation and orchestration Industry-standard build tooling, including Gradle and GitHub What We Value A deep understanding of server-side ...

least 2 years in a customer-facing, embedded, or solutions engineering capacityStrong hands-on experience with streaming and event-driven architectures — Kafka, Flink, Kinesis, Spark Streaming, or equivalent (DynamoDB and MemoryDB/Redis)AWS Cloud Fluency - understand how to design scalable end-to-end architectures and pipelines on large ...

Engineer, maintaining and further developing a well-established cloud and CI/CD setup to implement and evolve reliable, well-tested data pipelines in Spark/Scala and to extend the dimensional data models. You will be part of the technical team collaborating closely with platform engineering, data science … ingesting from various sources into Databricks Collaborate closely with software engineers to evolve our software platform and address technical challenges Design, develop, and optimize Spark/Scala jobs for ingestion and transformation, using the existing GitHub Actions and Terraform setup for deployments Implement, run and monitor dbt tests ...