101 to 125 of 186 Remote Apache Spark Jobs

Engineering Strong experience working with AWS in data engineering environments. Hands‐on experience with AWS Glue for data integration and processing. Strong experience using Apache Airflow for workflow orchestration and pipeline management. Advanced Python development skills. Experience with Apache Spark or similar distributed data processing technologies would ...

strong understanding of modern data engineering best practicesYou have experience with Databricks, ideally also AzureYou have experience of working with technologies such as SQL, Apache Spark and Python including PySpark and PandasYou're excited to join a start-up in a role that you can shape and influenceYou ...

auction systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. What ...

auction systems Excellent communication skills, able to explain complex technical systems to commercial, operations and senior leadership teams. Experience with Google Cloud, Docker, Kafka, Spark, Airflow, ElasticSearch, ClickHouse or similar production infrastructure Experience owning product roadmaps, release governance or cross-functional delivery across distributed teams and time zones. Experience ...

Annapolis Junction. Flexibility is essential to adapt to schedule changes as needed. Preferred Requirements Experience with big data technologies like: Hadoop, Accumulo, Ceph, Spark, NiFi, Kafka, PostgreSQL, ElasticSearch, Hive, Drill, Impala, Trino, Presto, etc. Experience with K9s, AWS CLI and other AWS services like SNS, SQS and Diode ...

processes, and the adoption of new tools and frameworks. What you need to succeed: Python, SQL and a distributed framework like Spark; we use Databricks. Experience with marketing or attribution data, and a working understanding of how bidding platforms consume the signal you send them. You don't need ...

role: Python SQL Cloud platform architecture (AWS, Azure, GCP) Data Lakes/Lakehouses and analytical tools (Databricks, Azure Fabric/OneLake, AWS Lake Formation, Spark, Athena, etc.) CI/CD and other DevOps practices such as IaC Testing Additional experience with the following would be beneficial but not essential … vector databases Streaming technologies (Kafka, Kinesis, Flink, etc.) Containers and related services (Docker, Kubernetes, container Registries, etc) Data modelling approaches (Kimball, Imnon) Orchestration tools - Apache Airflow, Prefect or cloud-native tools Backend software development (Java, APIs, Scalability, Logging and Monitoring etc.) MLFlow and other MLOps/Machine Learning Engineering ...

years' experience delivering enterprise data platforms or analytics solutions. Strong commercial experience delivering Databricks solutions. Consulting or customer-facing delivery experience. Strong understanding of Apache Spark, modern Lakehouse architecture and Unity Catalog. Experience with Azure, AWS or GCP. Experience implementing CI/CD and deployment automation. Excellent communication ...

services, with secure integration of enterprise data sources. AWS engineering experienceincluding containers, Kubernetes, CI/CD , a pplication resilience and security . Experience with Apache Spark or similar data-processing frameworks and large structured datasets. Sound judgement when using AI-assisted development tools , including validation of outputs ...

data pipelines. Strong SQL, including performance tuning on large tables. Python for data transformation and pipeline orchestration. Experience with a distributed query engine (Presto, Spark, Hive or similar) and a workflowscheduler (Airflow or similar). Comfortable working in a code review based workflow with Git. Ability to work ...

into robust approaches and measurable outcomes. Strong practical capability in Python and SQL across data exploration, feature engineering, modelling and validation, with experience of Spark and modern cloud data platforms. Expertise in statistical analysis, forecasting, predictive modelling, classification, clustering and optimisation, including developing and deploying production machine-learning solutions. ...

into robust approaches and measurable outcomes. Strong practical capability in Python and SQL across data exploration, feature engineering, modelling and validation, with experience of Spark and modern cloud data platforms. Expertise in statistical analysis, forecasting, predictive modelling, classification, clustering and optimisation, including developing and deploying production machine-learning solutions. ...

read through code, especially Java. Demonstrable experience collaborating with engineering on services Experience working with large datasets and data processing technologies (e.g., Hadoop, Spark, SQL) Familiarity with anomaly detection, supervised and unsupervised learning methods, and real-time data analysis Experience with statistical analysis and good presentation skills to drive ...

approaches Experience with Generative AI techniques such as retrieval-augmented generation, fine-tuning, or prompt engineering Exposure to distributed data processing frameworks, such as Spark Exposure to fraud detection, anti-money laundering, anomaly detection, or graph and network-based modelling Awareness of model risk management, governance, and regulatory controls ...

designing and enforcing data contracts and quality SLAs in distributed data ecosystems. Proficiency in Python, Java, or Scala, and experience building pipelines with Databricks, Spark, or Kafka. Strong understanding of data governance principles, privacy regulations (GDPR, CCPA), and secure data handling practices. Familiarity with metadata management and data catalog ...

more of the following is preferred: Advanced Python or Scala CI/CD tools (Jenkins, Ansible, Fortify, DevOps pipelines) Microservices and Big Data (Spark, Hadoop/HDFS) AWS services (EC2, EMR, Lambda, S3, Glue) QA automation (Cucumber, Selenium, Karate) Preferred Domain Knowledge Financial data systems: Equity, Commodities, Forex, Futures ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

Feast, AWS Feature Store) Experience with containerisation (Docker) and orchestration (Kubernetes, ECS) Familiarity with Infrastructure as Code (Terraform, CloudFormation) Experience with data processing frameworks (Spark, Dask) for large-scale workloads Understanding of data governance and compliance frameworks Experience working in regulated industries (finance, healthcare, or similar) Security Clearance ...

develop homogenous data repositories to deliver data led decision making. Ingest data from various sources, including on premise SQL databases, REST API’s, Apache Kafka streams, etc. Define and apply cleansing rules to the data to ensure meets quality expectations. Define and model data for its appropriate use including … etc.) Other languages, such as C#, PowerShell IaC, e.g. Terraform, ARM, Bicep, etc. Test driven development (TDD) Streaming technologies, such as Azure Stream Analytics, Spark Structured Streaming, etc. Power BI Engineering experience Certified SCRUM Developer (CSD) Data Governance tool experience e.g. Purview Machine learning and Artificial intelligence ...

edge and implementing secure configuration distribution Building streaming ingestion, event pipelines, batch/stream processing workloads using both distributed event streaming technologies and Apache Flink and/or Apache Spark with clear SLA’s and repeatable deployment Implementing metadata, lineage and catalogue capabilities and implementing policy-based ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

Familiarity in Cloud-based application development (we use AWS & Azure) Familiarity with one or more ML frameworks and technologies: scikit-learn, xgboost, TensorFlow, PyTorch, Spark, SageMaker, Vertex AI, Kubeflow, Seldon, Triton Strong communication skills, able to express ideas clearly and collaborate across teams Additional Information Bring ...

Hands-on experience with ETL/ELT pipelines, data engineering, and large-scale data processing. Experience with containerization (Docker) and scalable data systems (e.g., Spark, Kafka). Strong leadership presence, excellent communication, strategic thinking, and empathy with a hands-on execution mindset. Desirable Requirements: Experience with AWS SageMaker ...