26 to 38 of 38 Remote Apache Spark Jobs

Event Hubs on Azure, or BigQuery, Dataflow and Pub/Sub on GCP Databricks and Snowflake running on top of any of the three Spark and distributed processing frameworks for large-scale data Streaming technologies such as Kafka or the native cloud equivalents You’re expected to pick ...

Familiarity with monitoring tools such as CloudWatch, Prometheus or Grafana. Useful additional experience Relevant AWS certifications. Exposure to data-processing environments using Databricks or Apache Spark. The person we are looking for You enjoy making systems easier to run and reducing the need for manual intervention. You investigate problems ...

management and international regulatory compliance. YOUR SKILLS AND EXPERIENCE ESSENTIAL CRITERIA Strong experience designing, building and maintaining scalable data pipelines and integration solutions using Apache Airflow (or similar orchestration platforms), Python, SQL and bash/linux. Experience developing cloud-native data platforms and solutions on AWS, including working with … platforms such as Amazon Bedrock. Experience developing data-based apps in Streamlit/Shiny Skills in 'Big Data' technology tools like Hadoop/Spark Knowledge of Infrastructure as Code tools like CloudFormation/Terraform If you can bring some of these skills and experiences, along with transferable strengths ...

Senior Data Engineer – SQL/Python/AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data ...

understanding of statistics, machine learning methodologies, and model evaluation. Experience delivering AI solutions from proof-of-concept through to production. Exposure to Databricks, MLflow, Spark, or similar cloud-based data platforms. Ability to influence senior stakeholders and lead technical strategy Desirable Experience Experience building multi-agent AI systems ...

translate complex AI capabilities into clear business use cases and recommendationsExperience in at least one programming language such as Python, R, SQL, SAS, Spark, Java, etc., with practical applications in AI or ML projectsFamiliarity with AI techniques and machine learning algorithms, including supervised and unsupervised learningStrong communication skills with ...

kubernetes) Experience with Key value stores Experience in git and basic Unix Commands You might also have (Desirable) Experience with large data processing technologies (Apache Spark) Other helpful information Hybrid Working: At EMBL-EBI we are pleased to offer hybrid working options for all our employees. A dedicated ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming).Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

deploying scalable data platforms end-to-end on AWS and/or Databricks. Production Coding: Strong, hands-on production experience with Python, SQL, Spark, and Databricks or AWS data tools. Consulting/Client Delivery: Experience working in a consultancy, systems integrator, or client-facing setting with direct accountability …/or AWS certifications at Associate or Professional level. Streaming Architectures: Practical experience with real-time or event-driven streaming (Kafka/MSK, Kinesis, Spark Structured Streaming). Advanced Data Architectures: Exposure to Data Mesh concepts, data products, or federated data governance models. AI/ML Infrastructure: Exposure ...

Senior Data Scientist - Python/Machine Learning Permanent Location: UK - Remote Salary: £65,000 - £70,000 (+ benefits) Skills: Python, Machine Learning, Statistical Modelling, AWS, Data Engineering An exciting opportunity has arisen for an experienced ...

work collaboratively and autonomously. Strong problem-solving skills with a pragmatic and analytical outlook. Desirable: Familiar with analytics tools such as Power BI, Apache Superset, or similar for building interactive and impactful visualizations Familiarity with distributed systems (e.g., Spark, Kafka) and how they support scalable analytics solutions Passion ...

Scala programming to develop robust, scalable, and high-performance data processing applications. Build and optimise large-scale data processing and analytics pipelines using Apache Spark. Develop and maintain end-to-end data pipelines for both batch and real-time data processing. Apply solid understanding of data engineering concepts, including … business stakeholders. Skills & Experience: Proven experience as a Software Engineer with a strong focus on Scala development. Extensive hands-on experience with OpenShift and Apache Spark. In-depth understanding of data engineering principles and practises. Experience in developing and maintaining scalable data pipelines for batch and real-time processing. ...