9 of 9 Permanent Apache Spark Jobs in Worcester

LLMs), and specialized models in Computer Vision and NLP. Big Data Engineering: Design, build, and optimize high-throughput, distributed data pipelines and features using Apache Spark (Scala) and Hive on massive datasets to support model training and inference. Cross-Cloud Execution: Lead the design and deployment … environment. Required Skills and Expertise (10+ Years) 1. Big Data and Cloud Mastery Programming & Big Data: 10+ years of extensive, hands-on experience with Apache Spark, with strong preference for production development using Scala. Deep expertise with Apache Hive for data querying and management. Cloud Proficiency: Demonstrated ...

required Do not rule yourself out if you do not have these. They are a bonus, not a checklist. Experience with Delta Lake, Iceberg, Spark or Databricks, Palantir Experience optimising LLM serving with tools such as vLLM, TGI or TensorRT LLM Search and ranking experience. For example Elasticsearch ...

models into production environments. Data & Engineering Skills Strong SQL skills and experience working with large datasets. Familiarity with data processing tools (Pandas, NumPy, Spark). Experience building APIs (FastAPI, Flask) for ML services. ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

Description: This is a generalist mid-level backend position within the Ads Organization. Technologies used on the team include: Languages: Go, Python, Scala Frameworks: Spark, Thrift, Kafka Data stores: Postgres, BigQuery, Redis, Druid Tools: Kubernetes, Airflow, Docker Job responsibilities: You will work with the existing engineering team and help ...

. • Experience building and deploying applications on Kubernetes (EKS). Ask anything Important skills: Must have: 1. Bash Scripting 2.Real time with Spark structured streaming Nice to have : Familiarity with IceBerg GCP/BQexposure ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilize programming expertise … experience in data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue ...

data. You'll work with engineering to move models from idea to production. Tech Stack: Python, scikit-learn, PyTorch, TensorFlow/Keras, LangChain, Pydantic, Spark, Kafka, Databricks, MLflow, Azure, Docker, Kubernetes. Salary: Up to £100k Working Environment: Fully Remote in UK What you'll do Research, prototype, and deploy … summarisation, document processing) Build and tune models using scikit-learn, PyTorch, TensorFlow/Keras, and XGBoost Create data pipelines for batch and streaming with Spark and Kafka Use Databricks and MLflow for experiments, deployment, and monitoring What you'll bring Hands-on Python for data science and ML Practical ...

Engineer Job Location: Remote (UK) Job Type: Full-time permanent Experience: 10 to 12 years Joining: Immediate joiner preferred Mandatory Skills: Scala, AWS, Spark Streaming, DevOps, Bash Scripting. Job Description: Design, develop, and maintain scalable backend services on AWS. Build event-driven architectures using SQS, Kafka, Kinesis, or RabbitMQ. … . Proficiency in SQL and NoSQL databases (DynamoDB, RDS). Experience building and deploying applications on Kubernetes (EKS). Bash Scripting Real time with Spark structured streaming ...