21 of 21 Apache Spark Jobs in the South East

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … progression within a growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data ...

data warehouses and data lakes. (Preferred) Experienced in data ingestion, processing, transformation, and delivery using batch, streaming, and real‐time methods. (Preferred) Experience supporting Spark‐based platforms (e.g. EMR, Glue, Databricks) (Preferred) Proficient with DevOps tools and practices (Git, Jenkins, Docker, Kubernetes) as well as ETL frameworks (Apache Spark, Airflow, AWS IoT Analytics/Device Management/Events). (Preferred) Programming expertise in Python, Java, Scala, SQL, Spark, Unix shell scripting, and AWS CLI. (Preferred) Great background in building and managing AWS platforms for analytics and data lake workloads, including application and framework management. (Preferred ...

common technical standards and ways of working, and presenting these to the community Here's a snapshot of the technologies that we use: Scala Apache Spark Databricks Apache Parquet Azure Cloud Platform Azure DevOps (Test plans, Backlogs, Pipelines) GIT What we're looking for We'd love ...

standards. Skills Strong experience in SQL and data modeling concepts. Hands-on experience with ETL/ELT development. Proficiency in Python, Scala, or Spark-based data processing. Experience with cloud platforms such as AWS, Azure, or GCP. Knowledge of Data Warehouse and Data Lake architectures. Experience with orchestration tools ...

with the Lead Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation … optimisation opportunities * Translate business requirements into technical solutions * Contribute to documentation and continuous improvement Required experience * Strong experience delivering production grade Databricks pipelines * Strong Spark, Delta Lake and Databricks Workflows experience * Python and SQL proficiency * Experience on Azure data platforms such as Azure Data Factory, ADLS Gen2, Azure DevOps ...

Skills MUST HAVE: Strong hands-on experience with Quantexa implementation and development. Proven experience building and supporting big data platforms . Advanced knowledge of Apache Spark/PySpark . Strong programming skills in Scala and/or Python . Expertise in SQL and large-scale data modelling. Strong ...

similar data-focused role Strong SQL and Python development skills Experience building and supporting scalable data pipelines and transformation processes Hands-on experience with Apache Spark, Databricks, dbt and cloud-based data platforms Good understanding of data modelling principles, data lakes and data warehouse architectures Experience working with … Essential: Degree in Computer Science, Software Engineering, Data Engineering, Information Systems, Mathematics or a related discipline; or equivalent practical experience Desirable: Relevant certifications in Spark, cloud technologies, data engineering or software engineering practices Why should you apply? We're not a perfect place - but we're a people place. ...

Lead Data Engineer (Databricks) - London (Hybrid) - Up to £110,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a growing technology business investing heavily in its cloud data platform and AI capabilities. This is a senior, hands-on leadership role where … machine learning and large-scale data initiatives. Key Experience Required Proven experience as a Senior or Lead Data Engineer. Deep expertise in Databricks, Spark, Delta Lake, Python and SQL. Experience designing and scaling cloud-based data platforms. Strong knowledge of ETL/ELT, data modelling and distributed processing. Experience ...

decisions, documentation and continuous improvement initiatives. Key Skills & Experience Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data ...

What we’re looking for: Strong Python development and software engineering experience Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable ...

used in modern AI systems Strong Python programming experience A habit of making, and justifying, architectural calls, not just executing them Nice to Have: Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, MongoDB, Qdrant/Milvus/pgvector, Pandas/Polars, cloud infra (GCP/AWS/Azure ...

workflows. Key Requirements Passion for building scalable, resilient cloud data platforms. Strong experience with Databricks or Snowflake on AWS. Proven Python skills, including Spark and Airflow expertise. Advanced SQL skills and end-to-end data modelling experience. Experience building batch and real-time data integrations. Hands-on CICD skills ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

data extraction, transformation, and pipeline development Experience with dashboarding and visualization tools (Tableau, Qlik, or similar) Familiarity with big data tools (Snowflake, Databricks, Spark) and ETL processes Experience with Python or R for advanced analytics, automation, or experimentation support preferred Knowledge of statistical methods and experimentation (A/ ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

requirements but feel excited about this role, we still encourage you to apply. We're building a diverse, talented team where different perspectives spark better ideas. Why should you apply? We're not a perfect place - but we're a people place. Our priority is supporting ...

market data normalisation. Experience handling time-series challenges such as timestamp precision, sequence gaps, duplicate and out-of-order events. Experience with distributed processing (Spark, Polars, Dask, Ray or similar). Experience with Kafka or similar streaming technologies. Familiarity with PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Docker, Linux … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

test-driven approach. Creating code that is open by default and easy for others to reuse. Required SkillsExperience with Big Data technologies, such as Apache Hadoop, Spark or PigAWSApache NiFiPython or JavaBuilt on over a 70 year heritage, Roke offers specialist knowledge in sensors, communications, cyber ...

Software Engineer - Space Domain Awareness | Job DetailsAre you looking for an innovative and consolidated place for your professional development? GMV is an experienced technology group and we are looking for a highly motivated Software Engineer ...