1 to 25 of 97 Apache Spark Jobs in the UK

Basic understanding of Python and SQL Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP) Knowledge of ETL tools, Apache Spark, Apache Airflow, or Kafka Understanding of relational and NoSQL databases Interest in big data, cloud computing, data warehousing, or machine learning ...

Basic understanding of Python and SQL Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP) Knowledge of ETL tools, Apache Spark, Apache Airflow, or Kafka Understanding of relational and NoSQL databases Interest in big data, cloud computing, data warehousing, or machine learning ...

Basic understanding of Python and SQL Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP) Knowledge of ETL tools, Apache Spark, Apache Airflow, or Kafka Understanding of relational and NoSQL databases Interest in big data, cloud computing, data warehousing, or machine learning ...

Basic understanding of Python and SQL Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP) Knowledge of ETL tools, Apache Spark, Apache Airflow, or Kafka Understanding of relational and NoSQL databases Interest in big data, cloud computing, data warehousing, or machine learning ...

Basic understanding of Python and SQL Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud Platform (GCP) Knowledge of ETL tools, Apache Spark, Apache Airflow, or Kafka Understanding of relational and NoSQL databases Interest in big data, cloud computing, data warehousing, or machine learning ...

databases utilised in modern AI applications. Strong Python development skills. Data & Infrastructure: Experience with some of the following technologies is highly desirable: Frameworks & Infrastructure: Apache Spark Apache Airflow Kafka Elasticsearch/OpenSearch Databases: PostgreSQL MongoDB Vector databases such as Qdrant, Milvus, or pgvector Python Ecosystem: Pandas Polars ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … progression within a growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data ...

frameworks, tools and applications that make that data available to other teams and systems.Your primary toolset in your work will be Java microservices, Spark/Scala data processing (also some Flink), Kubernetes and AWS native tooling.What You'll Do: Write highly fault-tolerant Java code within Apache Spark … combined between backend/cloud development and data platform engineering roles5+ years of experience building data platform product(s) or features with (one of) Apache Spark, Flink or Iceberg, or with comparable tools in GCP5+ years of experience programming with Java, Scala or Kotlin.Proven experience utilizing AI technologies ...

Experience using Git/version control Strong communication skills with both technical and non-technical stakeholders Desirable Skills: Microsoft Fabric Azure Synapse Analytics Databricks Apache Spark/PySpark dbt Apache Airflow Power BI Azure DevOps CI/CD pipelines Terraform What's On Offer: £65k+ salary package ...

standards. Skills Strong experience in SQL and data modeling concepts. Hands-on experience with ETL/ELT development. Proficiency in Python, Scala, or Spark-based data processing. Experience with cloud platforms such as AWS, Azure, or GCP. Knowledge of Data Warehouse and Data Lake architectures. Experience with orchestration tools ...

plusAbility to write clean, maintainable, scalable, and robust code in PythonFamiliarity with analytics libraries (e.g., pandas, numpy, matplotlib), distributed computing frameworks (e.g., Spark, Dask), and cloud platforms (e.g., AWS, Azure, GCP)Basic understanding or exposure to containerization technologies such as Docker and Kubernetes would be beneficialExposure to software engineering ...

/ELT processes and data pipeline development Experience working with cloud platforms such as AWS, Azure or GCP Knowledge of technologies such as Spark, Kafka, Airflow or similar Experience with databases, data modelling and data warehouse solutions Understanding of CI/CD, software engineering best practices and secure development ...

modern frameworksWorking knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with expertise in at least oneDeep experience with distributed computing with Apache Spark and knowledge of Spark runtime internalsFamiliarity with CI/CD for production deploymentsWorking knowledge of MLOps, ML/AI models ...

able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills:- Proficient in Python and SQL, with experience in Apache Spark.- Strong experience building backend services and APIs. Database Expertise:- Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

quality and consistency across platforms while complying with governance policies. What You Bring: Core Development Skills: - Proficient in Python and SQL, with experience in Apache Spark. - Strong experience building backend services and APIs. Database Expertise: - Solid understanding of relational databases (SQL Server, PostgreSQL) and distributed NoSQL databases (Cassandra, MongoDB ...

engineering best practices Skills & Experience * 3+ years' commercial experience in Data Engineering * Strong Java or Python development skills * Experience with Kafka, Flink or Spark * Experience building ETL/ELT or streaming data pipelines * Knowledge of AWS (Azure or GCP also considered) * Experience with Docker, Kubernetes and CI/ ...

large request volumes, large data volumes, or both.Deep expertise in distributed data systems: streaming (Kafka or similar), batch and ELT/ETL frameworks (Spark, Flink, dbt, Airflow or Argo Workflows), and modern lakehouse or warehouse technologies.Strong production Python experience and awareness of other relevant languages (Java/Kotlin) sufficient ...

Market Risk, XVA, CCR, FRTB, portfolio risk, fixed income analytics, securitised products, liquidity risk, or managed risk-as-a-service solutions.Familiarity with Snowflake, Databricks, Spark, AWS, Azure, Confluence, Azure DevOps, Git, Tableau, Power BI, message queues, or comparable tools and platforms.Knowledge of market data vendors, data mastering, golden-source ...

Lead Data Engineer (Databricks) - London (Hybrid) - Up to £110,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a growing technology business investing heavily in its cloud data platform and AI capabilities. This is a senior, hands-on leadership role where … machine learning and large-scale data initiatives. Key Experience Required Proven experience as a Senior or Lead Data Engineer. Deep expertise in Databricks, Spark, Delta Lake, Python and SQL. Experience designing and scaling cloud-based data platforms. Strong knowledge of ETL/ELT, data modelling and distributed processing. Experience ...

volume, data-intensive environments. Extensive AWS experience including services such as Glue, Lambda, S3, Redshift, and wider cloud-native data tooling. Strong knowledge of Apache Spark and large-scale data processing. Experience working with streaming technologies such as Kafka. Expert SQL and data modelling capabilities. Experience designing modern ...

range of projects that include internal and external suppliers. Essential Skills Proven experience as a Data Engineer Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience with Microsoft Fabric Experience of using programming languages such as Python, Scala and SQL. Demonstrable ...

progression taking customers from EDW to the full capabilities of the Data Intelligence Platform.Demonstrated competence in the Lakehouse architecture including hands-on experience with Apache Spark, Python and SQLExcellent communication skills; both written and verbalPublic Cloud Architecting: Hands-on experience designing and implementing big data architectures within major ...

What we’re looking for: Strong Python development and software engineering experience Experience building scalable data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments ...

platform and lakehouse transformation journeys What We're Looking For Strong commercial experience in Data Engineering Hands-on experience with Databricks and/or Apache Spark Strong PySpark and SQL development skills Experience designing, building and supporting data pipelines Experience working with Azure or AWS Good understanding ...