76 to 100 of 297 Apache Spark Jobs in England

/CloudFormation). Cloud experience with AWS, Azure or GCP (networking, IAM, storage, compute), including secure architecture principles. Experience with data platforms and pipelines (Spark, Airflow, Kafka, dbt or similar) and working with large-scale datasets. Knowledge of model serving patterns (REST/gRPC, batch scoring, streaming inference ...

grade code Distributed Data Processing Experience building and operating: batch processing systems distributed data systems near real-time streaming pipelines Familiarity with: Kafka Flink Spark Databricks experience is a strong plus Databases & Data Warehousing Comfortable working with: RDBMS data warehousing solutions Strong understanding of data modelling aligned with business ...

Stock Exchange. With 3,000 employees and 32 offices in 12 countries we’re a business with lots of opportunity for people with talent, spark and lots of ambition. If you want to build a great career with a company that prioritises strong values – such as integrity and courage ...

plusWhat it takesMust-Have Skills & ExperienceDeep expertise designing and building scalable data pipelines and data products in cloud environmentsStrong hands-on experience with Databricks, Spark, SQL, and PythonExperience developing and optimizing batch and real-time data processing solutionsStrong understanding of modern data architecture patterns, including Kimball, Data Vault, dimensional ...

large request volumes, large data volumes, or both. Deep expertise in distributed data systems: streaming (Kafka or similar), batch and ELT/ETL frameworks (Spark, Flink, dbt, Airflow or Argo Workflows), and modern lakehouse or warehouse technologies. Strong production Python experience and awareness of other relevant languages (Java/ ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering … Science, Engineering, or a quantitative discipline (or equivalent experience)Nice to Have: Databricks certification or experience with the Databricks PlatformExperience with Unity Catalog, Lakeflow Spark Declarative Pipelines, or MLflowBackground at a data/AI company, cloud provider, or technical consulting firmInterview Process: Recruiter Screen Hiring Manager Screen Design ...

experience designing and implementing data solutions on at least one public cloud platform (AWS, Azure, or GCP)Working knowledge of distributed data systems: Apache Spark, Delta Lake, or equivalent (Hadoop, Kafka, Flink)Experience leading technical customer conversations — discovery, whiteboarding, architecture reviewsFamiliarity with one or more: data engineering … Science, Engineering, or a quantitative discipline (or equivalent experience)Nice to Have: Databricks certification or experience with the Databricks PlatformExperience with Unity Catalog, Lakeflow Spark Declarative Pipelines, or MLflowBackground at a data/AI company, cloud provider, or technical consulting firmInterview Process: Recruiter Screen Hiring Manager Screen Design ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

Experience working for a software vendor, SaaS provider, or professional services organisation. Familiarity with Ataccama or similar enterprise data platforms. Exposure to Hadoop/Spark, NoSQL, XSLT, Hive, Kafka, Snowflake, or Databricks. Python or other programming languages. Familiarity with SDLC, code management and upgrades, and Unix/Linux. Perks ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable ...

platforms for performance and reliability Collaborate with Agile teams to deliver secure, high-quality solutions Write clean, tested, maintainable code Required Skills: Experience with Apache Spark, Hadoop or similar big data technologies AWS Apache NiFi Python or Java Experience building data pipelines and ETL processes Desirable ...

data transformations. Basic to intermediate proficiency in Python with experience using data libraries (e.g., pandas, NumPy) through coursework, projects, or internships. Exposure to Databricks, Apache Spark, or cloud-based data environments. Experience working with real-world datasets and performing data validation and troubleshooting. Strong curiosity and willingness ...

Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts across financial ...

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

driven decision making. What youll be doing Building and optimising scalable data pipelines using Java (primary) and technologies like Kafka, Flink, or Spark Developing real-time streaming solutions and event-driven architectures using AWS services (Kinesis, MSK, Lambda, Glue) Contributing to data architecture, modelling, and best practices across modern ...

complex joins, window functions, aggregation, and query optimization; Experience with a major data warehouse or big data technology such as BigQuery, Snowflake, Redshift, Hive, Spark, Vertica, or StarRocks; Experience with ETL/ELT tools or orchestration frameworks such as Airflow, dbt, or internal frameworks; Proficiency in at least ...

Data Engineer (Python SQL Spark Azure Databricks) London to 160kAre you a tech savvy Data Engineer with a first class education? You could be progressing your career working on complex and challenging systems at a Hedge Fund with over $17 billion under management. What … Python or C# and SQL, experience with version control and contributing to a shared codebasesYou have experience with modern data tools and technologies including Apache Spark and Azure Databricks preferredYou have a strong knowledge of data management principles and best practicesYou have experience with data analysis, visualisation tools ...

internal knowledge Competencies and Behaviours: 3+ years experience in data platform engineering, cloud engineering, or similar roles Strong hands-on experience with Databricks, including Apache Spark, Delta Lake, Workflows Proven experience designing and deploying data platforms on AWS and/or Azure (e.g. ADLS, S3, VNet/ ...

hands-on engineering experience delivering enterprise data platforms or analytics solutions. Strong commercial experience building and delivering Databricks solutions in production. Deep expertise in Apache Spark, including performance tuning and optimisation at scale. Expert-level Python and SQL, with a track record of writing production-quality, well-tested ...

Analytics, Artificial Intelligence or a related field, or equivalent experience, would be effective, although practical experience is equally valuable."Nice to Have"Azure, Databricks, Apache Spark or comparable cloud and data platforms. Data pipelines, ETL and workflow-orchestration technologies. Machine-learning methods such as forecasting, prediction or classification. ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

systems Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

model deployment using MLOps frameworks; Strong Python and SQL programming skills, with experience in PyTorch, TensorFlow, and Hugging Face; Experience with Databricks, Spark, and other data platforms and big data frameworks; Excellent communication skills for engaging C-level stakeholders and aligning technical outcomes with business value; Nice to have ...

security Practical cloud-native experience Preferred Qualifications, Capabilities, and Skills: Experience with modern data technologies such as Databricks or Snowflake Hands-on experience with Spark/PySpark and other big data processing technologies Demonstrated proficiency in software applications and technical processes within disciplines such as data engineering, cloud, artificial ...

learning, model development, and data engineering, with an appreciation of responsible AI principles. Experience with cloud environments (Azure, AWS, GCP) and tools such as Spark, Hive, and Redshift. Demonstrated ability to lead cross-functional teams and mentor junior practitioners. Ability to communicate complex technical concepts clearly to non-technical ...