11 of 11 Spark SQL Jobs in London

department. This is an exciting opportunity to work on cutting edge technologies including Big Data and NoSQL databases (Hadoop, HBase, Hive, Spark and MongoDB) to allow the business to gain advanced insight into their portfolios and valuation metrics. Key responsibilities include:* Lead business requirements elicitation, analysis, and documentation … efforts* Assist technology team in Azure cloud solution implementation; write Spark SQL queries to analyze and transform data; provide feedback to User Interface team; testing and documentation* Analyze solution requirements; understand project scope; determine project priorities; ensure efficient and on-time delivery of project tasks ...

approved; relevant data solutions development qualifications (e.g. Microsoft certification; Databricks Certification; data apprenticeship qualifications)Additional InformationTech StackSQL Server, SSIS, T-SQLADF, ADLS, Azure SQL Database, PySpark, Spark SQL, Databricks, Delta lakeSummaryType: Full-timeFunction: Analyst ...

build real world value. What you'll do Design, build, and operate production ETL pipelines on Databricks using Delta Live Tables and Spark, applying medallion architecture patterns across Finance, Payments, Product, and GTM domains. Own data modeling in different layers — defining governed, reusable table structures that serve … maintaining production pipelines and data models. Deep proficiency with Databricks or similar platforms — Delta Live Tables, Unity Catalog, Delta Lake, and PySpark or Spark SQL as primary tools. Strong SQL skills: complex transformations, incremental materialization, schema evolution, and data quality validation. Solid AWS experience ...

build real world value. What you'll do Design, build, and operate production ETL pipelines on Databricks using Delta Live Tables and Spark, applying medallion architecture patterns across Finance, Payments, Product, and GTM domains. Own data modeling in different layers - defining governed, reusable table structures that serve … maintaining production pipelines and data models. Deep proficiency with Databricks or similar platforms - Delta Live Tables, Unity Catalog, Delta Lake, and PySpark or Spark SQL as primary tools. Strong SQL skills: complex transformations, incremental materialization, schema evolution, and data quality validation. Solid AWS experience ...

Application Resiliency, and SecurityIn-depth knowledge of the financial services industry and their IT systemsPractical cloud native experiencePreferred qualifications, capabilities, and skillsAdvanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing ...

real-time pipelines powering trading decisions, risk management, and market analytics.Key Responsibilities Data Architecture: Build and optimize ETL/ELT pipelines using Databricks, Spark (PySpark/Scala), and Delta Lake for structured/unstructured energy market data.Trading Analytics: Partner with Quants, Risk Managers, and Traders to implement models … curve generation, position tracking, and risk metrics (e.g., VaR). Performance Optimization: Tune clusters and SQL warehouses for low-latency, time-sensitive trading environments. Data Governance: Enforce governance, access controls, and compliance with UK/EU financial and energy regulations via Unity Catalog. Consultative Leadership: Provide architectural best ...

interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. ...

experience designing canonical data models or enterprise logical data models. Strong understanding of data modelling techniques, including conceptual, logical and physical modelling. Strong SQL skills and ability to interrogate complex relational data structures. Experience working with cloud data platforms, ideally AWS and Databricks. Understanding of modern data architecture … platforms. Experience with regulatory, client or operational reporting. Experience supporting data migration, legacy system modernisation or platform replacement. Experience with Databricks, Delta Lake, Spark SQL or PySpark. Experience defining validation frameworks, data quality dashboards or metadata-driven rule engines. Experience working in consultancy or client-facing ...

emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building ...

drive to make a positive impact on the world Experience working within a finance team and senior stakeholders Proficiency in writing robust, performant SQL queries Some experience building robust data pipelines, using Python and dbt Strong analytical skills, commercial interest, and the ability to distinguish between what matters … tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing SparkSQL for analytics Why else you'll love it here Wondering what the salary for this role is Just ask us! On a call with ...

experience in data/network security would be a nice-to-have Datadog/Grafana/Prometheus Data related products (Airflow, Jupyter, Spark, dbt etc) The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems and solve … formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data modelling SparkSQL for analytics Streamlit for data applications Are you ready for a career with us? We want to ensure you have the right tools and environment ...