26 to 38 of 38 Spark SQL Jobs

interested in engineers with strong hands-on Databricks experience. Essential Skills Commercial experience building solutions within Databricks. Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake. Experience designing and supporting production-grade ETL/ELT pipelines. Strong Python and SQL development skills. ...

experience designing canonical data models or enterprise logical data models. Strong understanding of data modelling techniques, including conceptual, logical and physical modelling. Strong SQL skills and ability to interrogate complex relational data structures. Experience working with cloud data platforms, ideally AWS and Databricks. Understanding of modern data architecture … platforms. Experience with regulatory, client or operational reporting. Experience supporting data migration, legacy system modernisation or platform replacement. Experience with Databricks, Delta Lake, Spark SQL or PySpark. Experience defining validation frameworks, data quality dashboards or metadata-driven rule engines. Experience working in consultancy or client-facing ...

your duties, you will be responsible for: Design, build, and deploy scalable data pipelines using Azure Databricks , Azure Data Factory , and Azure SQL Database . Optimise Databricks environments for performance, cost, and security. Develop and maintain data integration , ETL/ELT processes, and data quality frameworks. Translate business … environments. 8–10 years’ experience in data engineering and integration , ideally in enterprise‐scale environments. Hands‐on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working ...

current as new views and data sources are onboarded; Engage with AI agents for test authoring, investigation, result analysis, and documentation. Требования Strong SQL skills, including window functions, CTEs, and aggregations; Hands-on experience with Databricks, Unity Catalog, and Spark job outputs; Working knowledge of PySpark … Spark SQL; Understanding of Lakehouse and medallion architecture; Familiarity with YAML-based configuration and structured test definitions; Comfortable with Git and basic engineering practices; Experience with AI-assisted workflows and large language model agents; 3-5+ Years of experience in data quality, data testing, analytics ...

specified, and tracked — not just worked around QA coverage expands naturally with each deployment increment rather than lagging behind Competencies & Behaviours Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data and verify correctness Hands-on experience with Databricks — running … queries, navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver ...

named, specified, and tracked — not just worked around QA coverage expands naturally with each deployment increment rather than lagging behind Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data and verify correctness Hands-on experience with Databricks — running queries … navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

results, and managing work items — treating AI-assisted tooling as a first-class part of your workflow rather than a novelty. Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data andverify correctness Hands-on experience with Databricks — running queries … navigatingUnity Catalog, reading Spark job outputs and understanding what they Working knowledge of PySpark or Spark SQL — enough to read pipelinecode, understand transformations, and trace where data Understanding of Lakehouse/medallion architecture (bronze-silvergold) and how data flows and changes shape across layers ...

emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building ...

with Pythonic data processing tools such as Pyspark and SparkSQL. Strong understanding of data modelling concepts relational and non-relational databases. Proficiency in SQL and experience with data integration and ETL tools. Excellent knowledge of big data technologies and frameworks. Strong knowledge of data governance and data security ...

with Pythonic data processing tools such as Pyspark and SparkSQL. Strong understanding of data modelling concepts relational and non-relational databases. Proficiency in SQL and experience with data integration and ETL tools. Excellent knowledge of big data technologies and frameworks. Strong knowledge of data governance and data security ...

translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with ...

drive to make a positive impact on the world Experience working within a finance team and senior stakeholders Proficiency in writing robust, performant SQL queries Some experience building robust data pipelines, using Python and dbt Strong analytical skills, commercial interest, and the ability to distinguish between what matters … tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing SparkSQL for analytics Why else you'll love it here Wondering what the salary for this role is? Just ask us! On a call with ...

experience in data/network security would be a nice-to-have Datadog/Grafana/Prometheus Data related products (Airflow, Jupyter, Spark, dbt etc) The projects will be varied and we're looking for someone who can work autonomously and proactively to scope problems and solve … formats on S3 for data lake storage Postgres/Aurora for our relational databases Spark for data processing dbt for data modelling SparkSQL for analytics Streamlit for data applications Are you ready for a career with us? We want to ensure you have the right tools and environment ...