1 to 25 of 28 Remote/Hybrid Permanent Spark SQL Jobs

reporting, analytics, and operational decision-making across the project programme. Working within a multidisciplinary Data & Analytics team, the successful candidate will use Databricks, SQL, Python, and Microsoft technologies to deliver robust, scalable, and maintainable data solutions. They will collaborate closely with Data Architects and Performance Analysts to transform … business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source ...

documentation, and deployment of configuration changes Develop and support integrations using Node.js Support related data integration work using Python, AWS Glue, PySpark, and Spark SQL as needed Support production issues as needed Primary Requirements: Approximately 3-4+ years of relevant experience Strong hands-on experience … code configuration and API support Desired Skills: Direct Newgen platform experience Healthcare provider-domain experience Experience with Python, AWS Glue, PySpark, Spark, and Spark SQL If you have the described qualifications and are interested in this exciting opportunity, please apply! Ranked a Top Staffing ...

knowledge‐sharing sessions to uplift team capability. Proven experience working as an Analytics Engineer, Data Engineer, or similar role. Deep proficiency in SQL for data transformation and analysis. Strong hands‐on experience with dbt (Data Build Tool) on Databricks, including advanced modelling techniques, test writing, and performance optimisation. … projects. Tools & Technologies You’ll Work With: Databricks (primary data platform for modelling and transformation) dbt for data modelling and transformation on Databricks SQL (Databricks SQL/Spark SQL) VS Code UI for SQL and dbt coding Git/GitHub ...

techniques for large‐scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands‐on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema ...

3+ years with strong Foundry depth). Proven, hands-on experience working with Palantir Foundry in a production environment. Strong proficiency in Python, SQL, PySpark and Spark SQL Experience delivering production pipelines in AWS, Azure, or GCP environments. Solid understanding of data modeling, schema ...

platforms,lakehousesand data warehouses. Data modelling and dimensional modelling techniques such as Kimball. ETL/ELT development and large-scale data integration. PySpark, Spark SQL and modern transformation techniques. Advanced SQL and relational databases including SQL Server, PostgreSQL and Oracle. Integrating structured … semi-structured and unstructured data sources. Migration and modernisation of legacy platforms such as Oracle, SQL Server/SSIS or Synapse. Power BI semantic models, curated datasets and supporting analytics platforms. Data quality, governance, metadata management and lineage. Data security, classification and access control approaches. Automated testing, validation ...

Implement automated testing, CI/CD patterns + deployment best practices for Foundry-based data assets. Technical Environment: Data Engineering, Palantir Foundry, Python, SQL, PySpark, Spark SQL + Cloud (AWS, Azure, GCP), CI/CD, Git based workflows. Benefits ...

team. This role is central to delivering Canopius' data strategy through development of new data capabilities and supporting the final transition from legacy SQL Server-based environments. Alongside hands-on delivery, the role provides technical leadership across data engineering initiatives, helping to establish a future-ready, cloud-native … secure, scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure ...

approved; relevant data solutions development qualifications (e.g. Microsoft certification; Databricks Certification; data apprenticeship qualifications)Additional InformationTech StackSQL Server, SSIS, T-SQLADF, ADLS, Azure SQL Database, PySpark, Spark SQL, Databricks, Delta lakeSummaryType: Full-timeFunction: Analyst ...

processes and solutions. Lead the execution of Microsoft Fabric migration activities. Lead and guide migration engineers and QA resources. Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks. Create data integration test cases and support testing activities. Implement and amend data … Azure Functions o Azure App Service o Logic Apps o Azure Kubernetes Service (AKS) o Web Apps Hands-on expertise in PySpark, SQL and Python. Good knowledge of real-time streaming applications, preferably including Kafka, Azure Functions or Azure Service Bus. Strong understanding of Spark processing ...

design and deliver cutting-edge data solutions for customers operating in some of the UK's most complex and challenging environments. Using Databricks, Apache Spark and cloud-native technologies, you’ll develop scalable Lakehouse architectures and production-grade data pipelines that enable organisations to trust, share and exploit … grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. ...

middleware, stabilise it, and begin replacing it with a cleaner, sustainable solution built around modern data tooling (Microsoft Fabric, Databricks, or equivalent, plus SQL and Power BI). Phase 2 (ongoing): Expand to include reporting. Once the core system is stable, the role broadens into wider business reporting … gather requirements from senior stakeholders across the business. Simultaneous: Bring Data Hub and Data Warehouse closer in design standards. Must-haves: Strong SQL Microsoft Fabric, Databricks, Synapse, or similar modern data platform experience Power BI (building and owning reports/dashboards, not just consuming) Comfortable working with ambiguous ...

PySpark Developer Location: Irving, TX (Hybrid 3 Days Onsite) Experience: 7+ Years Job Type: Full-Time Required Skills: PySpark, Python, Apache Spark, SparkSQL, Hadoop, SQL, Unix/Linux, Shell Scripting, and Apache Airflow. Interested candidates, please DM me or share your updated resume. VITS Consulting ...

work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve incidents, fixes + … role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver/executor logs, Spark UI + performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark ...

named, specified, and tracked — not just worked around QA coverage expands naturally with each deployment increment rather than lagging behind Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data and verify correctness Hands-on experience with Databricks — running queries … navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

your duties, you will be responsible for: Design, build, and deploy scalable data pipelines using Azure Databricks , Azure Data Factory , and Azure SQL Database . Optimise Databricks environments for performance, cost, and security. Develop and maintain data integration , ETL/ELT processes, and data quality frameworks. Translate business … environments. 8–10 years’ experience in data engineering and integration , ideally in enterprise‐scale environments. Hands‐on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working ...

your duties, you will be responsible for: Design, build and deploy scalable data pipelines using Azure Databricks , Azure Data Factory and Azure SQL Database . Optimise Databricks environments for performance, cost and security. Develop and maintain data integration , ETL/ELT processes and data quality frameworks. Translate retail … environments. 8-10 years' experience in data engineering and integration, ideally in enterprise-scale environments. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working with ...

your duties, you will be responsible for: Design, build and deploy scalable data pipelines using Azure Databricks , Azure Data Factory and Azure SQL Database . Optimise Databricks environments for performance, cost and security. Develop and maintain data integration , ETL/ELT processes and data quality frameworks. Translate retail … environments. Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse and Azure Data Lake. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working with ...

/CD and platform engineering standards using Databricks Asset Bundles (DABs), Terraform and Azure DevOps or GitHub Actions Lead migrations from legacy platforms (SQL Server, Oracle, SAS, Hadoop, Synapse) with clear cut-over and reconciliation plans Monitor and optimise DBU consumption and cluster policies, giving the client clear … lead or architect Deep Unity Catalog experience: you have configured it, not just used it, including RBAC on sensitive data Strong PySpark, Spark SQL and Delta Lake, including performance tuning at scale Production pipeline experience with Lakeflow/DLT, Workflows and DABs Desirable: Databricks Certified Data ...

data quality and governance. Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark. Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models. Monitor and troubleshoot performance issues within Fabric environments. Implement security, access controls, and governance standards. Support … architects, and stakeholders to translate requirements into technical solutions. Produce technical documentation and knowledge transfer materials. What you will have Technical Skills Strong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...

will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J-18808-Ljbffr ...

translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with ...

leading delivery teams. Strong experience designing and delivering cloud-native data platforms within Azure and/or AWS environments. Advanced experience with PySpark, SparkSQL and large-scale distributed data processing. Strong understanding of modern data architecture, ETL/ELT frameworks and data modelling. Excellent SQL skills and experience ...

shape our core modeling practices. Your qualifications and experience ESSENTIAL Pro at using SQL for data manipulation (PostgreSQL, MSSQL, Google BigQuery, SparkSQL, etc.) Strong coding skills in Python, R, Scala, C, Java, or JS Proven track record of using data manipulation and machine learning libraries Practical modelling & statistical … apprenticeship/work experience in a subject such as Mathematics, Statistics, Computer Science, or Geography DESIRABLE Exposure to Big Data Analytics (BigQuery, Apache Spark), Parallel Computing (Kubernetes, Databricks), Cloud Engineering (AWS, GCP, Azure), Spatial Query Optimisation, Graph Computing, or Microservices Relevant industry sector knowledge across retail, property ...