26 to 50 of 57 Permanent Spark SQL Jobs

processes and solutions. Lead the execution of Microsoft Fabric migration activities. Lead and guide migration engineers and QA resources. Develop Python, PySpark and Spark SQL code for data validation and curation within notebooks. Create data integration test cases and support testing activities. Implement and amend data … Azure Functions o Azure App Service o Logic Apps o Azure Kubernetes Service (AKS) o Web Apps Hands-on expertise in PySpark, SQL and Python. Good knowledge of real-time streaming applications, preferably including Kafka, Azure Functions or Azure Service Bus. Strong understanding of Spark processing ...

design and deliver cutting-edge data solutions for customers operating in some of the UK's most complex and challenging environments. Using Databricks, Apache Spark and cloud-native technologies, you’ll develop scalable Lakehouse architectures and production-grade data pipelines that enable organisations to trust, share and exploit … grow your career while helping customers unlock the power of their data. Responsibilities Design, build and optimise scalable data pipelines using Databricks, Apache Spark, PySpark and Delta Lake. Develop and implement modern Lakehouse architectures, applying Medallion (Bronze, Silver, Gold) design principles to support trusted and reusable data products. ...

Security In-depth knowledge of the financial services industry and their IT systems Practical cloud native experience Preferred qualifications, capabilities, and skills Advanced Apache Spark experience(PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream ...

Application Resiliency, and SecurityIn-depth knowledge of the financial services industry and their IT systemsPractical cloud native experiencePreferred qualifications, capabilities, and skillsAdvanced Apache Spark experience (PySpark/Spark SQL), including performance tuning (partitioning, shuffle optimization, caching), troubleshooting, and designing scalable batch/stream processing ...

middleware, stabilise it, and begin replacing it with a cleaner, sustainable solution built around modern data tooling (Microsoft Fabric, Databricks, or equivalent, plus SQL and Power BI). Phase 2 (ongoing): Expand to include reporting. Once the core system is stable, the role broadens into wider business reporting … gather requirements from senior stakeholders across the business. Simultaneous: Bring Data Hub and Data Warehouse closer in design standards. Must-haves: Strong SQL Microsoft Fabric, Databricks, Synapse, or similar modern data platform experience Power BI (building and owning reports/dashboards, not just consuming) Comfortable working with ambiguous ...

PySpark Developer Location: Irving, TX (Hybrid 3 Days Onsite) Experience: 7+ Years Job Type: Full-Time Required Skills: PySpark, Python, Apache Spark, SparkSQL, Hadoop, SQL, Unix/Linux, Shell Scripting, and Apache Airflow. Interested candidates, please DM me or share your updated resume. VITS Consulting ...

partner Jooble Skills Required Good understanding of Data Lakes, Data Warehouses. Software development, coding expertise in the Data Science field using Python/Spark/Scala/R/Java. Any visualization experience is an advantage. Experience using ANSI SQL and Spark SQL ...

automation programs (preferably in financial services). Strong proficiency in Python for data engineering and automation. Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet. Advanced SQL skills; experience with Oracle databases. Experience designing ...

structured governance. Strong domain understanding of Capital Markets data and reporting expectations (front-to-back awareness is a plus). Proficiency in SQL (advanced querying, performance tuning, reconciliation logic). Strong proficiency in Python for data analysis and automation (pandas, validation frameworks, scripting). Experience supporting or validating … outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Exposure to data governance concepts: critical data elements (CDEs), lineage, data quality dimensions, audit ...

with challenges Proactive, with a solution-oriented mindset Experience Experience Proven experience in data activities Experience with Microsoft Fabric including Data Factory, OneLake, Spark Knowledge of DataFrames, Partitioning, Performance Tuning, Notebook Development, Spark SQL, Delta Tables, Data Modelling, Semantic Models Good customer service, communication ...

plus). Familiarity with risk appetite concepts as applied to data quality thresholds and control exceptions. Technical/Analytical Skills Proficiency in SQL (advanced querying, reconciliation logic, data validation). Strong proficiency in Python for data analysis and automation (pandas, validation frameworks, scripting). Experience supporting or validating … outputs such as Tableau dashboards (data validation, extract refresh checks, reconciliation to source). Nice-to-Have Experience with tools such as PySpark, Spark SQL, Hive, Impala, HDFS, Parquet, and Oracle databases. Hands-on exposure to DCRM tooling and operational exception management processes. Experience with governance ...

work with a leading global IT transformation on a large-scale Government project: Data Engineer with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on Data Engineer to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve incidents, fixes + … role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver/executor logs, Spark UI + performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark ...

relevant in data governance, compliance, or automation programs. Strong proficiency in Python for data engineering and automation. Hands‐on experience with PySpark and Spark SQL in production environments. Solid knowledge of Hive, Impala, HDFS, and Parquet . Advanced SQL skills; experience with Oracle databases ...

named, specified, and tracked — not just worked around QA coverage expands naturally with each deployment increment rather than lagging behind Technical Strong SQL skills — comfortable writing and reading complex analytical queries (window functions, CTEs, aggregations) to interrogate data and verify correctness Hands-on experience with Databricks — running queries … navigating Unity Catalog, reading Spark job outputs and understanding what they mean for data quality Working knowledge of PySpark or Spark SQL — enough to read pipeline code, understand transformations, and trace where data issues originate Understanding of Lakehouse/medallion architecture (bronze-silver-gold ...

your duties, you will be responsible for: Design, build, and deploy scalable data pipelines using Azure Databricks , Azure Data Factory , and Azure SQL Database . Optimise Databricks environments for performance, cost, and security. Develop and maintain data integration , ETL/ELT processes, and data quality frameworks. Translate business … environments. 8–10 years’ experience in data engineering and integration , ideally in enterprise‐scale environments. Hands‐on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working ...

your duties, you will be responsible for: Design, build and deploy scalable data pipelines using Azure Databricks , Azure Data Factory and Azure SQL Database . Optimise Databricks environments for performance, cost and security. Develop and maintain data integration , ETL/ELT processes and data quality frameworks. Translate retail … environments. 8-10 years' experience in data engineering and integration, ideally in enterprise-scale environments. Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration. Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working with ...

/CD and platform engineering standards using Databricks Asset Bundles (DABs), Terraform and Azure DevOps or GitHub Actions Lead migrations from legacy platforms (SQL Server, Oracle, SAS, Hadoop, Synapse) with clear cut-over and reconciliation plans Monitor and optimise DBU consumption and cluster policies, giving the client clear … lead or architect Deep Unity Catalog experience: you have configured it, not just used it, including RBAC on sensitive data Strong PySpark, Spark SQL and Delta Lake, including performance tuning at scale Production pipeline experience with Lakeflow/DLT, Workflows and DABs Desirable: Databricks Certified Data ...

emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building ...

stakeholders across the enterprise to build custom solutions, owning the architecture, modeling, and cloud + data engineering. Job Details AWS Databricks ETL Python Spark SQL IaC CI/CD AI LLM Interested in hearing more? Easy Apply now by clicking the "Quick Apply" button. Jobot ...

/CD pipelines, version control, and DevOps practices related to data engineering. Proficiency in SQL (T-SQL/SparkSQL) for data manipulation, transformation, and querying. Solid understanding and practical application of Data Vault and Kimball data modelling methodologies. Experience designing and optimising ETL/ELT workflows ...

data quality and governance. Create and maintain ETL/ELT processes using Data Factory, Data Pipelines, Notebooks, and Spark. Develop, maintain, and enhance SQL scripts, stored procedures, views, and data models. Monitor and troubleshoot performance issues within Fabric environments. Implement security, access controls, and governance standards. Support … architects, and stakeholders to translate requirements into technical solutions. Produce technical documentation and knowledge transfer materials. What you will have Technical Skills Strong SQL development skills. Experience designing and implementing ETL/ELT solutions. Knowledge of Lakehouse and Data Warehouse architectures. Experience with Data Pipelines, Dataflows Gen2 ...

will lead a small team, mentor junior engineers, and collaborate with stakeholders to deliver scalable data pipelines on Databricks using PySpark and Spark SQL. The role involves guiding data strategy, migrating legacy processes, and delivering enterprise data solutions in a cloud-native environment, with hybrid #J-18808-Ljbffr ...

translate technical content in commercial terms for non-technical stakeholders, such as customers, sales and product teams Desirable B2B product experience Experience with Spark SQL, PySpark, databricks Experience with CausalAI modelling frameworks (e.g. dowhy, econml) Diversity & Inclusion Intent HQ is an equal opportunities employer with ...

leading delivery teams. Strong experience designing and delivering cloud-native data platforms within Azure and/or AWS environments. Advanced experience with PySpark, SparkSQL and large-scale distributed data processing. Strong understanding of modern data architecture, ETL/ELT frameworks and data modelling. Excellent SQL skills and experience ...

deliver measurable value. Job Details End-to-end product life cycle Python Spark/PySpark SQL/MySQL/SparkSQL Tableau Databricks AWS Glue Linear Regression - GLM - Random Forrest GenAI/LLM No sponsorship or C2C Interested in hearing more? Easy Apply now by clicking ...