14 of 14 Spark SQL Jobs

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines … support code reviews, design reviews, and engineering best practices. Your Skills: Strong hands-on experience with Databricks on AWS. Strong experience with Apache Spark/PySpark. Excellent programming skills in Python. Strong SQL skills including complex queries, joins, CTEs, window functions, and query optimization. Experience with ...

department. This is an exciting opportunity to work on cutting edge technologies including Big Data and NoSQL databases (Hadoop, HBase, Hive, Spark and MongoDB) to allow the business to gain advanced insight into their portfolios and valuation metrics. Key responsibilities include:* Lead business requirements elicitation, analysis, and documentation … efforts* Assist technology team in Azure cloud solution implementation; write Spark SQL queries to analyze and transform data; provide feedback to User Interface team; testing and documentation* Analyze solution requirements; understand project scope; determine project priorities; ensure efficient and on-time delivery of project tasks ...

strategic asset that's as beautiful as it is actionable. If you dream in data layers and speak the languages of both SQL and stakeholder, this is your stage. Who We Are: The Architects of What Connects Us You find the signal in the noise of data. … engineering teams monitor, measure, and optimize the performance of the very networks we design. Automate Intelligence: Unleash the power of Tableau, Alteryx, SQL, and Python/R to automate complex analysis, transforming raw data streams into a wellspring of proactive, business-ready insights-freeing our minds from manual ...

solutions. Create and maintain technical documentation, standards, and knowledge repositories. Your Profile: Essential skills/knowledge/experience Strong proficiency in Python, PySpark, Spark SQL, DataFrame APIs, Structured Streaming, and Java. Experience developing and maintaining large-scale data processing and mastering pipelines using PySpark. Strong experience … GitLab CI, Azure DevOps, or similar. Experience working with YAML and JSON configuration files and Infrastructure-as-Code concepts. Strong experience with Spark for large-scale data processing using DataFrames, RDDs, and Spark SQL. Hands-on experience developing ETL workflows, lakehouse solutions, dataflows, pipelines, and semantic ...

high-stakes and the upside of getting it right is visible at company level. You'll design and deploy solutions in Python, SQL and Spark SQL, working across Azure Synapse, Snowflake, Databricks and Azure ML. The environment is modern and evolving. You'll help … afterthought Strong candidates will come from a background in data science or applied statistics, with hands-on ML experience and the Python/SQL depth to match. Experience with Azure tooling is useful; the ability to think clearly about business problems and turn them into scalable analytical solutions ...

bring: Extensive data engineering/integration/migration experience Hands-on Azure Data Factory, Fabric Data Factory, or Synapse Pipelines Strong SQL - profiling, transformation, validation, reconciliation PySpark/Spark SQL notebook experience (Databricks or Fabric) Proven experience on a data migration project ...

experience designing canonical data models or enterprise logical data models. Strong understanding of data modelling techniques, including conceptual, logical and physical modelling. Strong SQL skills and ability to interrogate complex relational data structures. Experience working with cloud data platforms, ideally AWS and Databricks. Understanding of modern data architecture … platforms. Experience with regulatory, client or operational reporting. Experience supporting data migration, legacy system modernisation or platform replacement. Experience with Databricks, Delta Lake, Spark SQL or PySpark. Experience defining validation frameworks, data quality dashboards or metadata-driven rule engines. Experience working in consultancy or client-facing ...

ways of working, supporting system enhancements, and ensuring Finance is equipped with modern, resilient data capabilities. Role Responsibilities Design, develop, and maintain SQL data structures within Microsoft Fabric, writing efficient SQL queries and Python-based transformations to support scalable data pipelines and optimise performance across Lakehouse … reports, dashboards, and visualisations. Troubleshoot database-related issues. Monitor and optimize database performance, identifying and resolving bottlenecks and inefficiencies. Conduct regular reviews of SQL code and queries for performance improvements Ensure data security and integrity. Stay updated with the latest database technologies and best practices. Data Warehouse Implement ...

emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building ...

7+ years Databricks solution design, implementation, engineering experience required Either 5+ years building data pipelines in Databricks using PySpark, Scala and/or Spark SQL and ability to work across structured, semi-structured and unstructured data OR 5+ years ML Engineering or AI Engineering deploying, promoting ...

operate Databricks pipelines using Python, Lakeflow Declarative Pipelines, and Jobs/Workflows Use Azure Data Factory for ingestion/orchestration; integrate and optimize SQL/NoSQL databases Ensure pipeline quality via versioning, testing, monitoring, SLAs/SLOs, and automated deployments Manage platform costs and pipeline performance. Support establishment … group-wide use Profil Degree in Computer Science, Software Engineering, Data Engineering/Data Science, or equivalent experience Hands-on with Databricks (Python, Spark SQL, Lakehouse), Declarative Pipelines, and Jobs/Workflows Experience with Azure Data Factory, data modeling, performance tuning, and production data systems DataOps ...

engineering, and modern Lakehouse solutions. The client is looking for a hands‐on engineering professional with strong Microsoft Fabric, Spark, PySpark and SparkSQL expertise, capable of designing, building and supporting enterprise‐scale data platforms. Start Date: ASAP Duration: 3 Months Initially Pay Rate: Competitive IR35 Status: Outside IR35 … Location: London (1 day per week onsite, remainder remote) The Role: The core requirement is strong Lakehouse expertise, particularly Spark, PySpark, and SparkSQL. Hands‐on experience in low‐code is essential. Data & Analytics Expertise: Data architecture (Lakehouse, Medallion, Kimball, 3NF). Data ingestion and processing. Data quality frameworks ...

leading delivery teams. Strong experience designing and delivering cloud-native data platforms within Azure and/or AWS environments. Advanced experience with PySpark, SparkSQL and large-scale distributed data processing. Strong understanding of modern data architecture, ETL/ELT frameworks and data modelling. Excellent SQL skills and experience ...

focused on data platforms, analytics engineering and Lakehouse solutions. You will design, build and support enterprise-scale data platforms with Spark, PySpark, SparkSQL and Microsoft Fabric. The role requires hands-on coding, data governance, ingestion and API integration capabilities, plus a passion for modern analytics and low-code ...