4 of 4 Apache Spark Jobs in Bristol

data processing solutions using BigQuery, Pub/Sub, Dataflow and Dataproc Experience developing end‐to‐end solutions using batch and streaming frameworks such as Apache Spark and Apache Beam. Expert understanding of when to use a range of data storage technologies including relational/non‐relational, document ...

associated Azure technologies. Build scalable ETL/ELT pipelines to ingest, transform, and load data from multiple enterprise systems. Develop data transformations using PySpark, Spark SQL, SQL, and Python. Create and maintain curated datasets that support business intelligence, reporting, and analytics. Implement data quality, reconciliation, validation, and monitoring processes. … engineering or business intelligence solutions. Strong proven commercial experience with Databricks. Strong experience developing ETL/ELT pipelines and data integration solutions. Experience using Apache Spark, PySpark, and Spark SQL. Strong SQL development and optimisation skills. Experience working with large and complex datasets from multiple source systems. ...

ResponsibilitiesData Pipeline DevelopmentData Platform ManagementStakeholder CollaborationLeadership & MentorshipTechnology StackThe successful candidate will have experience working with some or all of the following technologies:Data Processing: Apache Spark/PySparkWorkflow Orchestration: Apache AirflowContainerisation: KubernetesCloud Platform: Microsoft AzureData APIs & Semantic Layer: CubeJSProgramming Languages: Python, SQLEssential Skills & ExperienceDegree in a STEM ...

architecture (SQL and NoSQL)* Experience with cloud technologies* Experience with cloud platforms* Experience with CI/CD practices* Experience with big data tools e.g. Spark, Hadoop, NiFi* Experience managing direct reports, providing regular guidance and performance feedback* Ability to create delivery plans with estimated timelines for personal and team ...