3 of 3 Apache Spark Jobs in Milton Keynes

warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering workflows - code generation, code review, pipeline debugging, or documentation Strong coding ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

their technical expertise. Technical competencies: Proven experience writing highly maintainable, performant Python and JavaScript code; Experience building modern data pipelines using dbt, Kafka, Spark, AWS Kinesis, AWS Lambda, and Apache Airflow (or similar); Understanding of Data Modelling patterns; Deep knowledge of complex SQL, with emphasis on Common Table ...