5 of 5 Apache Spark Jobs in Edinburgh

techniques, including machine learning and their application in financial services.* Proficiency in data tools, programming languages, and database technologies, in particular in Azure Databricks, Spark, Spark SQL and Python/pyspark.* Understanding of data governance, security, and data quality management principles.* Experience working with large, complex datasets … producing audit evidence**Essential Skills:*** Data pipeline development and optimisation* Data modelling and architecture (in Azure Databricks)* Programming and scripting for data processing (Spark, Spark SQL, Python, pyspark)* Data science and machine learning* Data governance and quality assurance* Problem-solving and analytical thinking* Stakeholder engagement and technical communication ...

processes. Technical Skills Programming: Proficiency in Python, Java, Scala, or similar languages. Big Data Technologies: Hands-on experience with big data tools e.g. (Databricks, Apache Spark, Hadoop). Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Expertise in relational databases (e.g. … postgres, sql server) Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment. Experience ...

have Programming: Proficiency in Python, Java, Scala, or similar languages for data processing. Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop. Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Database Management: Expertise in relational … databases (e.g. Postgres, SQL Server). Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines ...

contribute to best practice engineering across cross-functional teams. Based in Scotland with hybrid work from Edinburgh or Glasgow, you’ll work with DeltaLake, Spark, Kafka, and Unity Catalog, applying DevOps ideas to CI/CD pipelines and GDPR-compliant data #J-18808-Ljbffr ...

need to potentially work in either Edinburgh or Glasgow depending on client expectations. What You’ll Do Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog Build real‐time event‐driven solutions with tools such as Kafka and Azure Event Hubs Apply DevOps principles … knowledge of DevOps tools and CI/CD processes Bonus Points For Development experience in Scala or Java Familiarity with Cloudera, Hadoop, HIVE, and Spark ecosystem Understanding of data privacy regulations, including GDPR, and experience working with sensitive data Ability to learn and adapt new technologies quickly to meet ...