1 to 25 of 29 Permanent Apache Spark Jobs in Scotland

would be advantageous. Experience with Atlan and Anomalo preferred. Utilising Cloud Agnostic Technologies : Experience designing and implementing solutions using open ecosystem technologies. Knowledge of Apache Iceberg, Apache Spark, Apache Kafka, and open table formats is highly desirable. Benefits and perks There's one thing people ...

processes. Technical Skills Programming: Proficiency in Python, Java, Scala, or similar languages. Big Data Technologies: Hands-on experience with big data tools e.g. (Databricks, Apache Spark, Hadoop). Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Expertise in relational databases (e.g. … postgres, sql server) Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment. Experience ...

with Snowflake and Databricks native solutions. Experience on Snowflake Cortex is must. Proficiency in programming skills in Python. Experience with data processing frameworks like Apache Spark or Hadoop. Knowledge of database systems (SQL and NoSQL). Familiarity with cloud platforms (AWS, Azure) and their data services. Understanding ...

have Programming: Proficiency in Python, Java, Scala, or similar languages for data processing. Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop. Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Database Management: Expertise in relational … databases (e.g. Postgres, SQL Server). Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines ...

control systems such as Git. Knowledge of cloud-based data platforms and modern data engineering practices. Desirable Skills Experience with Scala development. Experience with Apache Spark or other distributed data processing frameworks. Experience with Kafka, Event Hubs, or other streaming technologies. Exposure to cloud platforms such as Azure ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security ...

data processing. Proficiency in SQL, Python, and PySpark, with experience in query optimization and performance tuning. Hands-on experience with data lake platforms (Databricks, Apache Spark, or similar). Experience with AWS cloud services (S3, ECS, SNS/SQS, Lambda, etc.). Strong understanding of data quality, security ...

understanding of Terraform enterprise. Experience in delivering system design, application development, testing, and operational stability. Knowledge of Big Data distributed compute frameworks like Spark, Glue, MapReduce etc. Excellent troubleshooting, analytical, and communication skills. Preferred Qualifications, Capabilities, and Skills: Multi Region disaster recovery setup, monitoring and testing. Experience in Data ...

related discipline. Eligible to obtain UK Security Clearance (SC). DESIRABLE SKILLS: Experience with MLflow, Weights & Biases or similar experiment-tracking tools. Experience with Spark or large-scale data processing platforms. Knowledge of ONNX, TorchScript, FastAPI or gRPC. Experience deploying AI solutions within Azure, AWS or Google Cloud environments.Experience ...

understanding of Terraform enterprise. Experience in delivering system design, application development, testing, and operational stability. Knowledge of Big Data distributed compute frameworks like Spark, Glue, MapReduce etc. Excellent troubleshooting, analytical, and communication skills. Preferred Qualifications, Capabilities, and Skills Multi Region disaster recovery setup, monitoring and testing. Experience in Data ...

with data warehousing concepts and dimensional modeling. Strong experience in cloud platforms such as AWS/Azure/GCP. Experience with big data technologies (Spark, Hadoop, Kafka). Knowledge of API integrations and microservices-based data ingestion. Experience working in Agile/Scrum environments. Banking Domain Experience (Mandatory) Experience ...

continuous delivery (CI/CD) pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design. Degree in STEM ...

continuous delivery (CI/CD) pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design. Degree in STEM ...

build tools (Maven/Gradle). Desirable skills: Scala: Hands-on experience or a strong functional programming background using Scala. Big Data: Exposure to Apache Spark, Kafka, or Hadoop ecosystems. Cloud: Experience with AWS, Azure, or Google Cloud Platform. Experience in banking or financial services industry. Soft Skills ...

restore methods, DDL generation and manipulation, database access management, schema and object-level high-performance data movement. Build solutions with pushdown optimization, ELT, Spark-based data processing and streaming, replication, partitioning, clustering, change data capture, and time travel. What you'll bring to the role At least 4 years … scripts, with a particular focus on building and optimizing multi-threaded, multi-process, concurrent, and asynchronous solutions. Experience with distributed computing tools (such as Apache Spark for Python) is a plus. Good to have experience in designing and developing AI-powered solutions using Large Language Models (LLMs), retrieval ...

inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworks Familiarity with AWS-native data services such as Glue, Lake Formation, Redshift, or Athena Exposure to data governance ...

Candidate Requirements: Extensive experience building and deploying machine learning models in production, with strong hands-on Python capability. Experience with distributed computing frameworks (e.g., Spark or similar) and scalable data processing environments. Exposure to cloud-based platforms (e.g., AWS, Azure, GCP or Databricks) and modern ML/MLOps practices. ...

functional and object-oriented programming skills with expertise in languages such as Scala, Java, and Python; and open-source frameworks/libraries, such as Apache Spark. Familiar with software engineering and project management tools such as Git, Jira, and Agile. Excellent problem-solving, analytical, and debugging skills. Ability ...

secure and scalable solutions Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with ...

secure and scalable solutions Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with ...

scaling patterns and reuse. Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Deep hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with ...

team. What We’re Looking For Proficient in one of Python, Scala or Java with strong experience in Big Data technologies such as: Spark, Hadoop etc. Practical knowledge of building real-time event streaming pipelines (e.g., Kafka, Spark Streaming, Kinesis). Proficiency in AWS cloud environments. Proven experience … Bonus Points For Expertise in Data Modelling, schema design, and handling both structured and semi-structured data. Familiarity with distributed systems such as Hadoop, Spark, HDFS, Hive, Databricks. Exposure to AWS Lake Formation and automation of ingestion and transformation layers. Background in delivering solutions for highly regulated industries. Passion ...

problem-solving and analytical skills Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Hands-on expertise with Spark/PySpark and big data processing technologies Knowledge of open-source table formats and catalog services such as Apache Iceberg Experience with ...

prototype to production - training, packaging, deployment, monitoring, retraining, and decommissioning - in real-world business applications. Proven experience working with large datasets and distributed compute (Spark/Databricks or equivalent), with SQL fluency and an understanding of partitioning, performance, and cost. Hands-on experience using enterprise-authorized AI-assisted software … Application Resiliency, and Security. Preferred qualifications, capabilities, and skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Strong working knowledge of both supervised and unsupervised techniques, including tree-based and kernel methods (gradient boosting, random forest, SVM) and anomaly-detection ...

problem-solving and analytical skills Preferred qualifications, capabilities, and skills Experience with modern data platforms such as Databricks or Snowflake Hands-on expertise with Spark/PySpark and big data processing technologies Knowledge of open-source table formats and catalog services such as Apache Iceberg Experience with ...