1 to 25 of 33 Apache Spark Jobs in Northern Ireland

data pipelines to serve the easyJet analyst and data science community. Highly competent hands-on experience with relevant Data Engineering technologies, such as Databricks, Spark, Spark API, Python, SQL Server, Scala. Work with data scientists, machine learning engineers and DevOps engineers to develop, develop and deploy machine learning … development experience with Terraform or CloudFormation. Understanding of ML development workflow and knowledge of when and how to use dedicated hardware. Significant experience with Apache Spark or any other distributed data programming frameworks (e.g. Flink, Hadoop, Beam) Familiarity with Databricks as a data and AI platform or the … data privacy, handling of sensitive data (e.g. GDPR) Experience in event-driven architecture, ingesting data in real time in a commercial production environment with Spark Streaming, Kafka, DLT or Beam. Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options More ❯

Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis … years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF More ❯

Engineer or similar role, focusing on data infrastructure management Proficiency in data technologies, such as relational databases, data warehousing, big data platforms (e.g., Hadoop, Spark), data streaming (e.g., Kafka), and cloud services (e.g., AWS, GCP, Azure) Programming skills in Python, Java, or Scala, with automation and scripting experience Experience More ❯

similar role, with a focus on data infrastructure management Proficiency in data technologies, such as relational databases, data warehousing, big data platforms (e.g., Hadoop, Spark), data streaming (e.g., Kafka), and cloud services (e.g., AWS, GCP, Azure). Ideally some programming skills in languages like Python, Java, or Scala, with More ❯

applications through bug fixes and code refactoring. Leverage the latest data technologies and programming languages, including Python, Scala, and Java, along with systems like Spark, Kafka, and Airflow, within cloud services such as AWS. Ensure the ongoing maintenance, troubleshooting, optimization, and reliability of data systems, including timely resolution of … NoSQL databases (e.g., PostgreSQL, MongoDB) and data modeling principles. Proven ability to design, build, and maintain scalable data pipelines and workflows using tools like Apache Airflow or similar. Strong problem-solving and analytical skills. Excellent communication and collaboration skills. Nice to have: Hands-on experience with data warehouse and … lakehouse architectures (e.g., Databricks, Snowflake, or similar). Experience with big data frameworks (e.g., Apache Spark, Hadoop) and cloud platforms (e.g., AWS, Azure, or GCP). More ❯

applications through bug fixes and code refactoring. Leverage the latest data technologies and programming languages, including Python, Scala, and Java, along with systems like Spark, Kafka, and Airflow, within cloud services such as AWS. Ensure the ongoing maintenance, troubleshooting, optimization, and reliability of data systems, including timely resolution of … NoSQL databases (e.g., PostgreSQL, MongoDB) and data modeling principles. Proven ability to design, build, and maintain scalable data pipelines and workflows using tools like Apache Airflow or similar. Strong problem-solving and analytical skills. Excellent communication and collaboration skills. Nice to have: Hands-on experience with data warehouse and … lakehouse architectures (e.g., Databricks, Snowflake, or similar). Experience with big data frameworks (e.g., Apache Spark, Hadoop) and cloud platforms (e.g., AWS, Azure, or GCP). More ❯

applications through bug fixes and code refactoring. Leverage the latest data technologies and programming languages, including Python, Scala, and Java, along with systems like Spark, Kafka, and Airflow, within cloud services such as AWS. Ensure the ongoing maintenance, troubleshooting, optimization, and reliability of data systems, including timely resolution of … NoSQL databases (e.g., PostgreSQL, MongoDB) and data modeling principles. Proven ability to design, build, and maintain scalable data pipelines and workflows using tools like Apache Airflow or similar. Strong problem-solving and analytical skills. Excellent communication and collaboration skills. Nice to have: Hands-on experience with data warehouse and … lakehouse architectures (e.g., Databricks, Snowflake, or similar). Experience with big data frameworks (e.g., Apache Spark, Hadoop) and cloud platforms (e.g., AWS, Azure, or GCP). More ❯

communication skills and demonstrated ability to engage with business stakeholders and product teams. Experience in data modeling , data warehousing (e.g., Snowflake , AWS Glue , EMR , Apache Spark ), and working with data pipelines . Leadership experience—whether technical mentorship, team leadership, or managing critical projects. Familiarity with Infrastructure as Code More ❯

have: Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Perform tasks such as writing scripts, extracting data using APIs, writing SQL queries etc. Work closely with other engineering teams to integrate data More ❯

Skills: x5 + experience with Python programming for data engineering tasks Strong proficiency in SQL and database management Hands-on experience with Databricks and Apache Spark Familiarity with Azure cloud platform and related services Knowledge of data security best practices and compliance standards Excellent problem-solving and communication More ❯

ETL processes, and data lakes using Databricks. Demonstrable experience with cloud services, all three public clouds (AWS, GCP, Azure) is beneficial. Solid understanding of Spark architecture, distributed computing, and cloud-based data engineering principles. Proficiency in programming languages such as Python or SQL. Strong leadership skills with experience managing More ❯

Scala Knowledge of two or more common Cloud ecosystems (Azure, AWS, GCP) with expertise in at least one. Deep experience with distributed computing with Apache Spark Working knowledge CI/CD for production deployments Working knowledge of MLOps Familiarity with designing and deploying performant end-to-end data More ❯

Scala Knowledge of two or more common Cloud ecosystems (Azure, AWS, GCP) with expertise in at least one. Deep experience with distributed computing with Apache Spark Working knowledge CI/CD for production deployments Working knowledge of MLOps Familiarity with designing and deploying performant end-to-end data More ❯

salesforce, hubspot, SaaS, data lakes, apis, adtech, GDPR, CCPA, B2B, Sales Consultant, Account Manager, DataOps, CICD, DevOPs, CI/CD, GenAI, RAG, AWS, Azure, Apache, Spark, Kafka – UK Wide - £80,000-£95,000 + OTE Our large Microsoft Partner client requires a Senior Data Analytics Sales Consultant to More ❯

will: Design and deploy production data pipelines from ingestion to consumption within a big data architecture. Work with technologies such as Python, Java, Scala, Spark, and SQL to extract, clean, transform, and integrate data. Build scalable solutions using AWS services like EMR, Glue, Redshift, Kinesis, Lambda, and DynamoDB. Process More ❯

data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost optimization as it relates to data platforms. Experience in total cost More ❯

problem structuring, analysis, and communication Presenting outcomes to senior management and stakeholders Responsibilities Extracting data using programming languages such as SQL, SAS, Python, or Spark/Scala from various database systems Transforming and connecting data across sources to generate meaningful insights Performing quantitative analysis to answer business questions and More ❯

in technical communication with both business stakeholders and technical peers. Experience working with big data concepts, strategies, methodologies, and tools such as MongoDB, Snowflake, Spark, or Hadoop. Knowledge and experience of deploying enterprise scale data science products. Experience in coaching and mentoring team members. Experience and skills we'd More ❯

experience in software development, with at proven experience in data integration/data pipeline developments. Proven experience in Data integration development (Ab Initio, Talend, Apache spark, AWS Glue, SSIS or equivalent) including optimization, tuning and benchmarking Proven experience in SQL Oracle, MSSQL and equivalents including optimization, tuning and More ❯

as ITRS Geneos, AppDynamics Good Experience with Log Aggregation tools such as ELK, Splunk, Grafana(GEM) is preferred Experience working with Oracle Database, Hadoop, Apache Spark, Hive, Starburst Experience with Middleware solutions such as Tibco EMS, Kafka Good written and verbal communication skills What we can offer you More ❯

AI design, build, deployment or management Proficiency or certification in Microsoft Office tools, as well as relevant technologies such as Python, TensorFlow, Jupiter Notebook, Spark, Azure Cloud, Git, Docker and/or any other relevant technologies Strong analytical and problem-solving skills, with the ability to work on complex More ❯

AI design, build, deployment or management Proficiency or certification in Microsoft Office tools, as well as relevant technologies such as Python, TensorFlow, Jupiter Notebook, Spark, Azure Cloud, Git, Docker and/or any other relevant technologies Strong analytical and problem-solving skills, with the ability to work on complex More ❯

Broad career progression opportunities across Group, including several high-profile household names What you'll bring: Experience with Cloud and big data technologies (e.g., Spark/Databricks/Delta Lake/BigQuery) Familiarity with eventing technologies (e.g., Event Hubs/Kafka) and file formats such as Parquet/Delta More ❯

Strong skills in Python and Pandas, with experience in converting and optimising CPU-based models and algorithms to run efficiently on GPUs. Familiarity with Spark and/or PySpark for handling large-scale data processing tasks. Deep understanding of machine learning techniques and approaches, ensuring best practices in model More ❯

Reliable, Relevant, Consistent, Complete, Scalable, Timely, Secure, Nimble. Olympus is built on Big data platform and technologies under Cloudera distribution like HDFS, Hive, Impala, Spark, YARN, Sentry, Oozie, Kafka. Our team interfaces with a vast client base and works in close partnership with Operations, Development and other technology counterparts … escalating, managing and reporting control issues with transparency. Skills & Qualifications: Working knowledge of various components and technologies under Cloudera distribution like HDFS, Hive, Impala, Spark, YARN, Sentry, Oozie, Kafka. Very good knowledge on analyzing the bottlenecks on the cluster - performance tuning, effective resource usage, capacity planning, investigating. Perform daily More ❯