1 to 25 of 27 Apache Spark Jobs in County Antrim

Databricks Must Have Hands on experience on at least 2 Hyperscalers (GCP/AWS/Azure platforms) and specifically in Big Data processing services (Apache Spark, Beam or equivalent). In-depth knowledge on key technologies like Big Query/Redshift/Synapse/Pub Sub/Kinesis … years’ experience in a similar role. Ability to lead and mentor the architects. Mandatory Skills [at least 2 Hyperscalers] GCP, AWS, Azure, Big data, Apache spark, beam on BigQuery/Redshift/Synapse, Pub Sub/Kinesis/MQ/Event Hubs, Kafka Dataflow/Airflow/ADF More ❯

Engineer or similar role, focusing on data infrastructure management Proficiency in data technologies, such as relational databases, data warehousing, big data platforms (e.g., Hadoop, Spark), data streaming (e.g., Kafka), and cloud services (e.g., AWS, GCP, Azure) Programming skills in Python, Java, or Scala, with automation and scripting experience Experience More ❯

similar role, with a focus on data infrastructure management Proficiency in data technologies, such as relational databases, data warehousing, big data platforms (e.g., Hadoop, Spark), data streaming (e.g., Kafka), and cloud services (e.g., AWS, GCP, Azure). Ideally some programming skills in languages like Python, Java, or Scala, with More ❯

and statistical tools. Strong analytical and problem-solving skills. Experience with social media analytics and user behavior. Familiarity with big data tools like Hadoop, Spark, Kafka. Knowledge of AWS ML services such as SageMaker and Comprehend. Understanding of data governance and security in AWS. Excellent communication and teamwork skills. More ❯

have: Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Perform tasks such as writing scripts, extracting data using APIs, writing SQL queries etc. Work closely with other engineering teams to integrate data More ❯

Skills: x5 + experience with Python programming for data engineering tasks Strong proficiency in SQL and database management Hands-on experience with Databricks and Apache Spark Familiarity with Azure cloud platform and related services Knowledge of data security best practices and compliance standards Excellent problem-solving and communication More ❯

ETL processes, and data lakes using Databricks. Demonstrable experience with cloud services, all three public clouds (AWS, GCP, Azure) is beneficial. Solid understanding of Spark architecture, distributed computing, and cloud-based data engineering principles. Proficiency in programming languages such as Python or SQL. Strong leadership skills with experience managing More ❯

Scala Knowledge of two or more common Cloud ecosystems (Azure, AWS, GCP) with expertise in at least one. Deep experience with distributed computing with Apache Spark Working knowledge CI/CD for production deployments Working knowledge of MLOps Familiarity with designing and deploying performant end-to-end data More ❯

will: Design and deploy production data pipelines from ingestion to consumption within a big data architecture. Work with technologies such as Python, Java, Scala, Spark, and SQL to extract, clean, transform, and integrate data. Build scalable solutions using AWS services like EMR, Glue, Redshift, Kinesis, Lambda, and DynamoDB. Process More ❯

skills and a collaborative mindset Agile development experience in a team setting Bonus Skills (nice to have) Experience with big data tools like Hadoop, Spark, or Scala Exposure to fraud, payments , or financial services platforms Understanding of cloud-native development and container orchestration Knowledge of test-driven development and More ❯

data architecture , including data modeling, warehousing, real-time and batch processing, and big data frameworks. Proficiency with modern data tools and technologies such as Spark, Databricks, Kafka, or Snowflake (bonus). Knowledge of cloud security, networking, and cost optimization as it relates to data platforms. Experience in total cost More ❯

in technical communication with both business stakeholders and technical peers. Experience working with big data concepts, strategies, methodologies, and tools such as MongoDB, Snowflake, Spark, or Hadoop. Knowledge and experience of deploying enterprise scale data science products. Experience in coaching and mentoring team members. Experience and skills we'd More ❯

experience in software development, with at proven experience in data integration/data pipeline developments. Proven experience in Data integration development (Ab Initio, Talend, Apache spark, AWS Glue, SSIS or equivalent) including optimization, tuning and benchmarking Proven experience in SQL Oracle, MSSQL and equivalents including optimization, tuning and More ❯

as ITRS Geneos, AppDynamics Good Experience with Log Aggregation tools such as ELK, Splunk, Grafana(GEM) is preferred Experience working with Oracle Database, Hadoop, Apache Spark, Hive, Starburst Experience with Middleware solutions such as Tibco EMS, Kafka Good written and verbal communication skills What we can offer you More ❯

AI design, build, deployment or management Proficiency or certification in Microsoft Office tools, as well as relevant technologies such as Python, TensorFlow, Jupiter Notebook, Spark, Azure Cloud, Git, Docker and/or any other relevant technologies Strong analytical and problem-solving skills, with the ability to work on complex More ❯

AI design, build, deployment or management Proficiency or certification in Microsoft Office tools, as well as relevant technologies such as Python, TensorFlow, Jupiter Notebook, Spark, Azure Cloud, Git, Docker and/or any other relevant technologies Strong analytical and problem-solving skills, with the ability to work on complex More ❯

dynamic data models, ensuring adaptability based on evolving datasets. - Implement caching and filtering techniques to improve query performance. - Utilize big data technologies such as Apache Spark for large-scale data processing. - Apply AWS expertise, including Infrastructure-as-Code (IaC) for scalable deployment. - Work with SQL transpilers and predicate More ❯

Strong skills in Python and Pandas, with experience in converting and optimising CPU-based models and algorithms to run efficiently on GPUs. Familiarity with Spark and/or PySpark for handling large-scale data processing tasks. Deep understanding of machine learning techniques and approaches, ensuring best practices in model More ❯

Reliable, Relevant, Consistent, Complete, Scalable, Timely, Secure, Nimble. Olympus is built on Big data platform and technologies under Cloudera distribution like HDFS, Hive, Impala, Spark, YARN, Sentry, Oozie, Kafka. Our team interfaces with a vast client base and works in close partnership with Operations, Development and other technology counterparts … escalating, managing and reporting control issues with transparency. Skills & Qualifications: Working knowledge of various components and technologies under Cloudera distribution like HDFS, Hive, Impala, Spark, YARN, Sentry, Oozie, Kafka. Very good knowledge on analyzing the bottlenecks on the cluster - performance tuning, effective resource usage, capacity planning, investigating. Perform daily More ❯

Belfast. Your day-to-day -Wrangle and draw meaningful insights from massive amounts of unstructured textual data using the latest tools and technologies like Spark, Iceberg, Athena, AWS SageMaker -Apply unsupervised learning algorithms across billions of email interactions to identify emerging threat patterns -Work with state-of-the-art More ❯

deployment pipelines Proficient in cloud AI services (AWS SageMaker/Bedrock) Deep understanding of distributed systems and microservices architecture Expert in data pipeline platforms (Apache Kafka, Airflow, Spark) Proficient in both SQL (PostgreSQL, MySQL) and NoSQL (Elasticsearch, MongoDB) databases Strong containerization and orchestration skills (Docker, Kubernetes) Experience with More ❯

deployment pipelines Proficient in cloud AI services (AWS SageMaker/Bedrock) Deep understanding of distributed systems and microservices architecture Expert in data pipeline platforms (Apache Kafka, Airflow, Spark) Proficient in both SQL (PostgreSQL, MySQL) and NoSQL (Elasticsearch, MongoDB) databases Strong containerization and orchestration skills (Docker, Kubernetes) Experience with More ❯

experience in an Application Development role. Demonstrated execution capabilities. Strong analytical and quantitative skills; Data driven and results-oriented Experience with Core Java required (Spark a plus) Experience with SQL Experience working with Hadoop, Hive, Sqoop and other technologies in Cloudera's CDP distribution. Understanding of version control (git … working as part of an agile team. Excellent written and oral communication skills Technical Skills: Strong knowledge in Java Some knowledge inHadoop, hive, SQL, Spark Understanding of Unix Shell Scripting CI/CD Pipeline Maven or Gradle experience Predictive analytics (desirable) PySpark (desirable) Trade Surveillance domain knowledge (desirable) Education More ❯

Role: Data Engineer Location: Belfast Duration: Long Term Contract Opportunity - Rolling Contract Rate: Market Rates - Inside IR35 Job Description : Fujitsu's Decision Intelligence practice in the UK helps organisations bridge the gap between data and insights, empowering businesses to make More ❯

Reliable, Relevant, Consistent, Complete, Scalable, Timely, Secure, Nimble. Olympus is built on Big data platform and technologies under Cloudera distribution like HDFS, Hive, Impala, Spark, YARN, Sentry, Oozie, Kafka. Our team interfaces with a vast client base and works in close partnership with Operations, Development and other technology counterparts … role. Hands-on experience in supporting applications built in Hadoop. Working knowledge of various components and technologies under Cloudera distribution like HDFS, Hive, Impala, Spark, YARN, Sentry, Oozie, Kafka. Experienced in Linux Very good knowledge on analyzing the bottlenecks on the cluster - performance tuning, effective resource usage, capacity planning More ❯