1 to 25 of 46 Apache Spark Jobs in the North West

data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. The data engineer is proficient in Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure functions with Python, Azure Purview, and Cosmos DB. They are also proficient in Azure Event Hub and Streaming Analytics, Managed Streaming for Apache Kafka, Azure … DataBricks with Spark, and other open source technologies like Apache Airflow and dbt, Spark/Python, or Spark/Scala. Preferred Education Bachelor's Degree Required Technical And Professional Expertise Commercial experience as a Data Engineer or similar role, with a strong emphasis on Azure technologies. Proficiency in Azure data services (Azure SQL Database, Azure Synapse … Analytics, Azure Data Factory, Azure Databricks). Experience with data modeling, data warehousing, and big data processing (Hadoop, Spark, Kafka). Strong understanding of SQL and NoSQL databases, data modeling, and ETL/ELT processes. Proficiency in at least one programming language (Python, C#, Java). Experience with CI/CD pipelines and tools (Azure DevOps, Jenkins). Knowledge More ❯

maintain the data platform. You will be working on complex data problems in a challenging and fun environment, using some of the latest Big Data open-source technologies like Apache Spark, as well as Amazon Web Service technologies including Elastic MapReduce, Athena and Lambda to develop scalable data solutions. Key Responsibilities: Adhering to Company Policies and Procedures with … and specifications. Good interpersonal skills, positive attitude, willing to help other members of the team. Experience debugging and dealing with failures on business-critical systems. Preferred Qualifications: Exposure to Apache Spark, Apache Trino, or another big data processing system. Knowledge of streaming data principles and best practices. Understanding of database technologies and standards. Experience working on large More ❯

Azure, or Google Cloud Platform (GCP) Strong proficiency in SQL and experience with relational databases such as MySQL, PostgreSQL, or Oracle Experience with big data technologies such as Hadoop, Spark, or Hive Familiarity with data warehousing and ETL tools such as Amazon Redshift, Google BigQuery, or Apache Airflow Proficiency in Python and at least one other programming language More ❯

related issues Strong analytical and problem-solving skills Strong teamwork, interpersonal and collaboration skills with colleagues and clients Desirable: Experience with Cloud ETL tools such as Databricks/Snowflake, Spark and Kafka Experience using source control tools such as GitHub or Azure DevOps Experience with Azure DevOps for CI/CD pipeline development and data operations (DataOps) Experience with More ❯

strategies. Strong experience in IaC and able to guide how one could deploy infrastructure into different environments. Knowledge of handling and transforming various data types (JSON, CSV, etc) with Apache Spark, Databricks or Hadoop. Good understanding of possible architectures involved in modern data system design (Data Warehouse, Data Lakes, Data Meshes) Ability to create data pipelines on a More ❯

experience working as a Software Engineer on large software applications Proficient in many of the following technologies – Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, Apache Spark. Experience working with one or more of the following database systems – DynamoDB, DocumentDB, MongoDB Demonstrated expertise in unit testing and tools – JUnit, Mockito, PyTest, Selenium. Strong working knowledge … experience working as a Software Engineer on large software applications Proficient in many of the following technologies – Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, Apache Spark. Experience working with one or more of the following database systems – DynamoDB, DocumentDB, MongoDB Demonstrated expertise in unit testing and tools – JUnit, Mockito, PyTest, Selenium. Strong working knowledge More ❯

experience working as a Software Engineer on large software applications Proficient in many of the following technologies – Python, REST, PyTorch, TensorFlow, Docker, FastAPI, Selenium, React, TypeScript, Redux, GraphQL, Kafka, Apache Spark. Experience working with one or more of the following database systems – DynamoDB, DocumentDB, MongoDB Demonstrated expertise in unit testing and tools – JUnit, Mockito, PyTest, Selenium. Strong working knowledge More ❯

Deep understanding in software architecture, object-oriented design principles, and data structures Extensive experience in developing microservices using Java, Python Experience in distributed computing frameworks like - Hive/Hadoop, Apache Spark. Good experience in Test driven development and automating test cases using Java/Python Experience in SQL/NoSQL (Oracle, Cassandra) database design Demonstrated ability to be proactive … HR related applications Experience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, Lambda Working experience with Terraform Experience in creating workflows for Apache Airflow About Roku Roku pioneered streaming to the TV. We connect users to the streaming content they love, enable content publishers to build and monetize large audiences, and provide More ❯

need to apply for the role: Solid hands-on experience with Azure data tools, including Data Factory, Data Lake, Synapse Analytics, and Azure SQL. Strong proficiency with Databricks, including Spark, Delta Lake, and notebooks. Skilled in Python and SQL for data transformation and processing. Experience with Git and modern CI/CD workflows. Strong analytical mindset and effective communication More ❯

need to apply for the role: Solid hands-on experience with Azure data tools, including Data Factory, Data Lake, Synapse Analytics, and Azure SQL. Strong proficiency with Databricks, including Spark, Delta Lake, and notebooks. Skilled in Python and SQL for data transformation and processing. Experience with Git and modern CI/CD workflows. Strong analytical mindset and effective communication More ❯

and presentation skills, with the ability to engage both technical and non-technical stakeholders Desirable Criteria Experience with Matillion products and competitive ETL solutions Knowledge of big data technologies (Spark, PySpark), data lakes, and MPP databases (Teradata, Vertica, Netezza) Familiarity with version control tools such as Git, and experience with Python Degree in Computer Science or related field (or More ❯

performance databases, including PostgreSQL, ClickHouse, Cassandra, and Redis. In-depth knowledge of ETL/ELT pipelines, data transformation, and storage optimization. Skilled in working with big data frameworks like Spark, Flink, and Druid. Hands-on experience with both bare metal and AWS environments. Strong programming skills in Python, Java, and other relevant languages. Proficiency in containerization technologies (Docker, Kubernetes More ❯

we’d love you to have... Understanding of cloud computing security concepts Experience in relational cloud based database technologies like Snowflake, BigQuery, Redshift Experience in open source technologies like Spark, Kafka, Beam Good understanding of Cloud providers – AWS, Microsoft Azure, Google Cloud Familiarity with DBT, Delta Lake, Databricks Experience working in an agile environment Here’s a taste of More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or More ❯

Engineering (open to professionals from various data eng. backgrounds - data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience - formal or More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or More ❯

Engineering (open to professionals from various data eng. backgrounds — data pipelines, ML Eng, data warehousing, analytics engineering, big data, cloud etc.) Technical Exposure: Experience with tools like SQL, Python, Apache Spark, Kafka, Cloud platforms (AWS/GCP/Azure), and modern data stack technologies Formal or Informal Coaching Experience: Any previous coaching, mentoring, or training experience — formal or More ❯

computer science, Engineering, Mathematics, or related field. - Proven experience (5+ years) in developing and deploying data engineering pipelines and products - Strong proficiency in Python - Experienced in Hadoop, Kafka or Spark - Experience leading/mentoring junior team members - Strong communication and interpersonal skills, with the ability to effectively communicate complex technical concepts to both technical and non-technical stakeholders. Operating More ❯

You're really awesome at: Object-oriented programming (Java) Data modeling using various database technologies ETL processes (transferring data in-memory, moving away from traditional ETLs) and experience with Apache Spark or Apache NiFi Applied understanding of CI/CD in change management Dockerized applications Using distributed version control systems Being an excellent team player Meticulous and More ❯

OpenShift, MicroK8s; • Infrastructure as Code (IaC), automation & configuration management; • Ansible (plus Puppet, Saltstack), Terraform, CloudFormation; • NodeJS, REACT/MaterialUI (plus Angular), Python, JavaScript; • Big data processing and analysis, e.g. Apache Hadoop (CDH), Apache Spark; • RedHat Enterprise Linux, CentOS, Debian or Ubuntu. • Java 8, Spring framework (preferably Spring boot), AMQP - RabbitMQ, • Open source technologies; • Experience of Agile software More ❯

to define Bicep templates and deploy Azure infrastructure effectively. Experience creating Azure DevOps YAML pipelines. Knowledge of source control, preferably Git. Strong skills in SQL, Azure Synapse, Data Factory, Spark, and Databricks. Experience establishing coding standards and testing practices. Deep understanding of the full data lifecycle. Involvement in implementing CI/CD methodologies. Experience working in agile, rapid development More ❯

to define Bicep templates and deploy Azure infrastructure effectively. Experience creating Azure DevOps YAML pipelines. Knowledge of source control, preferably Git. Strong skills in SQL, Azure Synapse, Data Factory, Spark, and Databricks. Experience establishing coding standards and testing practices. Deep understanding of the full data lifecycle. Involvement in implementing CI/CD methodologies. Experience working in agile, rapid development More ❯

Research/Statistics or other quantitative fields. Experience in NLP, image processing and/or recommendation systems. Hands on experience in data engineering, working with big data framework like Spark/Hadoop. Experience in data science for e-commerce and/or OTA. We welcome both local and international applications for this role. Full visa sponsorship and relocation assistance More ❯

Research/Statistics or other quantitative fields. Experience in NLP, image processing and/or recommendation systems. Hands on experience in data engineering, working with big data framework like Spark/Hadoop. Experience in data science for e-commerce and/or OTA. We welcome both local and international applications for this role. Full visa sponsorship and relocation assistance More ❯