11 of 11 Apache Spark Jobs in Manchester

Data Engineer Salary up to £60,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via chelsea.hackett@searchability.com ABOUT THE CLIENT Due to continued growth, we’re seeking a skilled Data … optimise scalable data pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering ...

code and sharing knowledge across the team Supporting production systems when needed The platform uses a mix of modern data tooling including AWS and Apache Spark, so there's plenty of opportunity to work with big data technologies. What they're hoping you'll bring … complex data problems An interest in improving pipelines, tooling and processes A collaborative approach and willingness to share ideas Exposure to tools like Apache Spark, modern data platforms, or AI tools for development productivity would be a bonus but isn't essential. The setup The team works ...

code and sharing knowledge across the team Supporting production systems when needed The platform uses a mix of modern data tooling including AWS and Apache Spark, so there's plenty of opportunity to work with big data technologies. What they're hoping you'll bring … complex data problems An interest in improving pipelines, tooling and processes A collaborative approach and willingness to share ideas Exposure to tools like Apache Spark, modern data platforms, or AI tools for development productivity would be a bonus but isn't essential. The setup The team works ...

direction within your team Required Skills and Experience Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka ...

/CD and Agile practices Required Skills & Experience 5+ years' experience building big data pipelines in distributed environments Strong experience with Kafka, Hadoop, Spark and/or Python Strong experience with DBT and Snowflake Experience with Data Vault and dimensional data modelling Strong SQL skills and experience with enterprise ...

excels in distributed environments and values technical excellence through CI/CD and Agile. The Essentials Pipeline Tech: 5+ years of experience with Kafka, Spark, Hadoop, and DBT . Modern Modeling: Expert skills in Data Vault and Dimensional modeling. Cloud & DevOps: Strong AWS experience with a focus ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

architecture into production-ready platforms • Solving complex data challenges across highly secure and technically demanding environments Technology exposure Typical environments include: • Python and SQL • Spark, Databricks and distributed data processing platforms • AWS, Azure and Google Cloud • Modern data lake, lakehouse and warehouse architectures Who we’re keen to speak ...

engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards, analytics, ML or LLM concepts is beneficial, but not essential. ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...

experience with Cloudera Data Platform (CDP) on AWS or recent Hadoop administration on similar platforms. AWS administration and platform management experience. Strong experience in Spark, Airflow, Apache Ranger, Impala (AWS CDK is a plus). Agile experience with Scrum, and tools including Jira, Slack. Familiarity with PowerBI ...