7 of 7 Permanent Apache Spark Jobs in Manchester

Data Engineer Salary up to £60,000 + discretionary bonus Hybrid working model with collaborative office environment Work with Databricks, Spark Structured Streaming, Kafka (MSK), and AWS Apply online or contact Chelsea Hackett via chelsea.hackett@searchability.com ABOUT THE CLIENT Due to continued growth, we’re seeking a skilled Data … optimise scalable data pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering ...

code and sharing knowledge across the team Supporting production systems when needed The platform uses a mix of modern data tooling including AWS and Apache Spark, so there's plenty of opportunity to work with big data technologies. What they're hoping you'll bring … complex data problems An interest in improving pipelines, tooling and processes A collaborative approach and willingness to share ideas Exposure to tools like Apache Spark, modern data platforms, or AI tools for development productivity would be a bonus but isn't essential. The setup The team works ...

direction within your team Required Skills and Experience Strong analytic skills related to working with unstructured datasets Python (PySpark, Pandas, PyArrow) Distributed data processing (Apache Spark) Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi) Cloud services (AWS, Azure or GCP) Messaging/Streaming (Kafka ...

Azure or AWS Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure/AWS Data Engineering certifications Databricks certifications What ...

architecture into production-ready platforms • Solving complex data challenges across highly secure and technically demanding environments Technology exposure Typical environments include: • Python and SQL • Spark, Databricks and distributed data processing platforms • AWS, Azure and Google Cloud • Modern data lake, lakehouse and warehouse architectures Who we’re keen to speak ...

engineering across ingestion, transformation and data centralisation. Experience with tools such as Glue, Fivetran, Kafka, Dataflow, ADF or Kinesis. Transformation experience using dbt, Spark or Databricks. Confidence working directly with clients and engaging in technical discussions. Exposure to dashboards, analytics, ML or LLM concepts is beneficial, but not essential. ...

data architectures Build scalable ETL workflows and data lake solutions using AWS-native services Engineer high-performance data processing solutions using Python, Java, Scala, Spark and SQL Integrate structured and unstructured data from multiple sources using APIs, scripting and cloud-native tooling Design and build robust AWS cloud data … expertise, particularly across data and analytics services Hands-on experience building ETL processes and data lake/warehouse architectures Proficiency in Python, Java, Scala, Spark and SQL Experience processing large-scale structured and unstructured datasets Experience extracting and integrating data via APIs and custom scripting Strong analytical thinking ...