51 to 65 of 65 Apache Spark Jobs in the South West

exchange connectivity Scripting abilities in Python, Bash, or similar languages Knowledge of monitoring tools and alerting frameworks Exposure to data technologies such as Kafka, Spark or Delta Lake is useful but not mandat Bachelor's degree in Computer Science, Engineering, or related technical field This role offers competitive compensation More ❯

leadership to teams working with neighboring technologies, most notably Microsoft’s Power Platform and Azure infrastructure. Experience across Microsoft Azure cloud platform = DataBricks/Spark The administration of database systems, primarily Microsoft SQL Server. Data Warehousing/Data Lakes/Master Data Management/Artificial Intelligence (AI)/Data More ❯

at a key stage of our growth. We are looking for someone who: Can demonstrate a strong technical data skillset in regard to Python, Spark, Pandas, SQL, NoSQL. Has strong experience working with modern database platforms such as Cosmos, Athena, and MySQL. Is comfortable with both NoSQL and relational More ❯

drive team alignment Work closely with stakeholders to translate business needs into scalable solutions Tech environment includes Python, SQL, dbt, Databricks, BigQuery, Delta Lake, Spark, Kafka, Parquet, Iceberg (If you haven’t worked with every tool, that’s totally fine — my client values depth of thinking and engineering craft More ❯

relational databases to build data solutions, such as SQL Server/Oracle , experience with relational and dimensional data structures. Experience in using distributed frameworks ( Spark, Flink, Beam, Hadoop ). Proficiency in infrastructure as code (IaC) using Terraform . Experience with CI/CD pipelines and related tools/frameworks. … understanding of cloud storage, networking, and resource provisioning. It would be great if you had... Certification in GCP “Professional Data Engineer”. Certification in Apache Kafka (CCDAK). Proficiency across the data lifecycle. Working for us: Our focus is to ensure we are inclusive every day, building an organisation More ❯

secure or regulated environments Ingest, process, index, and visualise data using the Elastic Stack (Elasticsearch, Logstash, Kibana) Build and maintain robust data flows with Apache NiFi Implement best practices for handling sensitive data, including encryption, anonymisation, and access control Monitor and troubleshoot real-time data pipelines to ensure high … experience as a Data Engineer in secure, regulated, or mission-critical environments Proven expertise with the Elastic Stack (Elasticsearch, Logstash, Kibana) Solid experience with Apache NiFi Strong understanding of data security, governance, and compliance requirements Experience building real-time, large-scale data pipelines Working knowledge of cloud platforms (AWS … with a strong focus on data accuracy, quality, and reliability Desirable (Nice to Have): Background in defence, government, or highly regulated sectors Familiarity with Apache Kafka, Spark, or Hadoop Experience with Docker and Kubernetes Use of monitoring/alerting tools such as Prometheus, Grafana, or ELK Understanding of More ❯

relational databases to build data solutions, such as SQL Server/Oracle , experience with relational and dimensional data structures. Experience in using distributed frameworks ( Spark, Flink, Beam, Hadoop ). Proficiency in infrastructure as code (IaC) using Terraform . Experience with CI/CD pipelines and related tools/frameworks. … understanding of cloud storage, networking and resource provisioning. It would be great if you had Certification in GCP "Professional Data Engineer". Certification in Apache Kafka (CCDAK). Proficiency across the data lifecycle. WORKING FOR US Our focus is to ensure we are inclusive every day, building an organisation More ❯

relational databases to build data solutions, such as SQL Server/Oracle , experience with relational and dimensional data structures Experience in using distributed frameworks ( Spark, Flink, Beam, Hadoop ) Proficiency in infrastructure as code (IaC) using Terraform Experience with CI/CD pipelines and related tools/frameworks Containerisation Good … Good understating of cloud storage, networking and resource provisioning It would be great if you had... Certification in GCP "Professional Data Engineer" Certification in Apache Kafka (CCDAK) Proficiency across the data lifecycle WORKING FOR US Our focus is to ensure we are inclusive every day, building an organisation that More ❯

handle. Required - Proficiency in AWS services and tools related to data storage, processing, and analytics. - Strong experience with big data technologies such as Hadoop, Spark, or similar frameworks. - Active SC clearance is essential for this role. More ❯

learning models, and create robust solutions that enhance public service delivery. Working in classified environments, you'll tackle complex challenges using tools like Hadoop, Spark, and modern visualisation frameworks while implementing automation that drives government efficiency. You'll collaborate with stakeholders to transform legacy systems, implement data governance frameworks … analytics platforms e.g. relevant AWS and Azure platform services; Data tools hands on experience with Palantir ESSENTIAL; Data science approaches and tooling e.g. Hadoop, Spark; Data engineering approaches; Database management, e.g. MySQL, Postgress; Software development methods and techniques e.g. Agile methods such as SCRUM; Software change management, notably familiarity More ❯

Extract, Load, Transform (ELT) workflows to move data from source systems to data stores. You will have used one or more supporting technologies, i.e. Apache, Kafka, NiFi, Spark, Flink, or Airflow. A history of working with SQL and NoSQL type databases (PostgreSQL, Mongo, ElasticSearch, Accumulo, or Neo4j). More ❯

Informatics, Information Systems, or another quantitative field. They should also have experience using the following software/tools: Experience with big data tools: Hadoop, Spark, Kafka, etc. Experience with relational SQL and NoSQL databases, including Postgres and Cassandra. Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow … etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. Experience with stream-processing systems: Storm, Spark-Streaming, etc. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. Salary: 30000 per annum + benefits Apply For This Job If you would like to apply More ❯

for purpose Experience that will put you ahead of the curve Experience using Python on Google Cloud Platform for Big Data projects, BigQuery, DataFlow (Apache Beam), Cloud Run Functions, Cloud Run, Cloud Workflows, Cloud Composure SQL development skills Experience using Dataform or dbt Demonstrated strength in data modelling, ETL … it! Our Future, Our Responsibility - Inclusion and Diversity at Future We embrace and celebrate diversity, making it part of who we are. Different perspectives spark ideas, fuel creativity, and push us to innovate. That's why we're building a workplace where everyone feels valued, respected, and empowered to More ❯

and implementation experience with Microsoft Fabric (preferred), along with familiarity with Azure Synapse and Databricks. Experience in core data platform technologies and methods including Spark, Delta Lake, Medallion Architecture, pipelines, etc. Experience leading medium to large-scale cloud data platform implementations, guiding teams through technical challenges and ensuring alignment … Strong knowledge of CI/CD practices for data pipelines, ensuring automated, repeatable, and scalable deployments. Familiarity with open-source data tools such as Spark, and an understanding of how they complement cloud data platforms. Experience creating and maintaining structured technical roadmaps, ensuring successful delivery and future scalability of More ❯

Principal Data Engineer, Consulting Bristol Based You must be eligible for SC Clearance Role Overview The Principal Data Engineer will be responsible for designing and implementing cloud-based data solutions using a range of AWS services. This role involves working More ❯