101 to 125 of 158 Apache Spark Jobs in the UK

communication skills and a collaborative approach Nice to have Edge AI or embedded ML deployment (Jetson, Raspberry Pi, ARM or similar) Computer Vision Databricks, Spark, MLflow Docker, Kubernetes and AWS Experience working in defence, aerospace, robotics, autonomous systems or other safety-critical environments Current SC or DV clearance (although ...

Experience with: Relational databases (SQL Server, PostgreSQL, Oracle, MySQL) Data visualisation tools (Power BI, Tableau, Spotfire) Cloud platforms, particularly Microsoft Azure Big data technologies (Spark, Elasticsearch, Hadoop) Statistical modelling and advanced analytics Machine learning and pattern recognition techniques Web technologies such as HTML and JavaScript ...

extracting, manipulating, and preparing data from relational databases. Experience with NoSQL databases is a plus. Big Data/Cloud: Experience with big data tools ( Spark, Hadoop ) and cloud computing platforms ( AWS, Azure, or GCP ) for scalable ML workflows. Visualization: Ability to create clear, compelling data visualizations using tools like ...

maintain operational continuity and quality output Knowledge of data processing frameworks, including ETL pipelines, data integration, and data modeling (experience with Databricks or Spark preferred) Strong analytical and problem-solving skills with the ability to communicate clearly to both technical and non-technical stakeholders Basic understanding of artificial intelligence ...

operations. Demonstrated depth in Big Data engineering, with proficiency in at least one of the following areas: distributed data processing platforms such as Databricks, Spark, or EMR search platforms such as OpenSearch or Elasticsearch graph databases such as Neo4j Strong understanding of Agile and Lean practices, including backlog refinement ...

Regulatory compliance automation Conversational AI/chatbots for banking use cases 2 . Data Engineering & Platform Development Build scalable data pipelines using tools like Spark, PySpark, Databricks, Snowflake, or BigQuery Design data lakes/lakehouse architectures on AWS/Azure/GCP Integrate structured and unstructured data (market feeds … data engineering/AI engineering Strong Python and SQL expertise Hands-on experience with: LLMs and GenAI frameworks (LangChain, LlamaIndex) Vector databases & embeddings Spark/Databricks/distributed computing Experience in REST APIs and microservices architecture Cloud & Tools Cloud: Azure (preferred), AWS, or GCP Data platforms: Snowflake, Databricks, Delta ...

Limited (DDIL) communications environments, supporting local processing at the edge Building streaming ingestion, event pipelines, and batch/stream processing workloads using technologies like Apache Flink and/or Apache Spark Implementing metadata, lineage, and catalogue capabilities along with policy-based security and governance Job Requirements: Extensive ...

data models and outputs Set up monitoring and ensure data health for outputs What we are looking for Proficiency in Python, with experience in Apache Spark and PySpark Previous experience with data analytics softwares Ability to scope new integrations and translate user requirements into technical specifications What ...

large-scale analytics platforms Expert knowledge of Quantexa, Entity Resolution (ER), Scoring Framework, Contextual Network Generation Expertise in Big Data technologies, including: Scala, Apache Spark, Hadoop, ElasticSearch Strong stakeholder management and communication skills - capable of interfacing with both business and technical audiences. The work is heavily weighted toward ...

data extraction, transformation, and pipeline development. Proficiency with data visualization tools (Tableau, Qlik, or similar). Experience with big data platforms (Snowflake, Databricks, Spark) and ETL processes . Working knowledge of Python or R for analytics or automation (preferred). Understanding of statistical methods and A/B testing ...

large-scale data platforms or complex data pipelines. Deep understanding of analytics, ML workflows, and data product development. Strong grasp of tools such as Spark, Cassandra, and Redshift. Comfortable discussing technical architecture and data systems with engineers. Strong communicator with the ability to explain complex concepts to non-technical ...

ongoing basis. Contribute to the planning, research, recommendation, and implementation of current and future IT strategies. PERSON SPECIFICATION: Essential Demonstrable experience of Spark (PySpark) and SQL. Demonstrable experience of Fabric components. Demonstrable experience of working on enterprise environments. Experience of designing architectural Fabric processes. Experience of Azure Data Factory. ...

manage multiple priorities and respond calmly to issues. • Self-motivated with a collaborative approach. Preferred: • Familiarity with AWS data tooling such as S3, Athena, Spark or notebooks. • Experience supporting data products or client-facing datasets. • Understanding of release testing, UAT or production support processes. • Exposure to Jira/Agile ...

machine learning frameworks and programming languages such as Python, R, or similar. Experience working with large datasets and data processing technologies (e.g., Hadoop, Spark, SQL). Familiarity with anomaly detection, supervised, unsupervised learning methods, deep learning, and graph based solutions Demonstrated ability to work collaboratively in cross-functional teams ...

loop, calibration, guardrails, auditability) Experience with hyperparameter tuning at scale and experimentation frameworks Strong technical experience with SQL and Python; significant experience with Spark for large-scale data processing Excellent interpersonal skills with demonstrable collaborative leadership capabilities (coaching, mentoring, guiding, delivery ownership and stakeholder engagement) and strong standards ...

using data mining techniques and with statistical rigor Strong technical skills regarding data analysis, statistics, machine learning and programming. Strong working knowledge of Hadoop, Spark, SQL, Python and/or R. Excellent communication skills Strong skills in translating business problems into data problems Strong data visualization skills Strong skills ...

processin g.Cloud Pub/Su b: For real-time messaging and event ingestio n.Cloud Storag e: For scalable and durable object storag e.Cloud Composer (Apache Airflow ): For workflow orchestration and pipeline managemen t.Cloud Datapro c: For managed Hadoop and Spark clusters (good to have ).Cloud SQL/Cloud ...

based languages (Java, Kotlin), ideally combined with Python and experience in Spring Boot Solid understanding of data engineering tools and frameworks, like Spark, Flink, Kafka, dbt, Trino, and Airflow. Hands-on experience with cloud environments (AWS, GCP, or Azure), infrastructure-as-code practices, and ideally container orchestration with Kubernetes. … with experience in CI/CD pipelines, monitoring, and observability tools (Grafana or equivalent). Exposure to analytics, reporting, and BI tools such as Apache Superset, Lightdash or OpenSearch Willingness to work across the stack by contributing to API development and, at times, UI components (Vue.js, Zoho, or similar ...

with large-scale and unstructured datasets. Python development experience, including technologies such as PySpark, Pandas and PyArrow. Expertise in distributed data processing frameworks, particularly Apache Spark. Experience building and maintaining ETL pipelines using technologies such as Apache Airflow, AWS Step Functions or Apache NiFi. Cloud platform experience ...

Swindon (we could consider twice per fortnight etc). Required Skills & Experience Strong experience testing data pipelines/ETL processes Hands-on experience with Apache Spark Experience working with Databricks and/or AWS Glue Proven ability to build automation or testing frameworks (Python preferred) Experience with large ...

teams Ensure solutions meet security, compliance, and regulatory standards Key Skills Required: Experience in solution design/architecture Strong background in big data tools (Spark, Airflow, Trino, etc.) Knowledge of cloud platforms (AWS/Azure/GCP) Understanding of cybersecurity principles (IAM, encryption, secure design) Ability to work hands ...

performance. Desirable Skills Experience working with data platforms, data pipelines or data-heavy environments. Exposure to batch or streaming data workloads, such as Kafka, Spark, Airflow, Glue, EMR, Databricks or similar. Data observability experience. Previous public sector experience across multiple engagements. Consultancy experience would be beneficial, especially within environments ...

like ER/Studio, ERwin, or Sparx EA (preferred). Experience reverse engineering models from existing databases. Proficiency in distributed data processing frameworks (e.g., Spark, Flink, Hadoop). Experience with API management and gateway tools and services. Experience of RESTful APIs for ingesting and exposing data. Proven track record … with data governance, quality, lineage, and security practices. Experience with real-time/streaming data technologies, data Ingestion/ETL (e.g., Apache Kafka, Apache NiFi, Kinesis, Pub/Sub). Experience in, or knowledge of, DevSecOps Tooling and Processes. Self-starter with the ability to appropriately prioritise ...

execute. You have deep experience designing, building, and operating production data pipelines, including owning reliability, monitoring, and failure recovery for orchestration systems such as Apache Airflow (or equivalent). You are fluent in modern data and cloud ecosystems, with hands-on experience (or the ability to ramp quickly … technologies such as Iceberg, Spark, PostgreSQL, and cloud-native infrastructure on AWS (or equivalent). You write clean, readable, and testable Python code, applying sound abstractions, naming, and review discipline to build systems that scale beyond the first version. You bring strong analytical and problem-solving skills ...

work on a Big Data Project in the finance sector. Required Experience: Java Big Data experience On-premises cluster experience Databases Kubernetes Kafka Apache Spark Preffered Experience: Finance sector experience ...