201 to 225 of 497 Apache Spark Jobs in the UK

Web App. Good knowledge in real-time streaming applications preferably with experience in Kafka Real-time messaging or Azure Stream Analytics/Event Hub. Spark processing and performance tuning. File formats partitioning for e.g. Parquet, JSON, XML, CSV. Azure DevOps, GitHub actions. Hands-on experience in at least one More ❯

and understanding of a broad range of data-oriented technologies and their applicability. Experience with one or more data warehouse platforms (preferably Databricks and Spark). Understanding of the principles, practices and techniques of Data Engineering and Machine Learning. Working knowledge of one or more data-oriented programming languages More ❯

build. Dojo home and away We believe our best work happens when we collaborate in-person. These "together days" foster communication, drive innovation and spark our brightest ideas. That's why we have an office-first culture. This means working from the office 4+ days per week. With offices More ❯

Familiarity with Infrastructure as Code (IaC) tools such as Terraform or CloudFormation. Knowledge of data engineering and experience with big data technologies like Hadoop, Spark, or Kafka. Experience with CI/CD pipelines and automation, such as using Jenkins, GitLab, or CircleCI. ABOUT BUSINESS UNIT IBM Consulting is IBM More ❯

Design and Maintenance, Apps, Hive Metastore Management, Network Management, Delta Sharing, Dashboards, and Alerts. Proven experience working with big data technologies, i.e., Databricks and Apache Spark. Proven experience working with Azure data platform services, including Storage, ADLS Gen2, Azure Functions, Kubernetes. Background in cloud platforms and data architectures, such … experience of ETL/ELT, including Lakehouse, Pipeline Design, Batch/Stream processing. Strong working knowledge of programming languages, including Python, SQL, PowerShell, PySpark, Spark SQL. Good working knowledge of data warehouse and data mart architectures. Good experience in Data Governance, including Unity Catalog, Metadata Management, Data Lineage, Quality More ❯

Remote Active SC clearance required £640 per day inside ir35 REQUIRED Strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management etc Strong understanding of complex JSON manipulation Experience working with Data Pipelines using a custom Python/PySpark frameworks Strong understanding of … PySpark Delta Lake Bash (both CLI usage and scripting) Git Markdown Scala DESIRABLE Azure SQL Server as a HIVE Metastore DESIRABLE TECHNOLOGIES Azure Databricks Apache Spark Delta Tables Data processing with Python PowerBI (Integration/Data Ingestion) JIRA If you meet the above requirements, please apply for the More ❯

MONTH INITIAL CONTRACT Seeking a Data Engineer who has a strong understanding of data concepts - data types, data structures, schemas (both JSON and Spark), schema management etc - Strong understanding of complex JSON manipulation - Experience working with Data Pipelines using a custom Python/PySpark frameworks - Strong understanding of the … Lake - Bash (both CLI usage and scripting) - Git - Markdown - Scala (bonus, not compulsory) - Azure SQL Server as a HIVE Metastore (bonus) Technologies - Azure Databricks - Apache Spark - Delta Tables - Data processing with Python - PowerBI (Integration/Data Ingestion) - JIRA Due to the nature and urgency of this post, candidates More ❯

+ 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform, who are hiring their first Data … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as More ❯

+ 10% bonus + benefits Purpose: Build and maintain large, scalable Data Lakes, processes and pipelines Tech: Python, Iceberg/Kafka, Spark/Glue, CI/CD Industry: Financial services/securities trading Immersum continue to support a leading SaaS securities trading platform, who are hiring their first Data … Infra tooling using Terraform, Ansible and Jenkins whilst automating everything with Python Tech (experience in any listed is advantageous) Python Cloud: AWS Lake house: Apache Spark or AWS Glue Cloud Native storage: Iceberg, RDS, RedShift, Kafka IaC: Terraform, Ansible CI/CD: Jenkins, Gitlab Other platforms such as More ❯

flexibility for remote work. Office visits required for key meetings and collaboration sessions. Key Responsibilities: Develop and maintain scalable data pipelines using Databricks and Apache Spark to process logistics and delivery data Design ETL workflows that integrate data from multiple delivery and warehouse systems Development of Data Marts More ❯

architecture and models • Working knowledge of designing architectures and data models for large volume data applications/reporting • Knowledge of Data technologies - SQL, Oracle, Apache Spark, Azure Synapse, etc. • Oversight with implementing, scaling and improving a variety of vendor and open-source technologies • Experienced in delivery, agile data More ❯

Experience in developing web infrastructure (Solr, kubernetes) Experience in git and basic Unix Commands You may also have Experience with large data processing technologies (Apache Spark) Other helpful information: The team work in a hybrid working pattern and spends 2 days per week in office Apply now! Benefits More ❯

advanced visualisations, ML model interpretation, and KPI tracking. Deep knowledge of feature engineering, model deployment, and MLOps best practices. Experience with big data processing (Spark, Hadoop) and cloud-based data science environments. Other: Ability to integrate ML workflows into large-scale data pipelines. Strong experience in data preprocessing, feature More ❯

advanced visualisations, ML model interpretation, and KPI tracking. Deep knowledge of feature engineering, model deployment, and MLOps best practices. Experience with big data processing (Spark, Hadoop) and cloud-based data science environments. Other: Ability to integrate ML workflows into large-scale data pipelines. Strong experience in data preprocessing, feature More ❯

Technology, or related field. Proficiency in software engineering with experience in Java & Spring or other major programming languages. Preferred Qualifications: Experience with Spring Boot, Spark (Big Data), and Message Bus Architecture. Familiarity with containerisation (e.g., Kubernetes), AWS Cloud, and CICD pipelines (Jenkins). If you meet the above criteria More ❯

. Experience in using cloud-native services for data engineering and analytics. Experience with distributed systems, serverless data pipelines, and big data technologies (e.g., Spark, Kafka). Ability to define and enforce data governance standards. Experience in providing architectural guidance, mentorship and leading cross-functional discussions to align on More ❯

. Experience in using cloud-native services for data engineering and analytics. Experience with distributed systems, serverless data pipelines, and big data technologies (e.g., Spark, Kafka). Ability to define and enforce data governance standards. Experience in providing architectural guidance, mentorship and leading cross-functional discussions to align on More ❯

with attention to detail and accuracy. Adept at queries, report writing, and presenting findings. Experience working with large datasets and distributed computing tools (Hadoop, Spark, etc.) Knowledge of advanced statistical techniques and concepts (regression, properties of distributions, statistical tests, etc.). Experience with data profiling tools and processes. Knowledge More ❯

with attention to detail and accuracy. Adept at queries, report writing, and presenting findings. Experience working with large datasets and distributed computing tools (Hadoop, Spark, etc.) Knowledge of advanced statistical techniques and concepts (regression, properties of distributions, statistical tests, etc.). Experience with data profiling tools and processes. Knowledge More ❯

governance. Cloud Computing : AWS, Azure, Google Cloud for scalable data solutions. API Strategy : Robust APIs for seamless data integration. Data Architecture : Finbourne LUSID, Hadoop, Spark, Snowflake for managing large volumes of investment data. Cybersecurity : Strong data security measures, including encryption and IAM. AI and Machine Learning : Predictive analytics, risk More ❯

governance. Cloud Computing : AWS, Azure, Google Cloud for scalable data solutions. API Strategy : Robust APIs for seamless data integration. Data Architecture : Finbourne LUSID, Hadoop, Spark, Snowflake for managing large volumes of investment data. Cybersecurity : Strong data security measures, including encryption and IAM. AI and Machine Learning : Predictive analytics, risk More ❯

Internet and Networking Protocols, Security Architecture, Version Control Systems, and CI/CD tools; Deep experience with several of the following platforms Kafka, Flink, Spark, Kubernetes, API Gateways, and messaging systems (Tibco, IBM MQ, and open-source variants); Deep familiarity with data streaming, integration patterns, and how they can More ❯

learning algorithms and general statistical methodologies and theory. Advanced knowledge of AB testing and design of experiment. Advanced Python and SQL skills, experience using Spark for processing large datasets. Understanding of software product development processes and governance, including CI/CD processes and release and change management. Familiarity with More ❯

Python); Software collaboration and revision control (e.g., Git or SVN). Desired skills and experiences: ElasticSearch/Kibana Cloud computing (e.g., AWS) Hadoop/Spark etc. Graph Databases Educational level: Master Degree More ❯

. Experience in using cloud-native services for data engineering and analytics. Experience with distributed systems, serverless data pipelines, and big data technologies (e.g., Spark, Kafka). Ability to define and enforce data governance standards. Experience in providing architectural guidance, mentorship and leading cross-functional discussions to align on More ❯