1 to 25 of 26 Permanent Apache Spark Jobs in the UK excluding London

DataBricks, Azure SQL (Indicative experience = 5yrs+) Build and test processes supporting data extraction, data transformation, data structures, metadata, dependency and workload management. Knowledge on Spark architecture and modern Datawarehouse/Data-Lake/Lakehouse techniques Build transformation tables using SQL. Moderate level knowledge of Python/PySpark or equivalent more »

processing, and analytics. Programming Skills: Proficiency in Python, SQL, and other relevant programming languages. Big Data Technologies: Experience with big data technologies such as Apache Spark. Data Warehousing: Strong knowledge of data warehousing concepts and solutions. Problem-Solving: Excellent problem-solving skills with a detail-oriented approach. Leadership: Proven more »

understanding of cloud technologies (AWS, GCP, Azure, etc.) Desired understanding of security frameworks/standards Desired understanding of data streaming and messaging frameworks (Kafka, Spark, etc.) Desired understanding of distributed tracing and monitoring (Zipkin, OpenTracing, Prometheus, Grafana, ELK stack, Micrometer metrics, etc.) Desired understanding of containers (Docker, Kubernetes, Helm more »

to join Ordnance Survey's Data Derivation team in developing our next generation geospatial data. Using big data technologies (primarily Databricks, an implementation of Apache Spark) you will develop, support, and maintain automated processes that enrich and transform Ordnance Survey's large-scale data. Working in a forward … testing approaches and best practice Providing technical coaching and mentoring Advocating software engineering industry best practice Interpreting detailed specifications and translating into simple solutions Apache Spark/Databricks experience (desirable) Experience of working in Microsoft Azure (desirable) Experience of working with the Scala language (desirable) The rewards We more »

development. Track record in problem resolution & selection of technical solutions. Qualified to relevant development certification (or equivalent experience). Experience working with Databricks/Apache Spark. Experience working with infrastructure-as-code. Desirable: Experience working with Geospatial Data. Experience using Terraform. The rewards: We want you to love what more »

Redshift, DynamoDB, Glue and SageMaker Infrastructure-as-Code tools and approaches (we use the AWS CDK with CloudFormation) Data processing frameworks such as pandas, Spark and PySpark Machine learning concepts like model training, model registry, model deployment and monitoring Development and CI/CD tools (we use GitHub, CodePipeline more »

data practices Possess strong knowledge of data tools, data management tools, and various data and information technologies. E.g. DAMA DMBOK, Microsoft SQL Server, Couchbase, Apache Druid, Spark, Kafka, Airflow, etc In-depth understanding of modern data principles, methodologies, and tools Excellent communication and collaboration skills, with the ability … native computing concepts and experience working with hybrid or private cloud platforms is a plus. Demonstrable technical experience working with a Microsoft, Redhat, and Apache data and software engineering environment. A team-oriented individual with a passion for engineered excellence and the ability to lead and motivate a team more »

sequence data and sequence data formats (incl. FASTQ, BAM, CRAM, SRA). Experience using workflow management systems and distributed data processing software (e.g., Hadoop, Spark). Excellent knowledge of the UNIX command line. Experience with the modern, fundamental software development practices, including continuous integration and deployment, application of various more »

least 6 years professional software development experience using Java and Spring framework. Experience using workflow management systems and distributed data processing software (e.g., Hadoop, Spark). At least 6 years professional experience with large scale databases (e.g., Oracle, MongoDB). Excellent knowledge of the UNIX command line. Experience with more »

Familiarity with CICD, containerisation, deployment technologies & cloud platforms (Jenkins, Kubernetes, Docker, AWS) or Familiarity with Big Data and Machine Learning technologies (NumPy, PyTorch, TensorFlow, Spark). Excellent communication, collaboration & problem solving skills, ideally with some experience in agile ways of working. Security clearance: You must be able to gain more »

detail and accuracy in data analysis Effective communication skills to convey complex findings to non-technical stakeholders. Experience with big data technologies using Hadoop, Spark and Cloud Object Storage Could this be you? We believe it's a positive attitude and passion to make things happen that matters most. more »

wrangling, modelling, feature engineering and deployment What’s equally relevant is programming knowledge and a gift for coding using: R, Python, SQL, SAS or Spark You’ll also need the capability to deliver projects, collaborate with operational teams and be able to explain technical concepts to non-specialists What more »

platforms - preferably in GCP - and experience with container orchestration technologies such as Kubernetes. Strong background in distributed computing and familiarity with technologies like Hadoop, Spark, Kafka, and distributed cache systems (Hazelcast, Redis). Experience with database management and proficiency in SQL and NoSQL databases. Knowledge of monitoring and logging more »

associated tools (EKS, EMR, Redshift, Postgres) Experience, or interest in orchestration and processing methods of large workloads using distributed data systems such as Airflow, Spark, Hadoop, and Trino Experience, or interest in processing real-time streaming workloads with distributed event systems such as Kafka, Kinesis, Google PubSub, Flink, or … Spark Streaming What You Can Expect Interesting work - working in a fast-paced and ever-changing industry, new problems and exciting solutions are never too far away. There are always opportunities to learn new skills and broaden your horizons A sense of achievement - Our teams own their own software more »

AWS Redshift, and Python Experience with ETL processes, data integration, and data warehousing. Strong SQL skills Experience with Big Data technologies such as Hadoop, Spark, and Kafka Familiarity with cloud platforms (AWS, Azure, Google Cloud) Working knowledge of data visualisation tools (PowerBI, Tableau, Qlik Sense) Additional skills: Client-facing more »

Data Scientists and Service Engineering teams Experience with design, development and operations that leverages deep knowledge in the use of services like Amazon Kinesis, Apache Kafka, Apache Spark, Amazon Sagemaker, Amazon EMR, NoSQL technologies and other 3rd parties Develop and define key business questions and to build … a related field Experience of Data platform implementation, including 3 years of hands-on experience in implementation and performance tuning Kinesis/Kafka/Spark/Storm implementations Experience with analytic solutions applied to the Marketing or Risk needs of enterprises Basic understanding of machine learning fundamentals Ability to … take Machine Learning models and implement them as part of data pipeline IT platform implementation experience Experience with one or more relevant tools ( Flink, Spark, Sqoop, Flume, Kafka, Amazon Kinesis) Experience developing software code in one or more programming languages (Java, JavaScript, Python, etc) Current hands-on implementation experience more »

complex data warehouses and/or data lakes. Familiarity with cloud-based analytics platforms such as AWS, Azure, Snowflake, Google Cloud Platform (Big Query), Spark, and Splunk. Proficiency in SQL and experience using one or more of the following languages: R, Python, Scala, and Julia, including relevant frameworks/ more »

processing, and analytics. Programming Skills: Proficiency in Python, SQL, and other relevant programming languages. Big Data Technologies: Experience with big data technologies such as Apache Spark. Data Warehousing: Strong knowledge of data warehousing concepts and solutions. Problem-Solving: Excellent problem-solving skills with a detail-oriented approach. Leadership: Proven more »

with Git for version control and project management, alongside some knowledge of Linux/Shell. data platform familiarity - previous experience of working with both Apache Spark and MapReduce data processing and analytics frameworks. and reporting expertise - experience with Tableau, Power BI, Excel alongside notebooks for experiment documentation. What more »

data quality issues and enhancing model performance. Creative Solutions : A history of innovative problem-solving and independent solution development. Big Data Technologies : Knowledge of Spark or PySpark for handling large-scale data processing. ML Expertise : A thorough understanding of machine learning techniques and best practices in model development. Desirable more »

data quality issues and enhancing model performance. Creative Solutions : A history of innovative problem-solving and independent solution development. Big Data Technologies : Knowledge of Spark or PySpark for handling large-scale data processing. ML Expertise : A thorough understanding of machine learning techniques and best practices in model development. Desirable more »

of different platforms. The data will be stored and transported security while still able to be queried efficiently. Technologies used include: Data Technologies: Kafka, Spark, Debezium, GraphQL Programming Languages: Java, Scripting Database Technologies: MongoDB, ElasticSearch, MemSQL, Sybase IQ/ASE Micro Service Technologies: REST, Spring Boot, Jersey Build and more »

Qualifications Knowledge of financial markets UI skills a plus (JavaScript/Angular/React etc.) Graph database experience Big data technology stack such as Spark, HDFS, Apache Flink etc. Data domain modelling Working knowledge of UNIX/Linux ABOUT GOLDMAN SACHS At Goldman Sachs, we commit our people more »

Spark Scala Developer - Scala/Apache Spark - Hybrid/Leeds - £450-£550 Spark Scala Developer to join our client, one of the biggest financial services organizations in the world, with operations in more than 38 countries. It has an IT infrastructure of 200,000+ servers … 000+ database instances, and over 150 PB of data. As a Spark Scala Engineer, you will have the responsibility to refactor L... more »

processing, and analytics. Programming Skills: Proficiency in Python, SQL, and other relevant programming languages. Big Data Technologies: Experience with big data technologies such as Apache Spark. Data Warehousing: Strong knowledge of data warehousing concepts and solutions. Problem-Solving: Excellent problem-solving skills with a detail-oriented approach. Leadership: Proven more »