15 of 15 Apache Spark Jobs in the East of England

design, implement, and optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. Key Responsibilities Design and architect Databricks Lakehouse solutions for large-scaleenterprise data … Databricks implementation, including architecture, development, deployment, and optimization. Define Databricks workspace architecture, clusterstrategies, security models, and operational standards. Design and developscalable data pipelinesusing Databricks, Apache Spark, PySpark, SQL, andDelta Lake . Implement data ingestionframeworks supporting batchand real-timedata processing. Design Delta Lake architectures including: Delta tables Medallion architecture ...

Modern data platforms: Databricks, Snowflake and cloud-native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch … streaming patterns Transformation and quality: dbt, Great Expectations and equivalent tooling Orchestration: Apache Airflow and cloud-native orchestration services Event streaming: Kafka and cloud-native event-streaming technologies Data modelling: Dimensional, analytical and domain-oriented modelling Data governance: Catalogue, metadata, lineage, quality, security, privacy and access controls Data products ...

solutions. Pipeline Development : Develop and configure meta-data driven data pipelines using data orchestration tools such as Azure Data factory and engineering tools like Apache Spark to ensure seamless data flow. Monitoring and Failure Recovery: Implement monitoring procedures to detect failures or unusual data profiles and establish recovery … using SQL to query and manipulate data. Proficiency in cloud data tooling such as Synapse Analytics, Microsoft Fabric, Azure Data Factory or AWS equivalents. Spark/Pyspark or Python skills. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. MS Fabric interest, skills or experience ...

Analytics, Artificial Intelligence or a related field, or equivalent experience, would be effective, although practical experience is equally valuable."Nice to Have"Azure, Databricks, Apache Spark or comparable cloud and data platforms. Data pipelines, ETL and workflow-orchestration technologies. Machine-learning methods such as forecasting, prediction or classification. ...

e.g. Python, R, KNIME, Splunk, Geo data. Desirable: Industry recognised qualification/accreditation in relevant field. Experience of using big data ecosystems e.g. Hadoop, Spark and/or MapReduce. Experience of working in NATO or similar multi-national defence/security sector. Why work with us? By joining ...

tools e.g. Microsoft Power BI, Spotfire, TableauExperience of briefing and communication analytical products to non-technical staffExperience of using big data ecosystems e.g. Hadoop, Spark and/or MapReduce. Experience of working in NATO or similar multi-national defence/security sector. Why work with us? By joining ...

related field, or equivalent practical experience. Experience with AWS or GCP, Kubernetes, SQL warehouse and batch-processing technologies such as Trino, Presto or Spark, Kafka or other streaming systems, and Go or Python would be useful. Node.js experience is a plus. We value transferable evidence over an exact match ...

with modern NLP stack and multi-modal representation learning is a plusWe'd love to see that you've worked with big data systems (Spark, S3, and Airflow) and can program (Java, Scala, or Python)Good understanding of system architecture. Have experience in big data technologies and streaming architecture ...

Experience supporting the Intelligence Community or Department of Defense environments Experience with Python and working with databases Experience with big data frameworks such as Spark, PySpark, or Elastic Search Experience with data visualization platforms such as Kibana, Foundry Workshop, or Palantir Knowledge of secure coding practices and development ...

core component of the role is applying, modifying and designing algorithms and mathematical models to solve business problems using big data architectures (Hadoop, Spark, Cloud). In this role, your primary modelling focus will be neural network‐based forecasting, with a strong emphasis on rigorous evaluation against robust baselines ...

leadership teams. Minimum Qualifications Masters in Statistics, Mathematics, Data Science, Machine Learning, Physics, Engineering, Computer Science or equivalent Proficiency in data querying using SQL, Spark, or equivalent technologies Proficiency in scripting languages for data processing and analysis (ex: Python, R, or Scala) Excellent communication and multi-functional collaboration skills ...

Python is required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation ...

such as Pipeline Builder or AIP, or similar platforms for application develop ment and data integration Experience with distributed data and compute, such as Spark or Databricks, Hadoop or Hive, Kafka, or AWS EMR, and cloud environments used by the IC Experience with geospatial analysis tools, such as ArcGIS … such as Pipeline Builder or AIP, or similar platforms for application develop ment and data integration Experience with distributed data and compute, such as Spark or Databricks, Hadoop or Hive, Kafka, or AWS EMR, and cloud environments used by the IC Experience with geospatial analysis tools, such as ArcGIS ...

spans a wide range of technologies, including Java and Python-based MicroServices, AWS/GCP cloud backend systems, Big Data technologies like Hive and Spark, and modern Web applications. With a globally distributed presence across the US, India and Europe, the team thrives on collaboration, bringing together diverse perspectives … needs, entrepreneurial spiritExcellent verbal and written communication skillsBS or MS degree in Computer Science or equivalentExperience in distributed computing frameworks like - Hive/Hadoop, Apache SparkExperience in developing Finance or HR related applicationsExperience with following cloud services: AWS Elastic Beanstalk, EC2, S3, CloudFront, RDS, DynamoDB, VPC, Elastic Cache, LambdaWorking ...

across devices and platforms to surface key metrics. The role emphasizes coding in Python, strong SQL, and experience with large data ecosystems (HDFS, Hive, Spark, Airflow). You will influence architecture and own initiatives from concept to delivery. Hybrid work from Cambridge Office. #J-18808-Ljbffr ...