11 of 11 PySpark Jobs in the East of England

process requirements into technical solutions and working across technical and non‐technical teams. “Nice To Have” Skills and Experience Experience with Databricks, Spark/PySpark, Delta Lake or comparable technologies. Experience with Celonis, process mining or process intelligence. Experience with enterprise platforms such as Salesforce, SAP or ServiceNow. Exposure ...

Informatica with thorough understanding of data transformation and orchestration processes Extensive Experience with AWS services including IAM Lambda EKS S3 Data Lake EMR PySpark MWAA and Lakehouse technologies, Apache Iceberg Proven expertise with Snowflake including performance tuning and efficient query design Experience with batch pipelines eg StreamSets and streaming ...

enterprise-scale data products, defining technical strategy, architectural patterns and engineering standards while providing technical leadership across teams. Expert programming experience with Python/PySpark and advanced proficiency in SQL for large-scale data transformation, optimisation and analytics workloads. Extensive experience designing and implementing scalable data solutions on cloud ...

data architecture, data warehousing and cloud data platforms Hands-on experience with Azure technologies Experience developing ETL/ELT solutions Experience with Python, PySpark, Microsoft Fabric, streaming technologies or large-scale enterprise environments would be advantageous. In return they are offering A salary of up to £50,000 Hybrid ...

successful candidate will design and implement modern data lakehouse architectures, ensuring high performance and scalability. Key responsibilities include developing data pipelines using Python and PySpark, working with AWS services, and implementing CI/CD pipelines. Strong experience in AWS Data Engineering and knowledge of data quality frameworks are essential. ...

genomic data. You have A BSc or MSc in computer science or related fields Expertise in Python, including popular Python libraries: NumPy, Pandas, PySpark and frameworks: Django, Django Rest Framework, FastAPI Hands-on experience with both relational (e.g. PostgreSQL) and non-relational databases (e.g. Elasticsearch, Redis) Extensive experience ...

supporting the Intelligence Community or Department of Defense environments Experience with Python and working with databases Experience with big data frameworks such as Spark, PySpark, or Elastic Search Experience with data visualization platforms such as Kibana, Foundry Workshop, or Palantir Knowledge of secure coding practices and development in classified ...

well as familiarity with software engineering best practices (such as version control, unit testing, CI/CD) and big data and cloud technologies (PySpark preferred). Experience training and deploying deep learning models in cloud environments (including GPU‐enabled workflows) is a strong advantage. What ...

including architecture, development, deployment, and optimization. Define Databricks workspace architecture, clusterstrategies, security models, and operational standards. Design and developscalable data pipelinesusing Databricks, Apache Spark, PySpark, SQL, andDelta Lake . Implement data ingestionframeworks supporting batchand real-timedata processing. Design Delta Lake architectures including: Delta tables Medallion architecture (Bronze/Silver … reviews. Skills & Experience Strong experience in dataengineering, big data, or solution architecture. Strong hands-onexperience with Databricks Platform . Expert knowledge of: Apache Spark PySpark/Scala Delta Lake SQL Data Lakehouse architecture ETL/ELT development Experience designing anddelivering enterprise Databrickssolutions. Strong experience withcloud platforms: Azure Databricks (preferred ...

scalable, secure, and high-performance data platforms.Key Responsibilities Design and implement scalable AWS Data Lakehouse solutions Develop and maintain data pipelines using Python and PySpark Work with AWS services including: S3, EMR, Lambda, IAM, EKS, MWAA Build and manage Apache Iceberg tables for efficient data storage and querying Implement … engineersRequired Skills & Experience Strong experience in AWS Data Engineering Hands-on expertise with: AWS services: IAM, Lambda, EKS, S3, EMR, MWAA Apache Iceberg Python & PySpark Solid understanding of data lakehouse architecture principles Experience with CI/CD pipelines and Terraform Knowledge of data quality frameworks and data security standards ...

Lakes*** Experience with **Apache Airflow/MWAA** for workflow orchestration* Strong knowledge of **dbt (data build tool)** for data transformation* Experience with **EMR/PySpark** for large-scale data processing* Ability to create **Architecture Design Documents (ADDs)*** Deep understanding of **cloud-native design patterns*** Experience in **performance tuning** across ...