2 Batch Processing Jobs in Northern Ireland

deadlines. Your main responsibilities will be: Designing and implementing highly performant data ingestion & transformation pipelines from multiple sources using Databricks and Spark Streaming and Batch processes in Databricks SPARK performance\tuning\optimisation Providing technical guidance for complex geospatial problems and spark dataframes Developing scalable and re-usable frameworks for more »

technologies for ETL, data warehouse, and data lake design. Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. Capable of processing large volumes of structured and unstructured data on AWS. Familiarity with AWS best practices in data engineering, data science, and product development. Knowledgeable in … both stream and batch processing. Comfortable designing and building for AWS cloud, including Platform-as-a-Service, server-less, and container technologies. Diverse skills in ETL, data warehouse, and Data Lake design. Familiarity with various tools, cloud technologies, and approaches. Expertise in AWS or equivalent open-source tools. Proficient more »