2 of 2 Spark SQL Jobs in the North East

cloud-based data pipelines for ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing … applications. Desired Qualifications: 3-5years of experience in data engineering, ideally working with large-scale or geospatial systems. Proficiency in Python and SQL (Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration ...

designing and building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem. Responsibilities Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark. Design and implement … . Advanced PySpark and Databricks experience (Delta Lake, Unity Catalog, Databricks Jobs etc). Proficient in SQL (T-SQL/SparkSQL) and Python for data transformation and scripting. Hands-on experience with workflow orchestration tools such as Airflow. Strong version control and DevOps exposure (Git, GitHub ...