4 of 4 Apache Spark Jobs in Derby

Data Engineering, with a minimum of 3 years of hands-on Azure Databricks experience delivering production-grade solutions. Strong programming proficiency in Python and Spark (PySpark) or Scala, with the ability to build scalable and efficient data processing applications. Advanced understanding of data warehousing concepts, including dimensional modelling

frameworks. Strong expertise in statistical modelling, predictive analytics, and machine learning. In-depth understanding of data architecture and big data technologies (e.g., Hadoop, Spark, AWS). Exceptional problem-solving skills and ability to think strategically. Outstanding communication abilities to convey technical concepts to non-technical stakeholders. Experience in managing

/test workloads are self-orchestrated using clustered Proxmox as a private cloud. Our analytics workloads are distributed on bare metal using Spark, HBASE and HDFS. We make extensive use of terraform and ansible for IaC, our CI/CD uses GitHub Actions, our observability is provided via Datadog

production at scale. Technical Skills: Strong backend Python; expertise with ML frameworks (TensorFlow, PyTorch, Scikit-Learn, etc.); familiarity with modern data pipeline tools (Airflow, Spark, Kafka) and workflow orchestration frameworks such as n8n or LangGraph. Applied AI: Background in enterprise applications such as chatbots, workflow automation, RAG pipelines