The rapid advancement of Internet of Things (IoT) and Artificial Intelligence (AI) offers great potentials for mitigating traffic congestion in metropolitan areas. In recent years, deep reinforcement learning (DRL)-based traffic signal control has attracted tremendous attentions and expectations from both researchers and practitioners. Despite the excellent performance of the DRL-based traffic signal control algorithms in simulated environments and pilot sites, few studies addressed their robustness and safety in real world. The discrepancies between training and testing environments can easily result in control failure with escalated congestion and collision risk. Additionally, conventional DRL-based traffic signal control algorithms tend to exhibit vulnerabilities to potential cyber-attacks (e.g., falsified information or maliciously modified data). In order to improve the trustworthiness in DRL-based traffic signal control, this thesis introduces a staged multiscale learning approach (referred to as StageLight) for isolated single intersections by exploiting physical-guided transportation knowledge for effective regularization. The experimental results revealed StageLight’s superior performance on generalization to diverse traffic flows and resilience to sensor failure or demand surge.