Abstract

The propose Fed-DRL++, a new framework that combines Personalized Federated Learning (PFL) with Hierarchical Deep Reinforcement Learning (HDRL) and Edge Aggregation. This setup enables real-time, decentralized, and privacy-friendly traffic management in smart cities. Unlike traditional centralized methods, Fed-DRL++ uses a layered decision-making structure. It applies Dueling DQN at the route level and a mix of Dueling DQN and PPO at zone and junction levels. This approach adjusts to local traffic conditions by using both historical and real-time mobility data. A key innovation is the use of Prioritized Experience Replay with Hindsight (HERPP). This method focuses learning on significant events like congestion and collisions. Edge aggregation lowers communication overhead by performing local updates before syncing with the cloud. Meanwhile, PFL maintains model performance even with non-IID data across users. In tests with 1,000 mobile agents across five congestion scenarios, Fed-DRL++ outperformed benchmarks. It reduced travel time by 23.4%, improved route convergence by 31%, and cut congestion by 18%. It also decreased communication costs by 41.6% and boosted prediction accuracy (RMSE: 2.91 km/h, MAE: 2.06 km/h), with results being statistically significant (p < 0.01). This shows its effectiveness for scalable and smart traffic optimization.