تقرير
DRew: Dynamically Rewired Message Passing with Delay
| العنوان: | DRew: Dynamically Rewired Message Passing with Delay |
|---|---|
| المؤلفون: | Gutteridge, Benjamin, Dong, Xiaowen, Bronstein, Michael, Di Giovanni, Francesco |
| سنة النشر: | 2023 |
| المجموعة: | Computer Science Statistics |
| مصطلحات موضوعية: | Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning |
| الوصف: | Message passing neural networks (MPNNs) have been shown to suffer from the phenomenon of over-squashing that causes poor performance for tasks relying on long-range interactions. This can be largely attributed to message passing only occurring locally, over a node's immediate neighbours. Rewiring approaches attempting to make graphs 'more connected', and supposedly better suited to long-range tasks, often lose the inductive bias provided by distance on the graph since they make distant nodes communicate instantly at every layer. In this paper we propose a framework, applicable to any MPNN architecture, that performs a layer-dependent rewiring to ensure gradual densification of the graph. We also propose a delay mechanism that permits skip connections between nodes depending on the layer and their mutual distance. We validate our approach on several long-range tasks and show that it outperforms graph Transformers and multi-hop MPNNs. Comment: Accepted at ICML 2023; 16 pages |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2305.08018 |
| رقم الأكسشن: | edsarx.2305.08018 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |