دورية أكاديمية
SA‐FlowNet: Event‐based self‐attention optical flow estimation with spiking‐analogue neural networks
| العنوان: | SA‐FlowNet: Event‐based self‐attention optical flow estimation with spiking‐analogue neural networks |
|---|---|
| المؤلفون: | Fan Yang, Li Su, Jinxiu Zhao, Xuena Chen, Xiangyu Wang, Na Jiang, Quan Hu |
| المصدر: | IET Computer Vision, Vol 17, Iss 8, Pp 925-935 (2023) |
| بيانات النشر: | Wiley, 2023. |
| سنة النشر: | 2023 |
| المجموعة: | LCC:Computer applications to medicine. Medical informatics LCC:Computer software |
| مصطلحات موضوعية: | computer vision, feature extraction, motion estimation, optical tracking, Computer applications to medicine. Medical informatics, R858-859.7, Computer software, QA76.75-76.765 |
| الوصف: | Abstract Inspired by biological vision mechanism, event‐based cameras have been developed to capture continuous object motion and detect brightness changes independently and asynchronously, which overcome the limitations of traditional frame‐based cameras. Complementarily, spiking neural networks (SNNs) offer asynchronous computations and exploit the inherent sparseness of spatio‐temporal events. Notably, event‐based pixel‐wise optical flow estimations calculate the positions and relationships of objects in adjacent frames; however, as event camera outputs are sparse and uneven, dense scene information is difficult to generate and the local receptive fields of the neural network also lead to poor moving objects tracking. To address these issues, an improved event‐based self‐attention optical flow estimation network (SA‐FlowNet) that independently uses criss‐cross and temporal self‐attention mechanisms, directly capturing long‐range dependencies and efficiently extracting the temporal and spatial features from the event streams is proposed. In the former mechanism, a cross‐domain attention scheme dynamically fusing the temporal‐spatial features is introduced. The proposed network adopts a spiking‐analogue neural network architecture using an end‐to‐end learning method and gains significant computational energy benefits especially for SNNs. The state‐of‐the‐art results of the error rate for optical flow prediction on the Multi‐Vehicle Stereo Event Camera (MVSEC) dataset compared with the current SNN‐based approaches is demonstrated. |
| نوع الوثيقة: | article |
| وصف الملف: | electronic resource |
| اللغة: | English |
| تدمد: | 1751-9640 1751-9632 |
| Relation: | https://doaj.org/toc/1751-9632; https://doaj.org/toc/1751-9640 |
| DOI: | 10.1049/cvi2.12206 |
| URL الوصول: | https://doaj.org/article/05714d28e2f74398a43eaba8c904f0e0 |
| رقم الأكسشن: | edsdoj.05714d28e2f74398a43eaba8c904f0e0 |
| قاعدة البيانات: | Directory of Open Access Journals |
Full Text Finder | تدمد: | 17519640 17519632 |
|---|---|
| DOI: | 10.1049/cvi2.12206 |