Yu, Sixing ; Mazaheri, Arya ; Jannesari, Ali (2022)
Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning.
39th International Conference on Machine Learning. Baltimore, USA (17.-23.07.2022)
Konferenzveröffentlichung, Bibliographie
Kurzbeschreibung (Abstract)
Model compression is an essential technique for deploying deep neural networks (DNNs) on power and memory-constrained resources. However, existing model-compression methods often rely on human expertise and focus on parameters’ local importance, ignoring the rich topology information within DNNs. In this paper, we propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify DNN topologies and use reinforcement learning (RL) to find a suitable compression policy. We performed resource-constrained (i.e., FLOPs) channel pruning and compared our approach with state-of-the-art model compression methods. We evaluated our method on various models from typical to mobile-friendly networks, such as ResNet family, VGG-16, MobileNet-v1/v2, and ShuffleNet. Results show that our method can achieve higher compression ratios with a minimal fine-tuning cost yet yields outstanding and competitive performance.
Typ des Eintrags: | Konferenzveröffentlichung |
---|---|
Erschienen: | 2022 |
Autor(en): | Yu, Sixing ; Mazaheri, Arya ; Jannesari, Ali |
Art des Eintrags: | Bibliographie |
Titel: | Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning |
Sprache: | Englisch |
Publikationsjahr: | 24 Juli 2022 |
Verlag: | PMLR |
Buchtitel: | Proceedings of the 39th International Conference on Machine Learning (ICML), |
Reihe: | Proceedings of Machine Learning Research |
Band einer Reihe: | 162 |
Veranstaltungstitel: | 39th International Conference on Machine Learning |
Veranstaltungsort: | Baltimore, USA |
Veranstaltungsdatum: | 17.-23.07.2022 |
URL / URN: | https://proceedings.mlr.press/v162/yu22e.html |
Kurzbeschreibung (Abstract): | Model compression is an essential technique for deploying deep neural networks (DNNs) on power and memory-constrained resources. However, existing model-compression methods often rely on human expertise and focus on parameters’ local importance, ignoring the rich topology information within DNNs. In this paper, we propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify DNN topologies and use reinforcement learning (RL) to find a suitable compression policy. We performed resource-constrained (i.e., FLOPs) channel pruning and compared our approach with state-of-the-art model compression methods. We evaluated our method on various models from typical to mobile-friendly networks, such as ResNet family, VGG-16, MobileNet-v1/v2, and ShuffleNet. Results show that our method can achieve higher compression ratios with a minimal fine-tuning cost yet yields outstanding and competitive performance. |
Freie Schlagworte: | deep learning, network pruning, reinforcement learning |
Fachbereich(e)/-gebiet(e): | 20 Fachbereich Informatik 20 Fachbereich Informatik > Parallele Programmierung |
Hinterlegungsdatum: | 13 Feb 2024 15:22 |
Letzte Änderung: | 25 Apr 2024 09:44 |
PPN: | 517482282 |
Export: | |
Suche nach Titel in: | TUfind oder in Google |
Frage zum Eintrag |
Optionen (nur für Redakteure)
Redaktionelle Details anzeigen |