TU Darmstadt / ULB / TUbiblio

Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning

Yu, Sixing ; Mazaheri, Arya ; Jannesari, Ali (2022)
Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning.
39th International Conference on Machine Learning. Baltimore, USA (17.-23.07.2022)
Konferenzveröffentlichung, Bibliographie

Kurzbeschreibung (Abstract)

Model compression is an essential technique for deploying deep neural networks (DNNs) on power and memory-constrained resources. However, existing model-compression methods often rely on human expertise and focus on parameters’ local importance, ignoring the rich topology information within DNNs. In this paper, we propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify DNN topologies and use reinforcement learning (RL) to find a suitable compression policy. We performed resource-constrained (i.e., FLOPs) channel pruning and compared our approach with state-of-the-art model compression methods. We evaluated our method on various models from typical to mobile-friendly networks, such as ResNet family, VGG-16, MobileNet-v1/v2, and ShuffleNet. Results show that our method can achieve higher compression ratios with a minimal fine-tuning cost yet yields outstanding and competitive performance.

Typ des Eintrags: Konferenzveröffentlichung
Erschienen: 2022
Autor(en): Yu, Sixing ; Mazaheri, Arya ; Jannesari, Ali
Art des Eintrags: Bibliographie
Titel: Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning
Sprache: Englisch
Publikationsjahr: 24 Juli 2022
Verlag: PMLR
Buchtitel: Proceedings of the 39th International Conference on Machine Learning (ICML),
Reihe: Proceedings of Machine Learning Research
Band einer Reihe: 162
Veranstaltungstitel: 39th International Conference on Machine Learning
Veranstaltungsort: Baltimore, USA
Veranstaltungsdatum: 17.-23.07.2022
URL / URN: https://proceedings.mlr.press/v162/yu22e.html
Kurzbeschreibung (Abstract):

Model compression is an essential technique for deploying deep neural networks (DNNs) on power and memory-constrained resources. However, existing model-compression methods often rely on human expertise and focus on parameters’ local importance, ignoring the rich topology information within DNNs. In this paper, we propose a novel multi-stage graph embedding technique based on graph neural networks (GNNs) to identify DNN topologies and use reinforcement learning (RL) to find a suitable compression policy. We performed resource-constrained (i.e., FLOPs) channel pruning and compared our approach with state-of-the-art model compression methods. We evaluated our method on various models from typical to mobile-friendly networks, such as ResNet family, VGG-16, MobileNet-v1/v2, and ShuffleNet. Results show that our method can achieve higher compression ratios with a minimal fine-tuning cost yet yields outstanding and competitive performance.

Freie Schlagworte: deep learning, network pruning, reinforcement learning
Fachbereich(e)/-gebiet(e): 20 Fachbereich Informatik
20 Fachbereich Informatik > Parallele Programmierung
Hinterlegungsdatum: 13 Feb 2024 15:22
Letzte Änderung: 25 Apr 2024 09:44
PPN: 517482282
Export:
Suche nach Titel in: TUfind oder in Google
Frage zum Eintrag Frage zum Eintrag

Optionen (nur für Redakteure)
Redaktionelle Details anzeigen Redaktionelle Details anzeigen