Zeng, Wenjun (2018)
Robust Low-Rank Approximation of Matrices in lp-Space.
Technische Universität Darmstadt
Dissertation, Erstveröffentlichung
Kurzbeschreibung (Abstract)
Low-rank approximation plays an important role in many areas of science and engineering such as signal/image processing, machine learning, data mining, imaging, bioinformatics, pattern classification and computer vision because many real-world data exhibit low-rank property. This dissertation devises advanced algorithms for robust low-rank approximation of a single matrix as well as multiple matrices in the presence of outliers, where the conventional dimensionality reduction techniques such as the celebrated principal component analysis (PCA) are not applicable. The proposed methodology is based on minimizing the entry-wise $\ell_p$-norm of the residual including the challenging nonconvex and nonsmooth case of $p<1$. Theoretical analyses are also presented. Extensive practical applications are discussed. Experimental results demonstrate that the superiority of the proposed methods over the state-of-the-art techniques.
Two iterative algorithms are designed for low-rank approximation of a single matrix. The first is the iteratively reweighted singular value decomposition (IR-SVD), where the SVD of a reweighted matrix is performed at each iteration. The second converts the nonconvex $\ell_p$-matrix factorization into a series of easily solvable $\ell_p$-norm minimization with vectors being variables. Applications to image demixing, foreground detection in video surveillance, array signal processing, and direction-of-arrival estimation for source localization in impulsive noise are investigated.
The low-rank approximation with missing values, i.e., robust matrix completion, is also addressed. Two algorithms are developed for it. The first iteratively solves a set of linear $\ell_p$-regression problems while the second applies the alternating direction method of multipliers (ADMM) in the $\ell_p$-space. At each iteration of the ADMM, it requires performing a least squares (LS) matrix factorization and calculating the proximity operator of the $p$th power of the $\ell_p$-norm. The LS factorization is efficiently solved using linear LS regression while the proximity operator is obtained by root finding of a scalar nonlinear equation. The two proposed algorithms are scalable to the problem size. Applications to recommender systems, collaborative filtering, and image inpainting are provided.
The $\ell_p$-greedy pursuit ($\ell_p$-GP) algorithms are devised for joint robust low-rank approximation of multiple matrices (RLRAMM) with outliers. The $\ell_p$-GP with $0<p<2$ solves the RLRAMM by decomposing it into a series of rank-one approximations. At each iteration, it finds the best rank-one approximation by minimizing the $\ell_p$-norm of the residual and then, the rank-one basis matrices are subtracted from the residual. A successive minimization approach is designed for the $\ell_p$-rank-one fitting. Only weighted medians are required to compute for solving the most attractive case with $p=1$, yielding that the complexity is near-linear with the number and dimension of the matrices. Thus, the $\ell_1$-GP is near-scalable to large-scale problems. The convergence of the $\ell_p$-GP is theoretically proved. In particular, the sum of the $\ell_p$-norms of the residuals decays exponentially. We reveal that the worst-case bound of the convergence rate is related to the $\ell_p$-correlation of the residual and the current solution. The $\ell_p$-GP has a higher compression ratio than the existing methods. For the special case of $p=2$, the orthogonal greedy pursuit (OGP) is further developed to accelerate the convergence, where the cost of weight re-computation is reduced by a recursive update manner. Tighter and more accurate bounds of the convergence rates are theoretically derived for $p=2$. Applications to data compression, robust image reconstruction and computer vision are provided.
Typ des Eintrags: | Dissertation | ||||
---|---|---|---|---|---|
Erschienen: | 2018 | ||||
Autor(en): | Zeng, Wenjun | ||||
Art des Eintrags: | Erstveröffentlichung | ||||
Titel: | Robust Low-Rank Approximation of Matrices in lp-Space | ||||
Sprache: | Englisch | ||||
Referenten: | Zoubir, Prof. Dr. Abdelhak ; So, Prof. Dr. Hing Cheung | ||||
Publikationsjahr: | 8 Juli 2018 | ||||
Ort: | Darmstadt | ||||
Datum der mündlichen Prüfung: | 5 Juli 2018 | ||||
URL / URN: | http://tuprints.ulb.tu-darmstadt.de/7564 | ||||
Kurzbeschreibung (Abstract): | Low-rank approximation plays an important role in many areas of science and engineering such as signal/image processing, machine learning, data mining, imaging, bioinformatics, pattern classification and computer vision because many real-world data exhibit low-rank property. This dissertation devises advanced algorithms for robust low-rank approximation of a single matrix as well as multiple matrices in the presence of outliers, where the conventional dimensionality reduction techniques such as the celebrated principal component analysis (PCA) are not applicable. The proposed methodology is based on minimizing the entry-wise $\ell_p$-norm of the residual including the challenging nonconvex and nonsmooth case of $p<1$. Theoretical analyses are also presented. Extensive practical applications are discussed. Experimental results demonstrate that the superiority of the proposed methods over the state-of-the-art techniques. Two iterative algorithms are designed for low-rank approximation of a single matrix. The first is the iteratively reweighted singular value decomposition (IR-SVD), where the SVD of a reweighted matrix is performed at each iteration. The second converts the nonconvex $\ell_p$-matrix factorization into a series of easily solvable $\ell_p$-norm minimization with vectors being variables. Applications to image demixing, foreground detection in video surveillance, array signal processing, and direction-of-arrival estimation for source localization in impulsive noise are investigated. The low-rank approximation with missing values, i.e., robust matrix completion, is also addressed. Two algorithms are developed for it. The first iteratively solves a set of linear $\ell_p$-regression problems while the second applies the alternating direction method of multipliers (ADMM) in the $\ell_p$-space. At each iteration of the ADMM, it requires performing a least squares (LS) matrix factorization and calculating the proximity operator of the $p$th power of the $\ell_p$-norm. The LS factorization is efficiently solved using linear LS regression while the proximity operator is obtained by root finding of a scalar nonlinear equation. The two proposed algorithms are scalable to the problem size. Applications to recommender systems, collaborative filtering, and image inpainting are provided. The $\ell_p$-greedy pursuit ($\ell_p$-GP) algorithms are devised for joint robust low-rank approximation of multiple matrices (RLRAMM) with outliers. The $\ell_p$-GP with $0<p<2$ solves the RLRAMM by decomposing it into a series of rank-one approximations. At each iteration, it finds the best rank-one approximation by minimizing the $\ell_p$-norm of the residual and then, the rank-one basis matrices are subtracted from the residual. A successive minimization approach is designed for the $\ell_p$-rank-one fitting. Only weighted medians are required to compute for solving the most attractive case with $p=1$, yielding that the complexity is near-linear with the number and dimension of the matrices. Thus, the $\ell_1$-GP is near-scalable to large-scale problems. The convergence of the $\ell_p$-GP is theoretically proved. In particular, the sum of the $\ell_p$-norms of the residuals decays exponentially. We reveal that the worst-case bound of the convergence rate is related to the $\ell_p$-correlation of the residual and the current solution. The $\ell_p$-GP has a higher compression ratio than the existing methods. For the special case of $p=2$, the orthogonal greedy pursuit (OGP) is further developed to accelerate the convergence, where the cost of weight re-computation is reduced by a recursive update manner. Tighter and more accurate bounds of the convergence rates are theoretically derived for $p=2$. Applications to data compression, robust image reconstruction and computer vision are provided. |
||||
Alternatives oder übersetztes Abstract: |
|
||||
URN: | urn:nbn:de:tuda-tuprints-75642 | ||||
Sachgruppe der Dewey Dezimalklassifikatin (DDC): | 000 Allgemeines, Informatik, Informationswissenschaft > 004 Informatik | ||||
Fachbereich(e)/-gebiet(e): | 18 Fachbereich Elektrotechnik und Informationstechnik 18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik 18 Fachbereich Elektrotechnik und Informationstechnik > Institut für Nachrichtentechnik > Signalverarbeitung |
||||
Hinterlegungsdatum: | 22 Jul 2018 19:55 | ||||
Letzte Änderung: | 27 Nov 2018 09:00 | ||||
PPN: | |||||
Referenten: | Zoubir, Prof. Dr. Abdelhak ; So, Prof. Dr. Hing Cheung | ||||
Datum der mündlichen Prüfung / Verteidigung / mdl. Prüfung: | 5 Juli 2018 | ||||
Export: | |||||
Suche nach Titel in: | TUfind oder in Google |
Frage zum Eintrag |
Optionen (nur für Redakteure)
Redaktionelle Details anzeigen |