Access the full text.
Sign up today, get DeepDyve free for 14 days.
T. Cai, Wenxin Zhou (2013)
Matrix completion via max-norm constrained optimizationArXiv, abs/1303.0341
Q Xie, Q Zhao, D Meng (2017)
IEEE Trans Pattern Anal Mach Intell, 40
Zhiqiang Xu (2010)
Deterministic sampling of sparse trigonometric polynomialsArXiv, abs/1006.2221
S. Foucart, H. Rauhut (2013)
A Mathematical Introduction to Compressive Sensing
T. Hansen, Peter Miltersen, Uri Zwick (2010)
Strategy Iteration Is Strongly Polynomial for 2-Player Turn-Based Stochastic Games with a Constant Discount Factor
Y Wang, L Lin, Q Zhao (2017)
IEEE Geosci Rem Sens Lett, 14
E. Candès, T. Tao (2009)
The Power of Convex Relaxation: Near-Optimal Matrix CompletionIEEE Transactions on Information Theory, 56
B. Recht, Maryam Fazel, P. Parrilo (2007)
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm MinimizationSIAM Rev., 52
(2011)
Deterministic sampling of sparse trigonometric polynomialsJ Complex, 27
D Takhar, JN Laska, M Wakin (2006)
Proceedings of Computational Imaging IV at SPIE Electronic Imaging
M Pauly, N Mitra, J Giesen (2005)
Proceedings of the Symposium on Geometry Processing
Han Guo, Chenlu Qiu, Namrata Vaswani (2013)
An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their SumIEEE Transactions on Signal Processing, 62
Zongben Xu, Hai Zhang, Yao Wang, Xiangyu Chang, Yong Liang (2010)
L1/2 regularizationScience China Information Sciences, 53
E. Candès, Y. Plan (2011)
Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random MeasurementsIEEE Transactions on Information Theory, 57
J Lin, S Li (2014)
Sparse recovery with coherent tight frames via analydsis Dantzig selector and analysis LASSOAppl Comput Harmon Anal, 37
D. Takhar, J. Laska, M. Wakin, Marco Duarte, D. Baron, S. Sarvotham, K. Kelly, Richard Baraniuk (2006)
A new compressive imaging camera architecture using optical-domain compression, 6065
E. Candès, T. Tao (2005)
Decoding by linear programmingIEEE Transactions on Information Theory, 51
V Voroninski, Z Xu (2016)
A strong restricted isometry property, with an application to phaseless compressed sensingAppl Comput Harmon Anal, 40
Yulong Liu, Tiebin Mi, Shidong Li (2011)
Compressed Sensing With General Frames via Optimal-Dual-Based $\ell _{1}$-AnalysisIEEE Transactions on Information Theory, 58
E Candès, Y Eldar, D Needell (2010)
Compressed sensing with coherent and redundant dictionariesAppl Comput Harmon Anal, 31
J. Justesen, C. Thommesen, T. Høholdt (2004)
Proceedings of the 2004 IEEE International Symposium on Information Theory
Mohammad Golbabaee, P. Vandergheynst (2012)
Hyperspectral image compressed sensing via low-rank and joint-sparse matrix recovery2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
M Elad, P Milanfar, R Rubinstein (2007)
Analysis versus synthesis in signal priorsAppl Comput Harmon Anal, 23
Csaba Beleznai, B. Frühstück, H. Bischof (2006)
Multiple Object Tracking Using Local PCA18th International Conference on Pattern Recognition (ICPR'06), 3
E. Candès, J. Romberg, T. Tao (2005)
Stable signal recovery from incomplete and inaccurate measurementsCommunications on Pure and Applied Mathematics, 59
Qiyu Sun (2010)
Recovery of sparsest signals via ℓq-minimizationArXiv, abs/1005.0267
Chun-Guang Li, R. Vidal (2016)
A Structured Sparse Plus Structured Low-Rank Framework for Subspace Clustering and CompletionIEEE Transactions on Signal Processing, 64
M. Yuan, Cun-Hui Zhang (2014)
On Tensor Completion via Nuclear Norm MinimizationFoundations of Computational Mathematics, 16
Zai Yang, Cisheng Zhang, Lihua Xie (2013)
Robust Compressive Phase Retrieval via L1 Minimization With Application to Image ReconstructionArXiv, abs/1302.0081
M. Moravec, J. Romberg, Richard Baraniuk (2007)
Compressive phase retrieval, 6701
E. Candès, J. Romberg, T. Tao (2004)
Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency informationIEEE Transactions on Information Theory, 52
Chaobing Song, Shutao Xia (2013)
Sparse Signal Recovery by ${\ell_q}$ Minimization Under Restricted Isometry PropertyIEEE Signal Processing Letters, 21
Yao Wang, Jianjun Wang, Zongben Xu (2013)
On recovery of block-sparse signals via mixed l 2 / l q ( 0 < q ≤ 1 ) normminimization
Zihan Zhou, Xiaodong Li, John Wright, E. Candès, Yi Ma (2010)
Stable Principal Component Pursuit2010 IEEE International Symposium on Information Theory
Man-Chung Yue, A. So (2016)
A perturbation inequality for concave functions of singular values and its applications in low-rank matrix recoveryApplied and Computational Harmonic Analysis, 40
Min Zhang, Zhenghai Huang, Y. Zhang (2013)
Restricted $p$ -Isometry Properties of Nonconvex Matrix RecoveryIEEE Transactions on Information Theory, 59
Jianqing Fan, Runze Li (2001)
Variable Selection via Nonconcave Penalized Likelihood and its Oracle PropertiesJournal of the American Statistical Association, 96
V. Vianu (2010)
Invited articles section forewordJ. ACM, 58
M. Wakin, J. Laska, Marco Duarte, D. Baron, S. Sarvotham, D. Takhar, K. Kelly, Richard Baraniuk (2006)
Compressive imaging for video representation and coding
Chengbo Li, Ting Sun, K. Kelly, Yin Zhang (2012)
A Compressive Sensing and Unmixing Scheme for Hyperspectral Data ProcessingIEEE Transactions on Image Processing, 21
Ji Liu, Przemyslaw Musialski, Peter Wonka, Jieping Ye (2009)
Tensor completion for estimating missing values in visual data2009 IEEE 12th International Conference on Computer Vision
Qian Zhao, Deyu Meng, Zongben Xu, W. Zuo, Lei Zhang (2014)
Robust Principal Component Analysis with Complex Noise
Shuyuan Yang, Min Wang, Peng Li, L. Jin, Bin Wu, L. Jiao (2015)
Compressive Hyperspectral Imaging via Sparse Tensor and Nonlinear Compressed SensingIEEE Transactions on Geoscience and Remote Sensing, 53
C. Hillar, Lek-Heng Lim (2009)
Most Tensor Problems Are NP-HardArXiv, abs/0911.1393
Y Eldar, G Kutyniok (2012)
Compressed Sensing: Theory and Applications
E. Candès, T. Tao (2004)
Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?IEEE Transactions on Information Theory, 52
D. Donoho, Michael Elad (2003)
Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimizationProceedings of the National Academy of Sciences of the United States of America, 100
Marco Duarte, Richard Baraniuk (2012)
Kronecker Compressive SensingIEEE Transactions on Image Processing, 21
E Candès, X Li, Y Ma (2011)
Robust principal component analysis?J ACM, 58
(2012)
Recovery of sparsest signals via ?q-minimizationAppl Comput Harmon Anal, 32
K Xie, L Wang, W X (2016)
Proceedings of the 35th Annual IEEE International Conference on Computer Communications
Jiawang Nie, Li Wang (2013)
Semidefinite Relaxations for Best Rank-1 Tensor ApproximationsSIAM J. Matrix Anal. Appl., 35
Yao Wang, Lin Lin, Qian Zhao, Tianwei Yue, Deyu Meng, Yee Leung (2017)
Compressive Sensing of Hyperspectral Images via Joint Tensor Tucker Decomposition and Weighted Total Variation RegularizationIEEE Geoscience and Remote Sensing Letters, 14
H Jiang, W Deng, Z Shen (2014)
Surveillance video processing using compressive sensingInverse Probl Imag, 6
V. Chandrasekaran, S. Sanghavi, P. Parrilo, A. Willsky (2009)
Rank-Sparsity Incoherence for Matrix DecompositionSIAM J. Optim., 21
H. Rauhut, K. Schnass, P. Vandergheynst (2007)
Compressed Sensing and Redundant DictionariesIEEE Transactions on Information Theory, 54
Praneeth Netrapalli, U. Niranjan, S. Sanghavi, Anima Anandkumar, Prateek Jain (2014)
Non-convex Robust PCAArXiv, abs/1410.7660
B Eriksson, L Balzano, R Nowak (2012)
Proceedings of the 15th International Conference on Artificial Intelligence and Statistics
Cun-Hui Zhang (2010)
Nearly unbiased variable selection under minimax concave penaltyAnnals of Statistics, 38
Tommaso Cai, Lie Wang, Guangwu Xu (2009)
New Bounds for Restricted Isometry ConstantsIEEE Transactions on Information Theory, 56
M Moravec, J Romberg, R Baraniuk (2007)
Proceedings of SPIE
M Wakin, JN Laska, MF Duarte (2006)
Proceedings of Picture Coding Symposium
Yuning Yang, Yunlong Feng, J. Suykens (2015)
A Rank-One Tensor Updating Algorithm for Tensor CompletionIEEE Signal Processing Letters, 22
E Acar, D Dunlavy, T Kolda (2001)
Scalable tensor factorizations for incomplete dataChemometr Intell Lab Syst, 106
Y Peng, D Meng, Z Xu (2014)
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
Arvind Ganesh, John Wright, Xiaodong Li, E. Candès, Yi Ma (2010)
Dense error correction for low-rank matrices via Principal Component Pursuit2010 IEEE International Symposium on Information Theory
E. Candès (2008)
The restricted isometry property and its implications for compressed sensingComptes Rendus Mathematique, 346
M Golbabaee, P Vandergheynst (2012)
Proceedings of 19th IEEE International Conference on Image Processing
Yudong Chen, Srinadh Bhojanapalli, Sujay Sanghavi, Rachel Ward (2013)
Coherent Matrix CompletionArXiv, abs/1306.2979
(2016)
Proceedings of Advances in Neural Information Processing Systems
Yonina Eldar, M. Mishali (2008)
Robust Recovery of Signals From a Structured Union of SubspacesIEEE Transactions on Information Theory, 55
V. Voroninski, Zhiqiang Xu (2014)
A strong restricted isometry property, with an application to phaseless compressed sensingArXiv, abs/1404.3811
S. Chen, D. Donoho, M. Saunders (1998)
Atomic Decomposition by Basis PursuitSIAM J. Sci. Comput., 20
Tommaso Cai, Lie Wang, Guangwu Xu (2010)
Shifting Inequality and Recovery of Sparse SignalsIEEE Transactions on Signal Processing, 58
J. Bourgain, S. Dilworth, Kevin Ford, S. Konyagin, D. Kutzarova (2010)
Explicit constructions of RIP matrices and related problemsArXiv, abs/1008.4535
Thommen Korah, C. Rasmussen (2007)
Spatiotemporal Inpainting for Recovering Texture Maps of Occluded Building FacadesIEEE Transactions on Image Processing, 16
Zongben Xu, Xiangyu Chang, Fengmin Xu, Hai Zhang (2012)
$L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast SolverIEEE Transactions on Neural Networks and Learning Systems, 23
Jing Wang, Meng Wang, Xuegang Hu, Shuicheng Yan (2015)
Visual data denoising with a unified Schatten-p norm and ℓq norm regularized principal component pursuitPattern Recognit., 48
V. Cevher, Aswin Sankaranarayanan, Marco Duarte, Dikpal Reddy, Richard Baraniuk, R. Chellappa (2008)
Compressive Sensing for Background Subtraction
Mohammad Golbabaee, P. Vandergheynst (2012)
Joint trace/TV norm minimization: A new efficient approach for spectral compressive imaging2012 19th IEEE International Conference on Image Processing
Song Li, Junhong Lin (2011)
Compressed Sensing with coherent tight frames via $l_q$-minimization for $0ArXiv, abs/1105.3299
V Cevher, A Sankaranarayanan, M Duarte (2008)
Proceedings of the 10th European Conference on Computer Vision
Hong Jiang, Songqing Zhao, Zuowei Shen, W. Deng, P. Wilford, R. Haimi-Cohen (2014)
Surveillance Video Analysis Using Compressive Sensing With Low LatencyBell Labs Tech. J., 18
E. Candès, Xiaodong Li, Yi Ma, John Wright (2009)
Robust principal component analysis?ArXiv, abs/0912.3599
Q Zhao, D Meng, Z Xu (2014)
Proceedings of the 31st International Conference on Machine Learning
Tiesheng Wang, Andrew Backhouse, I. Gu (2008)
Online subspace learning on Grassmann manifold for moving object tracking in video2008 IEEE International Conference on Acoustics, Speech and Signal Processing
Tommaso Cai, Anru Zhang (2013)
Compressed Sensing and Affine Rank Minimization Under Restricted IsometryIEEE Transactions on Signal Processing, 61
Marco Duarte, M. Davenport, D. Takhar, J. Laska, Ting Sun, K. Kelly, Richard Baraniuk (2008)
Single-pixel imaging via compressive samplingIEEE Signal Process. Mag., 25
CJ Hillar, LH Lim (2013)
Most tensor problems are NP-hardJ ACM, 60
Yi Peng, Deyu Meng, Zongben Xu, Chenqiang Gao, Yi Yang, Biao Zhang (2014)
Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising2014 IEEE Conference on Computer Vision and Pattern Recognition
Michael Elad, P. Milanfar, R. Rubinstein (2006)
Analysis versus synthesis in signal priorsInverse Problems, 23
Silvia Gandy, B. Recht, I. Yamada (2011)
Tensor completion and low-n-rank tensor recovery via convex optimizationInverse Problems, 27
D. Gross (2009)
Recovering Low-Rank Matrices From Few Coefficients in Any BasisIEEE Transactions on Information Theory, 57
M. Pauly, N. Mitra, Joachim Giesen, M. Gross, L. Guibas (2005)
Example-Based 3D Scan Completion
A Waters, A Sankaranarayanan, R Baraniuk (2011)
Proceedings of Conference on Neural Information Processing Systems 24
M Golbabaee, P Vandergheynst (2012)
Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing
Tommaso Cai, Anru Zhang (2013)
Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-Rank MatricesIEEE Transactions on Information Theory, 60
R. DeVore (2007)
Deterministic constructions of compressed sensing matricesJ. Complex., 23
D. Donoho, Michael Elad, V. Temlyakov (2006)
Stable recovery of sparse overcomplete representations in the presence of noiseIEEE Transactions on Information Theory, 52
Brian Eriksson, L. Balzano, R. Nowak (2012)
High-Rank Matrix Completion
Qian Zhao, Deyu Meng, Zongben Xu, W. Zuo, Yan Yan (2015)
$L_{1}$ -Norm Low-Rank Matrix Factorization by Variational Bayesian MethodIEEE Transactions on Neural Networks and Learning Systems, 26
B. Jiang, Shiqian Ma, Shuzhong Zhang (2012)
Tensor principal component analysis via convex optimizationMathematical Programming, 150
A. Goetz (2009)
Three decades of hyperspectral remote sensing of the Earth: a personal view.Remote Sensing of Environment, 113
D. Donoho, X. Huo (2001)
Uncertainty principles and ideal atomic decompositionIEEE Trans. Inf. Theory, 47
G. Arce, D. Brady, L. Carin, H. Arguello, D. Kittle (2014)
Compressive Coded Aperture Spectral Imaging: An IntroductionIEEE Signal Processing Magazine, 31
R. Chartrand (2007)
Exact Reconstruction of Sparse Signals via Nonconvex MinimizationIEEE Signal Processing Letters, 14
J Bourgain, SJ Dilworth, K Ford (2011)
Explicit constructions of RIP matrices and related problemsDuke Math J, 159
Qi Xie, Qian Zhao, Deyu Meng, Zongben Xu (2018)
Kronecker-Basis-Representation Based Tensor Sparsity and Its Applications to Tensor RecoveryIEEE Transactions on Pattern Analysis and Machine Intelligence, 40
Jinshan Zeng, Shaobo Lin, Yao Wang, Zongben Xu (2013)
$L_{1/2}$ Regularization: Convergence of Iterative Half Thresholding AlgorithmIEEE Transactions on Signal Processing, 62
Kun Xie, Lele Wang, Xin Wang, Gaogang Xie, Jigang Wen, Guangxin Zhang (2016)
Accurate recovery of Internet traffic data: A tensor completion approachIEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications
Rina Foygel, R. Salakhutdinov, O. Shamir, N. Srebro (2011)
Learning with the weighted trace-norm under arbitrary sampling distributions
Q Zhao, D Meng, X Kong (2015)
Proceedings of International Conference on Computer Vision
DL Donoho, M Elad (2003)
Optimally sparse representation in general (nonorthogonal) dictionaries via ?1 minimizationProc Natl Acad Sci, 100
S Li, J Lin (2014)
Compressed sensing with coherent tight frames via ?q-minimization for 0�<�q ? 1Inverse Probl Imag, 8
Philip Schniter, S. Rangan (2012)
Compressive Phase Retrieval via Generalized Approximate Message PassingIEEE Transactions on Signal Processing, 63
Xin Yuan, Tsung-Han Tsai, Ruoyu Zhu, P. Llull, D. Brady, L. Carin (2015)
Compressive Hyperspectral Imaging With Side InformationIEEE Journal of Selected Topics in Signal Processing, 9
Junhong Lin, Song Li, Yi Shen (2013)
New Bounds for Restricted Isometry Constants With Coherent Tight FramesIEEE Transactions on Signal Processing, 61
E. Acar, T. Kolda, Daniel Dunlavy, Morten Mørup (2010)
Scalable Tensor Factorizations for Incomplete DataArXiv, abs/1005.2197
TT Cai, WX Zhou (2016)
Matrix completion via max-norm constrained optimizationElectron J Stat, 10
Wenfei Cao, Yao Wang, Jian Sun, Deyu Meng, Can Yang, A. Cichocki, Zongben Xu (2015)
Total Variation Regularized Tensor RPCA for Background Subtraction From Compressive MeasurementsIEEE Transactions on Image Processing, 25
E. Candès, Yonina Eldar, D. Needell
Applied and Computational Harmonic Analysis Compressed Sensing with Coherent and Redundant Dictionaries
Qian Zhao, Deyu Meng, Xu Kong, Qi Xie, Wenfei Cao, Yao Wang, Zongben Xu (2015)
A Novel Sparsity Measure for Tensor Recovery2015 IEEE International Conference on Computer Vision (ICCV)
Gitta Kutyniok (2012)
Compressed Sensing
Junhong Lin, Song Li (2013)
Sparse Recovery with Coherent Tight Frame via Analysis Dantzig Selector and Analysis LASSOArXiv, abs/1301.3248
E. Candès, B. Recht (2008)
Exact Matrix Completion via Convex OptimizationFoundations of Computational Mathematics, 9
Ryota Tomioka, K. Hayashi, H. Kashima (2010)
Estimation of low-rank tensors via convex optimizationarXiv: Machine Learning
R. Willett, Marco Duarte, M. Davenport, Richard Baraniuk (2014)
Sparsity and Structure in Hyperspectral Imaging : Sensing, Reconstruction, and Target DetectionIEEE Signal Processing Magazine, 31
B. Natarajan (1995)
Sparse Approximate Solutions to Linear SystemsSIAM J. Comput., 24
Hong Jiang, W. Deng, Zuowei Shen (2012)
Surveillance Video Processing Using Compressive SensingArXiv, abs/1302.1942
C Beleznai, B Fruhstuck, H Bischof (2006)
Proceedings of the 18th International Conference on Pattern Recognition
A. Cohen, W. Dahmen, R. DeVore (2008)
Compressed sensing and best k-term approximationJournal of the American Mathematical Society, 22
Ehsan Elhamifar (2016)
High-Rank Matrix Completion and Clustering under Self-Expressive Models
Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 National Science Review 5: 756–767, 2018 REVIEW doi: 10.1093/nsr/nwx069 Advance access publication 14 July 2017 INFORMATION SCIENCE 1,2,∗ 1,3,∗ 4,∗ Yao Wang , Deyu Meng and Ming Yuan ABSTRACT Recent advances in various fields such as telecommunications, biomedicine and economics, among others, have created enormous amount of data that are often characterized by their huge size and high dimensionality. It has become evident, from research in the past couple of decades, that sparsity is a flexible and powerful notion when dealing with these data, both from empirical and theoretical viewpoints. In this survey, we review some of the most popular techniques to exploit sparsity, for analyzing high-dimensional vectors, matrices and higher-order tensors. Keywords: high-dimensional data, sparsity, compressive sensing, low-rank matrix recovery, tensors nary. Another common example is the recovery of INTRODUCTION School of n ×n 1 2 low-rank matrices where x ∈ R is assumed to Mathematics and The problem of sparse recovery is ubiquitous in have a rank much smaller than min {n , n }. In many 1 2 Statistics, Xi’an modern science and engineering applications. In practical situations, we are also interested in signals Jiaotong University, these applications, we are interested in inferring a of higher-order multilinear structure. For example, Xi’an 710049, China; high-dimensional object, namely a vector, a ma- it is natural to represent multispectral images by Shenyang Institute of trix or a higher-order tensor, from very few obser- a third-order multilinear array, or tensor, with the Automation, Chinese vations. Notable examples include identifying key Academy of Sciences, third index corresponding to different bandwidths. genes driving a complex disease, and reconstruct- Shenyang 10016, Clearly, vectors and matrices can be viewed as first- ing high-quality images or videos from compressive China; Ministry of order and second-order tensors as well. Despite the measurements, among many others. Education Key Lab of connection, moving from vectors and matrices to More specifically, consider linear measurements Intelligent Networks higher-order tensors could present significant new of an n-dimensional object x of the form: and Network Security, challenges. A common way to address these chal- Xi’an Jiaotong lenges is to unfold tensors to matrices; see e.g. [1–4]. University, Xi’an y =a , x, k = 1, ··· , m, (1) k k However, as recently pointed out in [5], the multi- 710049, China and 4 linear structure is lost in such matricization and, as a Department of where ·, · stands for the usual inner product in result, methods based on these techniques often lead Statistics, Columbia R , and a are a set of prespecified n-dimensional to suboptimal results. University, New York, k vectors. The number of measurements m is typically NY 10027, USA A general approach to sparse recovery is through much smaller than n so that the linear system (1) is solving the following constrained optimization underdetermined whenever m < n. Thus it is impos- Corresponding problem: authors. E-mails: sible to recover x from y in the absence of any ad- min S(z)subjectto y =a , z, yao.s.wang@gmail.com; ditional assumption. The idea behind sparse recov- k k dymeng@mail. ery is to assume that x actually resides in a subspace xjtu.edu.cn; k = 1, ··· , m, (2) whose dimensionality is much smaller than the am- ming.yuan@columbia.edu bient dimension n. where S(·) is an objective function that encourages A canonical example of sparse recovery is the so- sparse solutions. The success of this approach hinges Received 25 called compressive sensing for vectors, where x is as- upon several crucial aspects including, among oth- November 2016; sumed to have only a small number of, albeit un- ers, Revised 21 February known, nonzero coordinates. More generally, we call 2017; Accepted 21 how to choose the object function S(·); avector x ∈ R k-sparse if it can be represented March 2017 how to solve the optimization problem (2); by up to k elements from a predetermined dictio- The Author(s) 2017. Published by Oxford University Press on behalf of China Science Publishing & Media Ltd. All rights reserved. For permissions, plea se e-mail: journals.permissions@oup.com Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 757 how to design the sampling vectors a to facilitate relaxation, which minimizes the norm instead, k 1 1 recovery; leading to what is the minimum sample size requirement m min z subject to y = Az. (4) to ensure recovery? z∈R There is, by now, an impressive literature address- Assuming that x is k-sparse and the unique solution ing these issues when x is a vector or matrix; see e.g. to (3), a key question pertaining to the relaxation [6–18], among numerous others. In this review, we (4) is: under what condition is it also the unique aim to survey some of the key developments, with a solution to (4)? The answer can be characterized particular focus on applications to image and video by various properties of the sensing matrix includ- analysis. ing the mutual incoherence property (MIP [20]), The rest of the paper is organized as follows. In null space property (NSP [21]) and restricted isom- the sections entitled ‘Recovery of sparse vectors’, etry property (RIP [9]), among others. We shall fo- ‘Recovery of low-rank matrices’ and ‘Recovery of cus primarily on RIP here. Interested readers are re- low-rank higher-order tensors’, we discuss the recov- ferred to [22,23] and references therein for further ery of sparse vector, low-rank matrix and low-rank discussions on MIP and NSP. tensor signals, respectively. Finally, a couple of illus- m×n Definition 2.1 [9] A sensing matrix A ∈ R is trative examples in image and video processing are said to satisfy the RIP of order k if there exists a con- given in the section entitled ‘Applications’. stant δ ∈ [0, 1) such that, for every k-sparse vector z ∈ R , RECOVERY OF SPARSE VECTORS 2 2 2 (1 − δ )z ≤Az ≤ (1 + δ )z . k k 2 2 2 We first consider recovering a sparse vector signal, (5) a problem more commonly known as compressive Similarly, A is said to obey the restricted orthogonal- sensing [10–13]. ity property (ROP) of order (k, k ) if there exists a constant θ such that, for every k-sparse vector z k,k and k -sparse vector z with non-overlapping support Compressive sensing of sparse signals sets, With slight abuse of notation, let y be an m- dimensional vector whose coordinates are the mea- |Az, Az | ≤ θ z z . (6) k ,k 1 2 2 2 surement y , and A be an m × n matrix whose rows The constants δ and θ are called the k-restricted are given by a . It is then not hard to see that (1) can k k,k isometry constant (RIC) and (k, k )-restricted or- be more compactly written as y = Ax, where x is an thogonality constant (ROC), respectively. n-dimensional vector. Following the jargon in com- The concept was first introduced by Cand es ` and pressive sensing, hereafter we shall refer to A as the Tao [9], who showed that, if δ + θ + θ < 1, sensing matrix. k k, k k,2k then x is the unique solution to (4). This condition Obviously, when m < n, there may be infinitely has since been weakened. For example, Candes ` and many z that agree with the measurements in that Tao [13] showed that δ + 3θ < 2 suffices, and y = Az. Since x is known a priori to be sparse, it is 2k k,2k Cai et al. [24] required that δ + θ < 1. then naturally to seek among all these solutions the 1.25k k,1.25k More recently, Cai and Zhang [25] further weak- one that is sparsest. As mentioned before, an obvious ened the condition to δ + θ < 1 and showed that way to measure sparsity of x is its norm: k k, k the upper bound 1 is sharp in the sense that, for any x =|{i : x = 0}|, > 0, the condition δ + θ < 1 + is not suf- k k, k where |·| stands for the cardinality of a set, leading to ficient to guarantee exact recovery. Sufficient condi- the following recovering x by asolutionto tions for exact recovery by (4) that only involve RIC have also been investigated in the literature. For ex- min z subject to y = Az. (3) ample, [26]arguedthat δ < 2 − 1 implies that 0 2k z∈R x is the unique solution to (4). This was improved to δ < 0.472 in [27]. More recently, [28] showed 2k Under mild regularity conditions on the sensing ma- that, for any given constant t ≥ 4/3, the condition trix A, it can be shown that the solution to (3) is in- δ < (t − 1)/t guarantees the exact recovery of deed well defined and unique, and thus correctly re- tk all k-sparse signals by minimization. Moreover, covers x [9]. However, it is also well known [19]that solving (3) is NP-hard in general and thus infeasible for any > 0, δ < (t − 1)/t + is not suffi- tk to compute for even moderate-size problems. The cient to ensure exact recovery of all k-sparse signals most popular way to overcome this challenge is the for large k. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 758 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW An immediate question following these results is tor z ∈ R , how to design a sensing matrix that satisfies these 2 2 2 (1 − δ )z ≤Az ≤ (1 + δ )z . conditions so that we can use (4) to recover x.It k|I k|I 2 2 2 (9) is now well understood that, for many random en- Similarly, A is said to obey the block-ROP of order sembles, that is, where each entry of A is indepen- (k, k ) if there exists a constant θ such that, for k,k |I dently sampled from a common distribution such every block k-sparse vector z and block k -sparse vec- as a Gaussian, Rademacher or other sub-Gaussian tor z with disjoint supports, distribution, δ < with overwhelming probabil- −2 ity, provided that m ≥ C k log (n/k), for some |Az, Az | ≤ θ z z . (10) k ,k |I 1 2 2 2 constant C > 0. There has also been some re- cent progress in constructing deterministic sensing The constants δ and θ are referred to as block k|I k,k |I matrices that satisfy these RIP conditions; see e.g. k-RIC and block (k, k )-ROC, respectively. [29–31]. Clearly, any sufficient RIP conditions of standard minimization can be naturally extended to the set- ting of block-sparse recovery via mixed / min- 2 1 Compressive sensing of block-sparse imization so that, for example, θ + δ < 1is k,k|I k|I signals also a sufficient condition for x to be the unique so- In many applications, the signal of interest may have lution to (8). more structured sparsity patterns. The most com- mon example is so-called block-sparsity where spar- Nonconvex methods sity occurs in a blockwise fashion rather than at the individual coordinate level. More specifically, let x ∈ In addition to the -minimization-based approach, R be the concatenation of b signal ‘blocks’: there is also an extensive literature on nonconvex methods for sparse recovery where instead of the norm of z one minimizes a nonconvex objective x =[x ··· x x ··· x ··· x ··· x ] , 1 n n +1 n +n n−n +1 n 1 1 1 2 b function in z. The most notable example is the x[1] x[2] x[b] (0 < q < 1) (quasi-)norm, leading to (7) where each signal ‘block’ x[i] is of length n .Weas- min z subject to y = Az. (11) z∈R sume that x is block k-sparse in that there are at most k nonzero blocks among x[i]. As before, we are in- Some recent studies, e.g. [34–36], have shown that terested in the most block-sparse signal that satis- the solution of (11) can recover a sparse signal fies y = Az. To circumvent the potential computa- based on much fewer measurements when com- tional challenge, the following relaxation is often em- pared with the minimization (4). In particular, the ployed: case of q = 1/2 has been treated extensively in [37– 39]. Other notable examples of nonconvex objective min z subject to y = Az, (8) functions include smoothly clipped absolute devia- 2 1 z∈R tion (SCAD) [40] and the minimax concave plus function [41], among others. where the mixed / norm is defined as 2 1 Compressive sensing with general z = z[i ] . 2 1 2 dictionaries i =1 Thus far, we have focused on sparsity with respect It is not hard to see that, when each block has to the canonical basis ofR . In many applications, it size n = 1, (8) reduces to the minimization given might be more appropriate to have sparsity with re- i 1 by (4). More generally, the optimization problem in spect to more general dictionaries; see e.g. [42–46]. (8) is convex and can be recast as a second-order More specifically, a signal x is represented by x = Dα n×n cone program, and thus can be solved efficiently. with respect to a dictionary D ∈ R , where α ∈ One can also extend the notion of RIP and ROP to R is the coordinate in the dictionary and is known the block-sparse setting; see e.g. [ 32,33]. For brevity, a priori to be sparse or nearly sparse. One can ob- write I ={n ,..., n }. viously treat A = AD as the new sensing matrix 1 b Definition 2.2 . A sensing matrix A is said to sat- and apply any of the aforementioned methods to ex- isfy the block-RIP of order k if there exists a constant ploit the sparsity of α. One of the drawbacks, how- δ ∈ [0, 1) such that, for every block k-sparse vec- ever, is that nice properties of the original sensing k|I Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 759 matrix A may not be inherited by A . In other words, fact and distinguish from the vector case treated in even if one carefully designs a sensing matrix A,ex- the last section, we shall write the underlying signal n ×n 1 2 act recovery of x may not be guaranteed despite its as a capital letter, X ∈ R , throughout this sec- sparsity with respect to D. Alternatively, some have tion. In these applications, we often observe a small advocated reconstructing x by the solution to the fol- fraction of the entries of X. The task of matrix com- lowing optimization problem: pletion is then to ‘complete’ the remaining entries. Formally, let be a subset of [n ] × [n ] where 1 2 min D z subject to y = Az; (12) 1 [n] = {1, ..., n}. The goal of matrix completion is z∈R to recover X based on {X :(i, j) ∈ }, particularly ij see e.g. [43,44,47,48]. with the sample size || much smaller than the total number n n of entries. To fix ideas, we shall assume 1 2 that is a uniformly sampled subset of [n ] × [n ], 1 2 Compressive phase retrieval although other sampling schemes have also been in- vestigated in the literature, e.g. [53–55]. In the previous subsections, we have mainly dis- Obviously, we cannot complete an arbitrary ma- cussed the problem of recovering a sparse signal trix from a subset of its entries. But it is possible from a small number of linear measurements. How- for low-rank matrices as their degrees of freedom ever, in some practical scenarios one can only ob- are much smaller than n n . But low rankness alone 1 2 serve some nonlinear measurements of the original is not sufficient. Consider a matrix with a single signal. The typical example is the so-called compres- nonzero entry; it is of rank one but it is impossible sive phase retrieval problem. In such a scenario, we ob- to complete it unless the nonzero entry is observed. serve the magnitude of the Fourier coefficients in- A formal way to characterize low-rank matrices that stead of their phase, which can be modeled as the can be completed from {X :(i, j) ∈ } was first in- ij form troduced in [14]. Definition 3.1 .Let U be a subspace of R of di- y =|a , x|, k = 1, ··· , m. (13) k k mension r and P be the orthogonal projection For recovering x, several studies [49–51]havecon- onto U. Then the coherence of U (with respect to the sidered the following minimization: 1 standard basis (e )) is defined to be μ(U ) ≡ max P e . min z subject to y =|a , z|, U i 1 k k n r 1≤i ≤n z∈R It is clear that the smallest possible value for k = 1, ··· , m. (14) μ(U) is 1 and the largest possible value for μ(U) is n/r.Let M be an n × n matrix of rank r and 1 2 By introducing a strong notion of RIP, Voroninski with column and row spaces denoted by U and V, and Xu [51] built a parallel result for compressive respectively. We shall say that M satisfies the inco- phase retrieval with the classical compressive sens- herence condition with parameter μ if max (μ(U), ing. Specifically, they proved that a k-sparse signal x μ(V)) ≤ μ . can be recovered from m = O(k log (n/k)) random Now let X be an incoherent matrix of rank r.Ina Gaussian phaseless measurements by solving (14). similar spirit as the vector case, a natural way to re- Unlike the standard convex minimization (4), the construct it from {X :(i, j) ∈ } is to seek, among ij problem (14) is a nonconvex problem, but some ef- all matrices whose entries indexed by agree with ficient algorithms have been developed to compute our observations, the one with the smallest rank: it; see e.g. [50,52]. min rank(Z)subjectto Z = X , ij ij n ×n 1 2 Z ∈R RECOVERY OF LOW-RANK MATRICES ∀(i , j ) ∈ . (15) We now consider recovering a low-rank matrix, a problem often referred to as matrix completion. Again, to overcome the computational challenges in directly minimizing matrix ranks, the following con- vex program is commonly suggested: Matrix completion via nuclear norm min Z subject to Z = X , (i , j ) ∈ , minimization ∗ ij ij n ×n 1 2 Z ∈R In many practical situations such as collaborative fil- (16) tering, system identification and remote sensing, to where the nuclear norm · is the sum of all name a few, the signal that we aim to recover often- singular values. As before, we are interested in times is a matrix rather than a vector. To signify this when the solution to (16) is unique and correctly Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 760 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW recovers x. Candes and Recht [14] were the first to Mixed sparsity show that this is indeed the case, for almost all that In some applications, the matrix that we want to re- are large enough. These results are probabilistic in cover is not necessarily of low rank but differs from nature due to the randomness of , that is, one can a low-rank matrix only by a small number of en- correctly recover x using (16) with high probability tries; see e.g. [17,18]. In other words, we may write regarding min {n , n }. The sample size requirement 1 2 X = S + L, where S is a sparse matrix with only a to ensure exact recovery of X by the solution of (16) small number of nonzero entries while L is a ma- was later improved in [16]to ||≥ Cr(n + n ) · 1 2 trix of low rank. It is clear that, even if we observe X polylog(n + n ), where C is a constant that de- 1 2 entirely, the decomposition of X into a sparse com- pends on the coherence coefficients only. It is clear ponent S and a low-rank component L may not be to see that this requirement is (nearly) optimal in uniquely defined. General conditions under which that there are O(r(n + n )) free parameters in spec- 1 2 such a decomposition is indeed unique are provided ifying a rank-r matrix. in, for example, [18]. In the light of the previous dis- cussions, it is natural to consider reconstructing X from observations {X :(i, j) ∈ } by the solution ij Matrix completion from affine to measurements More generally, one may consider recovering a low- min Z + λvec(Z ) rank matrix based on affine measurements. More 1 ∗ 2 n ×n 1 2 Z ,Z ∈R 1 2 n ×n m 1 2 specifically, let A : R → R be a linear map subject to (Z + Z ) = X , ∀(i , j ) ∈ . 1 2 ij ij such that A(X) = y. We aim to recover X based on the information that A(X) = y.Itisclear (19) that the canonical matrix completion problem dis- cussed in the previous subsection corresponds to the case when A(X) ={X :(i , j ) ∈ }. Similarly, ij This strategy has been investigated extensively in we can proceed to reconstruct X by the solution to the literature; see e.g. [17]. Further developments in this direction can also be found in [61–64], among others. min Z subject to y = A(Z). n ×n 1 2 Z ∈R (17) Nonconvex methods It is of interest to know under which kind of sensing operator A can X be recovered in exactly this way. Just as norm is a convex relaxation of the norm, 1 0 An answer is given in [56], which extends the con- the nuclear norm is also a convex relaxation of the cept of RIP to general linear operator for matrices. rank for a matrix. In addition to these convex ap- n ×n 1 2 Definition 3.2 . A linear operator A : R → proaches, nonconvex methods have also been pro- R is said to satisfy the matrix RIP of order r if there posed by numerous authors. The most common ex- exists a constant δ such that ample is the Schatten- q (0 < q < 1) (quasi-)norm defined by Z Z 1 − δ Z ≤A(Z) ≤ 1 + δ Z F F r 2 r 1/q (18) min{n ,n } 1 2 n ×n 1 2 holds for all matrices Z ∈ R of rank at most r. X = σ , (20) q i Recht et al. [56] further proved that, if A i =1 satisfies the matrix RIP (18) with δ < 1/10, 5r then one can recover a rank-r matrix X from where σ , σ , ··· are the singular values of X.Itis 1 2 n = O(r(n + n ) log (n n )) measurements by the 1 2 1 2 clear that, when q → 0, X → rank(X)while the sq solution to (17). Note that the condition δ < 5r nuclear norm corresponds to the case when q = 1. 1/10 has been dramatically weakened; see e.g. One may now consider reconstructing X by the so- [28,57]. lution to Besides the aforementioned low-rank matrix completion problems, some recent studies, e.g. [58– 60], have drawn attention to the so-called high-rank min Z subject to y = A(Z); matrix completion, in which the columns of the ma- n ×n 1 2 Z ∈R trix belong to a union of subspaces and, as a result, (21) the rank can be high or even full. see e.g. [65–68], among others. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 761 matrix nuclear norm, computing the tensor nuclear RECOVERY OF LOW-RANK norm, and thereby the problem (23), is NP-hard. HIGHER-ORDER TENSORS Hence, various relaxations and approximate algo- In an increasing number of modern applications, rithms have been introduced in the literature; see e.g. the object to be estimated has a higher-order tensor [75–79], and references therein. This is a research structure. Typical examples include video inpaint- area in its infancy and many interesting issues need ing [69], scan completion [70], multichannel EEG to be addressed. (electroencephalogram) compression [71], traffic data analysis [72] and hyperspectral image restora- tion [73], among many others. Similar to matrices, APPLICATIONS in many of these applications, we are interested in In the previous sections, we have given an overview recovering a low-rank tensor either from observing a subset of its entries or a collection of affine mea- of some basic ideas and techniques in dealing with surements. sparsity in vectors and low rankness in matrices and Despite the apparent similarities between matri- tensors. We now give a couple of examples to illus- ces and higher-order tensors, it is delicate to extend trate how they can be used in action. the idea behind nuclear norm minimization to the latter because matrix-style singular value decompo- sition does not exist for higher-order tensors. A com- Background subtraction with mon approach is to first unfold a higher-order tensor compressive imaging to a matrix and then apply a matrix-based approach Background subtraction in image and video has to recover a tensor. Consider, for example, a third- attracted a lot of attention in the past couple of d ×d ×d 1 2 3 order tensorX ∈ R . We can collapse its sec- decades. It aims at simultaneously separating video ond and third indices, leading to a d × (d d )ma- 1 2 3 background and extracting the moving objects from trix X .If X is of low rank, then so is X .Wecan (1) (1) a video stream, and can provide important clues for exploit the low rankness of X by minimizing its nu- (1) various applications such as moving object detection clear norm. Clearly we can also collapse the first and [80] and object tracking in surveillance [81], among third indices of X , leading to a matrix X , and the (2) numerous others. first and second, leading to X . If we want to com- (3) Conventional background subtraction tech- plete a low-rank tensor X based on its entries X (ω) niques usually consist of four steps: video acqui- for ω ∈ ⊂[d ] × [d ] × [d ], we can then con- 1 2 3 sition, encoding, decoding and separating the sider recovering X by the solution to the following moving objects from the background. This scheme convex program: needs to fully sample the video frames with large computational and storage requirements, followed by well-designed video coding and background min Z ( j ) ∗ d ×d ×d 1 2 3 Z∈R subtraction algorithms. To alleviate the burden of j =1 computation and storage, a newly-developed com- subject to Z(ω) = X (ω), ∀ω ∈ . (22) pressive imaging scheme [82–84] has been used for background subtraction by combining the video Efficient algorithms for solving matrix nuclear norm acquisition, coding and background subtraction minimization (16) such as the alternating direc- into a single framework, as illustrated in Fig. 1.It tion method of multipliers (ADMM) and Douglas– simultaneously achieves background subtraction Rachford operator splitting methods can then be and video reconstruction. In this setting, the main readily adapted to solve (22); see e.g. [3,4]. objective is then to maximize the reconstruction As pointed out in [5], however, such an approach and separation accuracies using as few compressive fails to exploit fully the multilinear structure of a ten- measurements as possible. sor and is thus suboptimal. Instead, directly minimiz- Several studies have been carried out for back- ing the tensor nuclear norm was suggested: ground subtraction from the perspective of com- pressive imaging. In the seminal work of Cevher et al. min Z d ×d ×d 1 2 3 Z∈R [85], the background subtraction problem is formu- lated as a sparse recovery problem. They showed that subject to Z(ω) = X (ω), ∀ω ∈ , (23) the moving objects can be recovered by learning a where the tensor nuclear norm · is defined as low-dimensional compressed representation of the the dual norm of the tensor spectral norm ·.For background image. More recently, the robust princi- more discussions on the tensor nuclear and spec- ple component analysis (RPCA) approach has also tral norms, please see [74]. Unfortunately, unlike the been used to deal with the problem of background Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 762 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW Compressive Reconstruction with foreground- measurements background separation form Transmission Figure 1. The framework of background subtraction with compressive imaging. subtraction with compressive imaging, in which they 3D-TV term · is defined as 3D-TV commonly model the video as a matrix with columns X : =X (i , j, k) of vectorized video frames and then decompose the 3D-TV h 1 matrix into a low-rank matrix L and a sparse matrix +X (i , j, k) +X (i , j, k) , v 1 t 1 S; see e.g. [86–89]. Although methods based on RPCA have where achieved satisfactory performance, they fail to exploit the finer structures of background and fore- X (i , j, k): = X (i , j + 1, k) − X (i , j, k), ground after vectorizing the video frames. It appears X (i , j, k): = X (i + 1, j, k) − X (i , j, k), more advantageous to model the spatio-temporal information of background and foreground using X (i , j, k): = X (i , j, k + 1) − X (i , j, k). direct tensor representation of a video. To this end, a novel tensor RPCA approach has been Because a 3D patch in a video background is simi- proposed in [90] for background subtraction lar to many other 3D patches over the video frames, from compressive measurements by decomposing one can model the video background using several the video into a static background with spatio- groups of similar video 3D patches, where each patch temporal correlation and a moving foreground group corresponds to a fourth-order tensor. Inte- with spatio-temporal continuity within a tensor grating the patch-based modeling idea into (24), representation framework. More specifically, one one can easily get a patch-group-based tensor RPCA can use 3D total variation (3D-TV) to characterize model. It should be noted that solving the noncon- the spatio-temporal continuity underlying the video vex tensor RPCA model (24) as well as its patch- foreground, and low-rank Tucker decomposition to group-based form are computationally difficult. In model the spatio-temporal correlation of the video practice, we can find a local solution using a multi- background, which leads to the following tensor block version of ADMM. For more details, please re- RPCA model: fer to Section V of [90]. Fig. 2 gives an example based on three real videos. min λS + E It is evident that the proposed tensor models enjoy 3D-TV X , S, E , a superior performance over the other popular ma- G, U , U , U 1 2 3 trix models both in terms of the quality of the recon- structed videos and in terms of the separation of the subject to X = L + E + S, moving objects. This suggests that using a direct ten- L = G × U × U × U , sor modeling technique to deal with practical higher- 1 1 2 2 3 3 order tensor data can utilize more useful structures. y = A(X ), (24) where the factor matrices U and U are orthogonal 1 2 Hyperspectral compressive sensing in columns for two spatial modes, the factor matrix U is orthogonal in columns for the temporal mode, Hyperspectral imaging employs an imaging spec- the core tensor G interacts with these factors and the trometer to collect hundreds of spectral bands Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 763 Ground Truth SparCS ReProCS SpLR H-TenRPCA PG-TenRPCA (a) (b) (c) Figure 2. Visual comparison of two tensor models (i.e. H-TenRPCA and PG-TenRPCA) proposed in [90] and three popular matrix models (i.e. SparCS [86], ReProcs [87] and SpLR [88]) under the sampling ratio 1/30. The first column shows the original video frames from different video volumes (a)–(c); the second to sixth columns correspond to the results produced by all the compared methods, respectively. Here, for each method, the reconstruction result of the original video frame (upper panels) and the detection result of moving objects in the foreground (lower panels) is shown. ranging from ultraviolet to infrared wavelengths for tensively investigated. Similar to other compressive the same area on the surface of the Earth. It has a sensing problems, the main objectives of HCS are wide range of applications including environmen- to design easy hardware encoding implementation tal monitoring, military surveillance and mineral ex- and develop an efficient sparse reconstruction pro- ploration, among numerous others [91,92]. Figura- cedure. In what follows we shall focus primarily tively speaking, a hyperspectral image can be treated on the latter. For hardware implementation, inter- as a 3D (x, y, λ) data cube, where x and y represent ested readers are referred to [93,94] and references two spatial dimensions of the scene, and λ represents therein. the spectral dimension comprising a range of wave- Like other natural images, hyperspectral images lengths. Typically, such hyperspectral cubes are col- can be sparsified by using certain transformations. lected by an airborne sensor or a satellite and sent Traditional sparse recovery methods such as min- to a ground station on Earth for subsequent process- imization and TV minimization are often used for ing. Noting that the dimension of λ is usually in the such purposes; see e.g. [95,96]. To further exploit hundreds, hyperspectral cubes are of a fairly large the inherent spectral correlation of hyperspectral im- size even for moderate dimensions of x and y.This ages, a series of work based on low-rank modeling makes it necessary to devise effective techniques for has been carried out in recent years. For example, hyperspectral data compression, due to the limited Golbabaee and Vandergheynst proposed in [97]a bandwidth of the link connection between the satel- joint nuclear and / minimization method to de- 2 1 lite/aerospace and the ground station. scribe the spectral correlation and the joint-sparse In the last few years, a popular hyperspectral com- spatial wavelet representations of hyperspectral im- pressive sensing (HCS) scheme based on the prin- ages. Then, they modeled the spectral correlation ciple of compressive sensing for hyperspectral data together with the spatial piecewise smoothness of compression, as illustrated in Fig. 3, has been ex- hyperspectral images by using a joint nuclear and Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 764 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW Satellites, aerospace Transmit measurements Encoding by compressive sensing Receiving Reconstruction via sparse modeling Ground station Figure 3. The framework of hyperspectral compressive sensing. TV norm minimization method [98]. Similar to the surveillance videos, however, modeling a hyperspec- tral cube as a matrix cannot utilize the finer spatial- and-spectral information, leading to suboptimal re- construction results under relatively low sampling ratios (e.g. 1% of the whole image size). To further exploit the compressibility underlying a hyperspectral cube, one may consider such a sin- gle cube as a tensor with three modes (width, height and band) and then identify the hidden spatial- and-spectral structures using direct tensor modeling techniques. Precisely, all the bands of a hyperspec- tral image have very strong correlation in the spec- tral domain, and each band, if considered as a ma- trix, has relatively strong correlation in the spatial domain; such spatial-and-spectral correlation can be modeled through low-rank Tucker decomposition. In addition, the intensity at each voxel is likely to be similar to its neighbors, which can be characterized by smoothness using the so-called 3D-TV penalty. In summary, [99] considered the following joint ten- sor Tucker decomposition and 3D-TV minimiza- tion model: min λX + E 3D-TV X , E , G, U , U , U 1 2 3 subject to X = G × U × U × U + E , 1 1 2 2 3 3 y = A(X ). (25) It is clear that the above minimization problem is highly nonconvex. One often looks for good local so- lutions using a multi-block ADMM algorithm. Figure 4. Vision comparison of the tensor method (25) over three other competing Fig. 4 gives an example of the first band of four hy- methods on the rst fi band of four different hyperspectral datasets (a)–(d). Here the perspectral datasets, respectively reconstructed by sampling ratio is 1%. The last column shows the original image bands. The columns different methods, with the sampling ratio at 1%. It from the first to the fourth correspond to the produced results from the KCS method [96], the JNTV method [98], the ST-NCS method [100] and the tensor method (25), is evident that the proposed tensor method could respectively. provide nearly perfect reconstruction. In addition, Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 765 sparse tensor and nonlinear compressive sensing 13. Candes ` E, Romberg J and Tao T. Stable signal recovery from (ST-NCS) performs slightly better than Kronecker incomplete and inaccurate measurements. Comm Pure Appl compressive sensing (KCS) and joint nuclear/TV Math 2006; 59: 1207–23. norm minimization (JNTV) in terms of reconstruc- 14. Candes ` E and Recht B. Exact matrix completion via convex op- tion accuracy, because of the use of a direct ten- timization. Found Comput Math 2009; 9: 717–72. sor sparse representation of a hyperspectral cube. 15. Candes ` E and Tao T. The power of convex relaxation: near- Both these findings demonstrate the power of ten- optimal matrix completion. IEEE Trans Inform Theor 2010; 56: sor modeling techniques. It is also worth noting that, 2053–80. compared with ST-NCS, the images reconstructed 16. Gross D. Recovering low-rank matrices from few coefficients with the method (25) are clearer and sharper. in any basis. IEEE Trans Inform Theor 2011; 57: 1548–66. 17. Candes ` E, Li X and Ma Y et al. Robust principal component analysis? JACM 2011; 58: 1–39. ACKNOWLEDGEMENT 18. Chandrasekaran V, Sanghavi S and Parrilo P et al. Rank- The authors thank the associate editor and three referees for help- sparsity incoherence for matrix decomposition. SIAM J Optim ful comments. 2011; 21: 572–96. 19. Natarajan B. Sparse approximate solutions to linear systems. SIAM J Comput 1995; 24: 227–34. FUNDING 20. Donoho D, Elad M and Temlyakov VN. Stable recovery of This work was supported in part by the National Natural Sci- sparse overcomplete representations in the presence of noise. ence Foundation of China (11501440 and 61273020 to IEEE Trans Inform Theor 2006; 52: 6–18. Y.W., 61373114, 61661166011 and 61721002 to D.Y.M.), 21. Cohen A, Dahmen W and DeVore R. Compressed sensing the National Basic Research Program of China (973 Program) and best k-term approximation. J Am Math Soc 2009; 22: (2013CB329404 to D.Y.M.) and the National Science Founda- 211–31. tion (DMS-1265202 to M.Y.). 22. Eldar Y and Kutyniok G. Compressed Sensing: Theory and Ap- plications. Cambridge: Cambridge University Press, 2012. REFERENCES 23. Foucart S and Rauhut H. A Mathematical Introduction to Com- 1. Liu J, Musialski P and Wonka P et al. Tensor completion for pressive Sensing. Berlin: Springer, 2013. estimating missing values in visual data. In: Proceedings of In- 24. Cai TT, Wang L and Xu G. New bounds for restricted isometry ternational Conference on Computer Vision, 2009. constants. IEEE Trans Inform Theor 2010; 56: 4388–94. 2. Tomioka R, Hayashi K and Kashima H. Estimation of low-rank 25. Cai TT and Zhang A. Compressed sensing and affine rank min- tensors via convex optimization. arXiv:10100789. imization under restricted isometry. IEEE Trans Signal Process 3. Gandy S, Recht B and Yamada I. Tensor completion and low- 2013; 61: 3279–90. n-rank tensor recovery via convex optimization. Inverse Probl 26. Candes ` E. The restricted isometry property and its implications 2011; 27: 025010. for compressed sensing. Compt Rendus Math 2008; 346: 589– 4. Liu J, Musialski P and Wonka P. Tensor completion for esti- mating missing values in visual data. IEEE Trans Pattern Anal 27. Cai TT, Wang L and Xu G. Shifting inequality and recovery of Mach Intell 2013; 34: 208–20. sparse signals. IEEE Trans Signal Process 2010; 58: 1300–8. 5. Yuan M and Zhang CH. On tensor completion via nuclear norm 28. Cai TT and Zhang A. Sparse representation of a polytope and minimization. Found Comput Math 2016; 16: 1031–68. recovery of sparse signals and low-rank matrices. IEEE Trans 6. Chen SS, Donoho DL and Saunders MA. Atomic decomposition Inform Theor 2014; 60: 122–32. by basis pursuit. SIAM J Sci Comput 1998; 20: 33–61. 29. DeVore R. Deterministic constructions of compressed sensing 7. Donoho DL and Huo X. Uncertainty principles and ideal atomic matrices. J Complex 2007; 23: 918–25. decomposition. IEEE Trans Inform Theor 2001; 47: 2845–62. 30. Bourgain J, Dilworth SJ and Ford K et al. Explicit constructions 8. Donoho DL and Elad M. Optimally sparse representation in of RIP matrices and related problems. Duke Math J 2011; 159: general (nonorthogonal) dictionaries via minimization. Proc 145–85. Natl Acad Sci 2003; 100: 2197–202. 31. Xu Z. Deterministic sampling of sparse trigonometric polyno- 9. Candes ` E and Tao T. Decoding by linear programming. IEEE mials. J Complex 2011; 27: 133–40. Trans Inform Theor 2005; 51: 4203–15. 32. Eldar Y and Mishali M. Robust recovery of signals from a struc- 10. Candes ` E, Romberg J and Tao T. Robust uncertainty principles: tured union of subspaces. IEEE Trans Inform Theor 2009; 55: exact vector reconstruction from highly incomplete frequency 5302–16. information. IEEE Trans Inform Theor 2006; 52: 489–509. 33. Wang Y, Wang J and Xu Z. On recovery of block-sparse signals 11. Donoho D. Compressed sensing. IEEE Trans Inform Theor 2006; via mixed / (0 < q ≤ 1) norm minimization. EURASIP J Adv 2 q 52: 1289–306. Signal Process 2013; 76: 1–17. 12. Candes ` E and Tao T. Near-optimal signal recovery from random 34. Chartrand R. Exact reconstruction of sparse signals via non- projections: universal encoding strategies. IEEE Trans Inform convex minimization. IEEE Signal Process Lett 2007; 14: Theor 2006; 52: 5406–25. 707–10. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 766 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW 35. Sun Q. Recovery of sparsest signals via -minimization.ApplComputHarmon 59. Elhamifar E. High-rank matrix completion and clustering under self-expressive Anal 2012; 32: 329–41. models. In: Proceedings of Advances in Neural Information Processing Sys- 36. Song CB and Xia ST. Sparse signal recovery by minimization under re- tems, 2016. stricted isometry property. IEEE Signal Process Lett 2014; 21: 1154–8. 60. Li CG and Vidal R. A structured sparse plus structured low-rank framework 37. Xu Z, Zhang H and Wang Y et al. regularization. Sci China Inform Sci for subspace clustering and completion. IEEE Trans Signal Process 2016; 64: 1/2 2010; 53: 1159–69. 6557–70. 38. Xu Z, Chang X and Xu F et al. regularization: a thresholding representa- 61. Zhou ZH, Li X and Wright J et al. Stable principal component pursuit. In: Pro- 1/2 tion theory and a fast solver. IEEE Trans Neural Network Learn Syst 2012; 23: ceedings of the 2010 IEEE International Symposium on Information Theory, 1013–27. 2010. 39. Zeng J, Lin S and Wang Y et al. regularization: convergence of it- 62. Ganesh A, Wright J and Li Xetal. Dense error correction for low-rank matrices 1/2 erative half thresholding algorithm. IEEE Trans Signal Process 2014; 62: via principal component pursuit. In: Proceedings of the 2010 IEEE International 2317–29. Symposium on Information Theory, 2010. 40. Fan J and Li R. Variable selection via nonconcave penalized likelihood and its 63. Zhao Q, Meng D and Xu Z et al. Robust principal component analysis with oracle properties. J Am Stat Assoc 2001; 96: 1348–60. complex noise. In: Proceedings of the 31st International Conference on Ma- 41. Zhang CH. Nearly unbiased variable selection under minimax concave penalty. chine Learning, 2014. Ann Stat 2010; 38: 894–942. 64. Netrapalli P, Niranjan U and Sanghavi S et al. Non-convex robust PCA. 42. Rauhut H, Schnass K and Vandergheynst P. Compressed sensing and redun- In: Proceedings of Advances in Neural Information Processing Systems 27, dant dictionaries. IEEE Trans Inform Theor 2013; 29: 1401–12. 2014. 43. Candes ` E, Eldar Y and Needell D et al. Compressed sensing with coherent and 65. Zhang M, Huang ZH and Zhang Y. Restricted p-isometry properties of noncon- redundant dictionaries. Appl Comput Harmon Anal 2010; 31: 59–73. vex matrix recovery. IEEE Trans Inform Theor 2013; 59: 4316–23. 44. Elad M, Milanfar P and Rubinstein R. Analysis versus synthesis in signal pri- 66. Wang J, Wang M and Hu X et al. Visual data denoising with a unified ors. Appl Comput Harmon Anal 2007; 23: 947–68. Schatten-p norm and norm regularized principal component pursuit. Pat- 45. Lin J, Li S and Shen Y. New bounds for restricted isometry constants with tern Recogn 2015; 48: 3135–44. coherent tight frames. IEEE Trans Signal Process 2013; 61: 611–21. 67. Zhao Q, Meng D and Xu Z et al. -norm low-rank matrix factorization by 46. Lin J and Li S. Sparse recovery with coherent tight frames via analydsis variational Bayesian method. IEEE Trans Neural Network Learn Syst 2015; 26: Dantzig selector and analysis LASSO. Appl Comput Harmon Anal 2014; 37: 825–39. 126–39. 68. Yue MC and So AMC. A perturbation inequality for concave functions of sin- 47. Liu Y, Mi T and Li S. Compressed sensing with general frames via optimal- gular values and its applications in low-rank matrix recovery. Appl Comput dual-based -analysis. IEEE Trans Inform Theor 2012; 58: 4201–14. Harmon Anal 2016; 40: 396–416. 48. Li S and Lin J. Compressed sensing with coherent tight frames via - 69. Korah T and Rasmussen C. Spatio-temporal inpainting for recovering texture minimization for 0 < q ≤ 1. Inverse Probl Imag 2014; 8: 761–77. maps of occluded building facades. IEEE Trans Image Process 2007; 16: 2262– 49. Moravec M, Romberg J and Baraniuk R. Compressive phase retrieval. In: Pro- 71. ceedings of SPIE, the International Society for Optics and Photonics, 2007. 70. Pauly M, Mitra N and Giesen J et al. Example-based 3D scan completion. In: 50. Yang Z, Zhang C and Xie L. Robust compressive phase retrieval via mini- Proceedings of the Symposium on Geometry Processing, 2005. mization with application to image reconstruction. arXiv:13020081. 71. Acar E, Dunlavy D and Kolda T et al. Scalable tensor factorizations for incom- 51. Voroninski V and Xu Z. A strong restricted isometry property, with an applica- plete data. Chemometr Intell Lab Syst 2001; 106: 41–56. tion to phaseless compressed sensing. Appl Comput Harmon Anal 2016; 40: 72. Xie K, Wang L and X Wetal. Accurate recovery of internet traffic data: a tensor 386–95. completion approach. In: Proceedings of the 35th Annual IEEE International 52. Schniter P and Rangan S. Compressive phase retrieval via general- Conference on Computer Communications, 2016. ized approximate message passing. IEEE Trans Signal Process 2015; 63: 73. Peng Y, Meng D and Xu Z et al. Decomposable nonlocal tensor dictionary 1043–55. learning for multispectral image denoising. In:ProceedingsofIEEEConference 53. Foygel R, Shamir O and Srebro N et al. Learning with the weighted trace-norm on Computer Vision and Pattern Recognition, 2014. under arbitrary sampling distributions. In: Proceedings of Advances in Neural 74. Hillar CJ and Lim LH. Most tensor problems are NP-hard. JACM 2013; 60: Information Processing Systems 24, 2011. 1–39. 54. Chen Y, Bhojanapalli S and Sanghavi S et al. Coherent matrix completion. In: 75. Nie J and Wang L. Semidefinite relaxations for best rank-1 tensor approxima- Proceedings of the 31st International Conference on Machine Learning, 2014. tions. SIAM J Matrix Anal Appl 2014; 35: 1155–79. 55. Cai TT and Zhou WX. Matrix completion via max-norm constrained optimiza- 76. Jiang B, Ma S and Zhang S. Tensor principal component analysis via convex tion. Electron J Stat 2016; 10: 1493–525. optimization. Math Program 2015; 150: 423–57. 56. Recht B, Fazel M and Parrilo P. Guaranteed minimum-rank solutions of linear 77. Yang Y, Feng Y and Suykens J. A rank-one tensor updating algorithm for tensor matrix equations via nuclear norm minimization.SIAMRev 2010; 52: 471–501. completion. IEEE Signal Process Lett 2015; 22: 1633–7. 57. Candes ` E and Plan Y. Tight oracle inequalities for low-rank matrix recovery 78. Zhao Q, Meng D and Kong X et al. A novel sparsity measure for tensor from a minimal number of noisy random measurements. IEEE Trans Inform recovery. In: Proceedings of International Conference on Computer Vision, Theor 2011; 57: 2342–59. 2015. 58. Eriksson B, Balzano L and Nowak R. High-rank matrix completion. In: Proceed- 79. Xie Q, Zhao Q and Meng Detal. Kronecker-Basis-Representation based tensor ings of the 15th International Conference on Artificial Intelligence and Statis- sparsity and its Applications to tensor recovery. IEEE Trans Pattern Anal Mach tics, 2012. Intell 2017; 40: 1888–902. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 767 80. Wang T, Backhouse A and Gu I. Online subspace learning on Grassmann man- 90. Cao W, Wang Y and Sun J et al. Total variation regularized tensor RPCA for ifold for moving object tracking in video. In: Proceedings of IEEE International background subtraction from compressive measurements. IEEE Trans Image Conference on Acoustics, Speech, and Signal Process, 2008. Process 2016; 25: 4075–90. 81. Beleznai C, Fruhstuck B and Bischof H. Multiple object tracking using local 91. Goetz AFH. Three decades of hyperspectral remote sensing of the Earth: a PCA. In: Proceedings of the 18th International Conference on Pattern Recog- personal view. Rem Sens Environ 2009; 113: S5–S6. nition, 2006. 92. Willett R, Duarte M and Davenport M et al. Sparsity and structure in hyper- 82. Wakin M, Laska JN and Duarte MF et al. Compressive imaging for video spectral imaging: sensing, reconstruction, and target detection. IEEE Signal representation and coding. In: Proceedings of Picture Coding Symposium, Process Mag 2014; 31: 116–26. 2006. 93. Arce G, Brady D and Carin L et al. Compressive coded aperture spectral imag- 83. Takhar D, Laska JN and Wakin M et al. A new compressive imaging camera ing: an introduction. IEEE Signal Process Mag 2014; 31: 105–15. architecture using optical-domain compression. In: Proceedings of Computa- 94. Yuan X, Tsai TH and Zhu R et al. Compressive hyperspectral imaging with side tional Imaging IV at SPIE Electronic Imaging, 2006. information. IEEE J Sel Top Signal Process 2015; 9: 964–76. 84. Duarte M, Davenport M and Takhar D et al. Single-pixel imaging via compres- 95. Li C, Sun T and Kelly KF et al. A compressive sensing and unmixing scheme for sive sampling. IEEE Signal Process Mag 2008; 25: 83–91. hyperspectral data processing. IEEE Trans Image Process 2012; 21: 1200–10. 85. Cevher V, Sankaranarayanan A and Duarte M et al. Compressive sensing for 96. Duarte M and Baraniuk R. Kronecker compressive sensing. IEEE Trans Image background subtraction. In: Proceedings of the 10th European Conference on Process 2012; 21: 494–504. Computer Vision, 2008. 97. Golbabaee M and Vandergheynst P. Hyperspectral image compressed sensing 86. Waters A, Sankaranarayanan A and Baraniuk R et al. A new compres- via low-rank and joint-sparse matrix recovery. In: Proceedings of IEEE Interna- sive imaging camera architecture using optical-domain compression. In: tional Conference on Acoustics, Speech and Signal Processing, 2012. Proceedings of Conference on Neural Information Processing Systems 24, 98. Golbabaee M and Vandergheynst P. Joint trace/TV norm minimization: a new 2011. efficient approach for spectral compressive imaging. In: Proceedings of 19th 87. Guo H, Qiu CL and Vaswani N. An online algorithm for separating sparse and IEEE International Conference on Image Processing, 2012. low-dimensional signal sequences from their sum. IEEE Trans Signal Process 99. Wang Y, Lin L and Zhao Q et al. Compressive sensing of hyperspectral images 2014; 62: 4284–97. via joint tensor tucker decomposition and weighted total variation regulariza- 88. Jiang H, Deng W and Shen Z. Surveillance video processing using compres- tion. IEEE Geosci Rem Sens Lett 2017; 14: 2457–61. sive sensing. Inverse Probl Imag 2014; 6: 201–14. 100. Yang S, Wang M and Li P et al. Compressive hyperspectral imaging via sparse 89. Jiang H, Zhao S and Shen Z et al. Surveillance video analysis using compres- tensor and nonlinear compressed sensing. IEEE Trans Geosci Rem Sens 2015; sive sensing with low latency. Bell Labs Tech J 2014; 18: 63–74. 53: 5943–57.
National Science Review – Oxford University Press
Published: Sep 1, 2018
You can share this free article with as many people as you like with the url below! We hope you enjoy this feature!
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.