Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Sparse recovery: from vectors to tensors

Sparse recovery: from vectors to tensors Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 National Science Review 5: 756–767, 2018 REVIEW doi: 10.1093/nsr/nwx069 Advance access publication 14 July 2017 INFORMATION SCIENCE 1,2,∗ 1,3,∗ 4,∗ Yao Wang , Deyu Meng and Ming Yuan ABSTRACT Recent advances in various fields such as telecommunications, biomedicine and economics, among others, have created enormous amount of data that are often characterized by their huge size and high dimensionality. It has become evident, from research in the past couple of decades, that sparsity is a flexible and powerful notion when dealing with these data, both from empirical and theoretical viewpoints. In this survey, we review some of the most popular techniques to exploit sparsity, for analyzing high-dimensional vectors, matrices and higher-order tensors. Keywords: high-dimensional data, sparsity, compressive sensing, low-rank matrix recovery, tensors nary. Another common example is the recovery of INTRODUCTION School of n ×n 1 2 low-rank matrices where x ∈ R is assumed to Mathematics and The problem of sparse recovery is ubiquitous in have a rank much smaller than min {n , n }. In many 1 2 Statistics, Xi’an modern science and engineering applications. In practical situations, we are also interested in signals Jiaotong University, these applications, we are interested in inferring a of higher-order multilinear structure. For example, Xi’an 710049, China; high-dimensional object, namely a vector, a ma- it is natural to represent multispectral images by Shenyang Institute of trix or a higher-order tensor, from very few obser- a third-order multilinear array, or tensor, with the Automation, Chinese vations. Notable examples include identifying key Academy of Sciences, third index corresponding to different bandwidths. genes driving a complex disease, and reconstruct- Shenyang 10016, Clearly, vectors and matrices can be viewed as first- ing high-quality images or videos from compressive China; Ministry of order and second-order tensors as well. Despite the measurements, among many others. Education Key Lab of connection, moving from vectors and matrices to More specifically, consider linear measurements Intelligent Networks higher-order tensors could present significant new of an n-dimensional object x of the form: and Network Security, challenges. A common way to address these chal- Xi’an Jiaotong lenges is to unfold tensors to matrices; see e.g. [1–4]. University, Xi’an y =a , x, k = 1, ··· , m, (1) k k However, as recently pointed out in [5], the multi- 710049, China and 4 linear structure is lost in such matricization and, as a Department of where ·, · stands for the usual inner product in result, methods based on these techniques often lead Statistics, Columbia R , and a are a set of prespecified n-dimensional to suboptimal results. University, New York, k vectors. The number of measurements m is typically NY 10027, USA A general approach to sparse recovery is through much smaller than n so that the linear system (1) is solving the following constrained optimization underdetermined whenever m < n. Thus it is impos- Corresponding problem: authors. E-mails: sible to recover x from y in the absence of any ad- min S(z)subjectto y =a , z, yao.s.wang@gmail.com; ditional assumption. The idea behind sparse recov- k k dymeng@mail. ery is to assume that x actually resides in a subspace xjtu.edu.cn; k = 1, ··· , m, (2) whose dimensionality is much smaller than the am- ming.yuan@columbia.edu bient dimension n. where S(·) is an objective function that encourages A canonical example of sparse recovery is the so- sparse solutions. The success of this approach hinges Received 25 called compressive sensing for vectors, where x is as- upon several crucial aspects including, among oth- November 2016; sumed to have only a small number of, albeit un- ers, Revised 21 February known, nonzero coordinates. More generally, we call 2017; Accepted 21 how to choose the object function S(·); avector x ∈ R k-sparse if it can be represented March 2017 how to solve the optimization problem (2); by up to k elements from a predetermined dictio- The Author(s) 2017. Published by Oxford University Press on behalf of China Science Publishing & Media Ltd. All rights reserved. For permissions, plea se e-mail: journals.permissions@oup.com Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 757 how to design the sampling vectors a to facilitate  relaxation, which minimizes the  norm instead, k 1 1 recovery; leading to what is the minimum sample size requirement m min z subject to y = Az. (4) to ensure recovery? z∈R There is, by now, an impressive literature address- Assuming that x is k-sparse and the unique solution ing these issues when x is a vector or matrix; see e.g. to (3), a key question pertaining to the  relaxation [6–18], among numerous others. In this review, we (4) is: under what condition is it also the unique aim to survey some of the key developments, with a solution to (4)? The answer can be characterized particular focus on applications to image and video by various properties of the sensing matrix includ- analysis. ing the mutual incoherence property (MIP [20]), The rest of the paper is organized as follows. In null space property (NSP [21]) and restricted isom- the sections entitled ‘Recovery of sparse vectors’, etry property (RIP [9]), among others. We shall fo- ‘Recovery of low-rank matrices’ and ‘Recovery of cus primarily on RIP here. Interested readers are re- low-rank higher-order tensors’, we discuss the recov- ferred to [22,23] and references therein for further ery of sparse vector, low-rank matrix and low-rank discussions on MIP and NSP. tensor signals, respectively. Finally, a couple of illus- m×n Definition 2.1 [9] A sensing matrix A ∈ R is trative examples in image and video processing are said to satisfy the RIP of order k if there exists a con- given in the section entitled ‘Applications’. stant δ ∈ [0, 1) such that, for every k-sparse vector z ∈ R , RECOVERY OF SPARSE VECTORS 2 2 2 (1 − δ )z ≤Az ≤ (1 + δ )z . k k 2 2 2 We first consider recovering a sparse vector signal, (5) a problem more commonly known as compressive Similarly, A is said to obey the restricted orthogonal- sensing [10–13]. ity property (ROP) of order (k, k ) if there exists a constant θ such that, for every k-sparse vector z k,k and k -sparse vector z with non-overlapping support Compressive sensing of sparse signals sets, With slight abuse of notation, let y be an m- dimensional vector whose coordinates are the mea- |Az, Az | ≤ θ z z  . (6) k ,k 1 2 2 2 surement y , and A be an m × n matrix whose rows The constants δ and θ are called the k-restricted are given by a . It is then not hard to see that (1) can k k,k isometry constant (RIC) and (k, k )-restricted or- be more compactly written as y = Ax, where x is an thogonality constant (ROC), respectively. n-dimensional vector. Following the jargon in com- The concept was first introduced by Cand es ` and pressive sensing, hereafter we shall refer to A as the Tao [9], who showed that, if δ + θ + θ < 1, sensing matrix. k k, k k,2k then x is the unique solution to (4). This condition Obviously, when m < n, there may be infinitely has since been weakened. For example, Candes ` and many z that agree with the measurements in that Tao [13] showed that δ + 3θ < 2 suffices, and y = Az. Since x is known a priori to be sparse, it is 2k k,2k Cai et al. [24] required that δ + θ < 1. then naturally to seek among all these solutions the 1.25k k,1.25k More recently, Cai and Zhang [25] further weak- one that is sparsest. As mentioned before, an obvious ened the condition to δ + θ < 1 and showed that way to measure sparsity of x is its  norm: k k, k the upper bound 1 is sharp in the sense that, for any x =|{i : x = 0}|, > 0, the condition δ + θ < 1 +  is not suf- k k, k where |·| stands for the cardinality of a set, leading to ficient to guarantee exact recovery. Sufficient condi- the following recovering x by asolutionto tions for exact recovery by (4) that only involve RIC have also been investigated in the literature. For ex- min z subject to y = Az. (3) ample, [26]arguedthat δ < 2 − 1 implies that 0 2k z∈R x is the unique solution to (4). This was improved to δ < 0.472 in [27]. More recently, [28] showed 2k Under mild regularity conditions on the sensing ma- that, for any given constant t ≥ 4/3, the condition trix A, it can be shown that the solution to (3) is in- δ < (t − 1)/t guarantees the exact recovery of deed well defined and unique, and thus correctly re- tk all k-sparse signals by  minimization. Moreover, covers x [9]. However, it is also well known [19]that solving (3) is NP-hard in general and thus infeasible for any > 0, δ < (t − 1)/t +  is not suffi- tk to compute for even moderate-size problems. The cient to ensure exact recovery of all k-sparse signals most popular way to overcome this challenge is the for large k. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 758 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW An immediate question following these results is tor z ∈ R , how to design a sensing matrix that satisfies these 2 2 2 (1 − δ )z ≤Az ≤ (1 + δ )z . conditions so that we can use (4) to recover x.It k|I k|I 2 2 2 (9) is now well understood that, for many random en- Similarly, A is said to obey the block-ROP of order sembles, that is, where each entry of A is indepen- (k, k ) if there exists a constant θ such that, for k,k |I dently sampled from a common distribution such every block k-sparse vector z and block k -sparse vec- as a Gaussian, Rademacher or other sub-Gaussian tor z with disjoint supports, distribution, δ < with overwhelming probabil- −2 ity, provided that m ≥ C k log (n/k), for some |Az, Az | ≤ θ z z  . (10) k ,k |I 1 2 2 2 constant C > 0. There has also been some re- cent progress in constructing deterministic sensing The constants δ and θ are referred to as block k|I k,k |I matrices that satisfy these RIP conditions; see e.g. k-RIC and block (k, k )-ROC, respectively. [29–31]. Clearly, any sufficient RIP conditions of standard minimization can be naturally extended to the set- ting of block-sparse recovery via mixed  / min- 2 1 Compressive sensing of block-sparse imization so that, for example, θ + δ < 1is k,k|I k|I signals also a sufficient condition for x to be the unique so- In many applications, the signal of interest may have lution to (8). more structured sparsity patterns. The most com- mon example is so-called block-sparsity where spar- Nonconvex methods sity occurs in a blockwise fashion rather than at the individual coordinate level. More specifically, let x ∈ In addition to the  -minimization-based approach, R be the concatenation of b signal ‘blocks’: there is also an extensive literature on nonconvex methods for sparse recovery where instead of the norm of z one minimizes a nonconvex objective x =[x ··· x x ··· x ··· x ··· x ] , 1 n n +1 n +n n−n +1 n 1 1 1 2 b function in z. The most notable example is the x[1] x[2] x[b] (0 < q < 1) (quasi-)norm, leading to (7) where each signal ‘block’ x[i] is of length n .Weas- min z subject to y = Az. (11) z∈R sume that x is block k-sparse in that there are at most k nonzero blocks among x[i]. As before, we are in- Some recent studies, e.g. [34–36], have shown that terested in the most block-sparse signal that satis- the solution of (11) can recover a sparse signal fies y = Az. To circumvent the potential computa- based on much fewer measurements when com- tional challenge, the following relaxation is often em- pared with the  minimization (4). In particular, the ployed: case of q = 1/2 has been treated extensively in [37– 39]. Other notable examples of nonconvex objective min z subject to y = Az, (8) functions include smoothly clipped absolute devia- 2 1 z∈R tion (SCAD) [40] and the minimax concave plus function [41], among others. where the mixed  / norm is defined as 2 1 Compressive sensing with general z = z[i ] . 2 1 2 dictionaries i =1 Thus far, we have focused on sparsity with respect It is not hard to see that, when each block has to the canonical basis ofR . In many applications, it size n = 1, (8) reduces to the  minimization given might be more appropriate to have sparsity with re- i 1 by (4). More generally, the optimization problem in spect to more general dictionaries; see e.g. [42–46]. (8) is convex and can be recast as a second-order More specifically, a signal x is represented by x = Dα n×n cone program, and thus can be solved efficiently. with respect to a dictionary D ∈ R , where α ∈ One can also extend the notion of RIP and ROP to R is the coordinate in the dictionary and is known the block-sparse setting; see e.g. [ 32,33]. For brevity, a priori to be sparse or nearly sparse. One can ob- write I ={n ,..., n }. viously treat A = AD as the new sensing matrix 1 b Definition 2.2 . A sensing matrix A is said to sat- and apply any of the aforementioned methods to ex- isfy the block-RIP of order k if there exists a constant ploit the sparsity of α. One of the drawbacks, how- δ ∈ [0, 1) such that, for every block k-sparse vec- ever, is that nice properties of the original sensing k|I Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 759 matrix A may not be inherited by A . In other words, fact and distinguish from the vector case treated in even if one carefully designs a sensing matrix A,ex- the last section, we shall write the underlying signal n ×n 1 2 act recovery of x may not be guaranteed despite its as a capital letter, X ∈ R , throughout this sec- sparsity with respect to D. Alternatively, some have tion. In these applications, we often observe a small advocated reconstructing x by the solution to the fol- fraction of the entries of X. The task of matrix com- lowing optimization problem: pletion is then to ‘complete’ the remaining entries. Formally, let  be a subset of [n ] × [n ] where 1 2 min D z subject to y = Az; (12) 1 [n] = {1, ..., n}. The goal of matrix completion is z∈R to recover X based on {X :(i, j) ∈ }, particularly ij see e.g. [43,44,47,48]. with the sample size || much smaller than the total number n n of entries. To fix ideas, we shall assume 1 2 that  is a uniformly sampled subset of [n ] × [n ], 1 2 Compressive phase retrieval although other sampling schemes have also been in- vestigated in the literature, e.g. [53–55]. In the previous subsections, we have mainly dis- Obviously, we cannot complete an arbitrary ma- cussed the problem of recovering a sparse signal trix from a subset of its entries. But it is possible from a small number of linear measurements. How- for low-rank matrices as their degrees of freedom ever, in some practical scenarios one can only ob- are much smaller than n n . But low rankness alone 1 2 serve some nonlinear measurements of the original is not sufficient. Consider a matrix with a single signal. The typical example is the so-called compres- nonzero entry; it is of rank one but it is impossible sive phase retrieval problem. In such a scenario, we ob- to complete it unless the nonzero entry is observed. serve the magnitude of the Fourier coefficients in- A formal way to characterize low-rank matrices that stead of their phase, which can be modeled as the can be completed from {X :(i, j) ∈ } was first in- ij form troduced in [14]. Definition 3.1 .Let U be a subspace of R of di- y =|a , x|, k = 1, ··· , m. (13) k k mension r and P be the orthogonal projection For recovering x, several studies [49–51]havecon- onto U. Then the coherence of U (with respect to the sidered the following  minimization: 1 standard basis (e )) is defined to be μ(U ) ≡ max P e  . min z subject to y =|a , z|, U i 1 k k n r 1≤i ≤n z∈R It is clear that the smallest possible value for k = 1, ··· , m. (14) μ(U) is 1 and the largest possible value for μ(U) is n/r.Let M be an n × n matrix of rank r and 1 2 By introducing a strong notion of RIP, Voroninski with column and row spaces denoted by U and V, and Xu [51] built a parallel result for compressive respectively. We shall say that M satisfies the inco- phase retrieval with the classical compressive sens- herence condition with parameter μ if max (μ(U), ing. Specifically, they proved that a k-sparse signal x μ(V)) ≤ μ . can be recovered from m = O(k log (n/k)) random Now let X be an incoherent matrix of rank r.Ina Gaussian phaseless measurements by solving (14). similar spirit as the vector case, a natural way to re- Unlike the standard convex  minimization (4), the construct it from {X :(i, j) ∈ } is to seek, among ij problem (14) is a nonconvex problem, but some ef- all matrices whose entries indexed by  agree with ficient algorithms have been developed to compute our observations, the one with the smallest rank: it; see e.g. [50,52]. min rank(Z)subjectto Z = X , ij ij n ×n 1 2 Z ∈R RECOVERY OF LOW-RANK MATRICES ∀(i , j ) ∈ . (15) We now consider recovering a low-rank matrix, a problem often referred to as matrix completion. Again, to overcome the computational challenges in directly minimizing matrix ranks, the following con- vex program is commonly suggested: Matrix completion via nuclear norm min Z  subject to Z = X , (i , j ) ∈ , minimization ∗ ij ij n ×n 1 2 Z ∈R In many practical situations such as collaborative fil- (16) tering, system identification and remote sensing, to where the nuclear norm · is the sum of all name a few, the signal that we aim to recover often- singular values. As before, we are interested in times is a matrix rather than a vector. To signify this when the solution to (16) is unique and correctly Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 760 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW recovers x. Candes and Recht [14] were the first to Mixed sparsity show that this is indeed the case, for almost all  that In some applications, the matrix that we want to re- are large enough. These results are probabilistic in cover is not necessarily of low rank but differs from nature due to the randomness of , that is, one can a low-rank matrix only by a small number of en- correctly recover x using (16) with high probability tries; see e.g. [17,18]. In other words, we may write regarding min {n , n }. The sample size requirement 1 2 X = S + L, where S is a sparse matrix with only a to ensure exact recovery of X by the solution of (16) small number of nonzero entries while L is a ma- was later improved in [16]to ||≥ Cr(n + n ) · 1 2 trix of low rank. It is clear that, even if we observe X polylog(n + n ), where C is a constant that de- 1 2 entirely, the decomposition of X into a sparse com- pends on the coherence coefficients only. It is clear ponent S and a low-rank component L may not be to see that this requirement is (nearly) optimal in uniquely defined. General conditions under which that there are O(r(n + n )) free parameters in spec- 1 2 such a decomposition is indeed unique are provided ifying a rank-r matrix. in, for example, [18]. In the light of the previous dis- cussions, it is natural to consider reconstructing X from observations {X :(i, j) ∈ } by the solution ij Matrix completion from affine to measurements More generally, one may consider recovering a low- min Z  + λvec(Z ) rank matrix based on affine measurements. More 1 ∗ 2 n ×n 1 2 Z ,Z ∈R 1 2 n ×n m 1 2 specifically, let A : R → R be a linear map subject to (Z + Z ) = X , ∀(i , j ) ∈ . 1 2 ij ij such that A(X) = y. We aim to recover X based on the information that A(X) = y.Itisclear (19) that the canonical matrix completion problem dis- cussed in the previous subsection corresponds to the case when A(X) ={X :(i , j ) ∈ }. Similarly, ij This strategy has been investigated extensively in we can proceed to reconstruct X by the solution to the literature; see e.g. [17]. Further developments in this direction can also be found in [61–64], among others. min Z  subject to y = A(Z). n ×n 1 2 Z ∈R (17) Nonconvex methods It is of interest to know under which kind of sensing operator A can X be recovered in exactly this way. Just as  norm is a convex relaxation of the  norm, 1 0 An answer is given in [56], which extends the con- the nuclear norm is also a convex relaxation of the cept of RIP to general linear operator for matrices. rank for a matrix. In addition to these convex ap- n ×n 1 2 Definition 3.2 . A linear operator A : R → proaches, nonconvex methods have also been pro- R is said to satisfy the matrix RIP of order r if there posed by numerous authors. The most common ex- exists a constant δ such that ample is the Schatten- q (0 < q < 1) (quasi-)norm defined by Z Z 1 − δ Z  ≤A(Z) ≤ 1 + δ Z F  F r 2 r 1/q (18) min{n ,n } 1 2 n ×n 1 2 holds for all matrices Z ∈ R of rank at most r. X  = σ , (20) q i Recht et al. [56] further proved that, if A i =1 satisfies the matrix RIP (18) with δ < 1/10, 5r then one can recover a rank-r matrix X from where σ , σ , ··· are the singular values of X.Itis 1 2 n = O(r(n + n ) log (n n )) measurements by the 1 2 1 2 clear that, when q → 0, X → rank(X)while the sq solution to (17). Note that the condition δ < 5r nuclear norm corresponds to the case when q = 1. 1/10 has been dramatically weakened; see e.g. One may now consider reconstructing X by the so- [28,57]. lution to Besides the aforementioned low-rank matrix completion problems, some recent studies, e.g. [58– 60], have drawn attention to the so-called high-rank min Z  subject to y = A(Z); matrix completion, in which the columns of the ma- n ×n 1 2 Z ∈R trix belong to a union of subspaces and, as a result, (21) the rank can be high or even full. see e.g. [65–68], among others. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 761 matrix nuclear norm, computing the tensor nuclear RECOVERY OF LOW-RANK norm, and thereby the problem (23), is NP-hard. HIGHER-ORDER TENSORS Hence, various relaxations and approximate algo- In an increasing number of modern applications, rithms have been introduced in the literature; see e.g. the object to be estimated has a higher-order tensor [75–79], and references therein. This is a research structure. Typical examples include video inpaint- area in its infancy and many interesting issues need ing [69], scan completion [70], multichannel EEG to be addressed. (electroencephalogram) compression [71], traffic data analysis [72] and hyperspectral image restora- tion [73], among many others. Similar to matrices, APPLICATIONS in many of these applications, we are interested in In the previous sections, we have given an overview recovering a low-rank tensor either from observing a subset of its entries or a collection of affine mea- of some basic ideas and techniques in dealing with surements. sparsity in vectors and low rankness in matrices and Despite the apparent similarities between matri- tensors. We now give a couple of examples to illus- ces and higher-order tensors, it is delicate to extend trate how they can be used in action. the idea behind nuclear norm minimization to the latter because matrix-style singular value decompo- sition does not exist for higher-order tensors. A com- Background subtraction with mon approach is to first unfold a higher-order tensor compressive imaging to a matrix and then apply a matrix-based approach Background subtraction in image and video has to recover a tensor. Consider, for example, a third- attracted a lot of attention in the past couple of d ×d ×d 1 2 3 order tensorX ∈ R . We can collapse its sec- decades. It aims at simultaneously separating video ond and third indices, leading to a d × (d d )ma- 1 2 3 background and extracting the moving objects from trix X .If X is of low rank, then so is X .Wecan (1) (1) a video stream, and can provide important clues for exploit the low rankness of X by minimizing its nu- (1) various applications such as moving object detection clear norm. Clearly we can also collapse the first and [80] and object tracking in surveillance [81], among third indices of X , leading to a matrix X , and the (2) numerous others. first and second, leading to X . If we want to com- (3) Conventional background subtraction tech- plete a low-rank tensor X based on its entries X (ω) niques usually consist of four steps: video acqui- for ω ∈ ⊂[d ] × [d ] × [d ], we can then con- 1 2 3 sition, encoding, decoding and separating the sider recovering X by the solution to the following moving objects from the background. This scheme convex program: needs to fully sample the video frames with large computational and storage requirements, followed by well-designed video coding and background min Z ( j ) ∗ d ×d ×d 1 2 3 Z∈R subtraction algorithms. To alleviate the burden of j =1 computation and storage, a newly-developed com- subject to Z(ω) = X (ω), ∀ω ∈ . (22) pressive imaging scheme [82–84] has been used for background subtraction by combining the video Efficient algorithms for solving matrix nuclear norm acquisition, coding and background subtraction minimization (16) such as the alternating direc- into a single framework, as illustrated in Fig. 1.It tion method of multipliers (ADMM) and Douglas– simultaneously achieves background subtraction Rachford operator splitting methods can then be and video reconstruction. In this setting, the main readily adapted to solve (22); see e.g. [3,4]. objective is then to maximize the reconstruction As pointed out in [5], however, such an approach and separation accuracies using as few compressive fails to exploit fully the multilinear structure of a ten- measurements as possible. sor and is thus suboptimal. Instead, directly minimiz- Several studies have been carried out for back- ing the tensor nuclear norm was suggested: ground subtraction from the perspective of com- pressive imaging. In the seminal work of Cevher et al. min Z d ×d ×d 1 2 3 Z∈R [85], the background subtraction problem is formu- lated as a sparse recovery problem. They showed that subject to Z(ω) = X (ω), ∀ω ∈ , (23) the moving objects can be recovered by learning a where the tensor nuclear norm · is defined as low-dimensional compressed representation of the the dual norm of the tensor spectral norm ·.For background image. More recently, the robust princi- more discussions on the tensor nuclear and spec- ple component analysis (RPCA) approach has also tral norms, please see [74]. Unfortunately, unlike the been used to deal with the problem of background Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 762 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW Compressive Reconstruction with foreground- measurements background separation form Transmission Figure 1. The framework of background subtraction with compressive imaging. subtraction with compressive imaging, in which they 3D-TV term · is defined as 3D-TV commonly model the video as a matrix with columns X  : =X (i , j, k) of vectorized video frames and then decompose the 3D-TV h 1 matrix into a low-rank matrix L and a sparse matrix +X (i , j, k) +X (i , j, k) , v 1 t 1 S; see e.g. [86–89]. Although methods based on RPCA have where achieved satisfactory performance, they fail to exploit the finer structures of background and fore- X (i , j, k): = X (i , j + 1, k) − X (i , j, k), ground after vectorizing the video frames. It appears X (i , j, k): = X (i + 1, j, k) − X (i , j, k), more advantageous to model the spatio-temporal information of background and foreground using X (i , j, k): = X (i , j, k + 1) − X (i , j, k). direct tensor representation of a video. To this end, a novel tensor RPCA approach has been Because a 3D patch in a video background is simi- proposed in [90] for background subtraction lar to many other 3D patches over the video frames, from compressive measurements by decomposing one can model the video background using several the video into a static background with spatio- groups of similar video 3D patches, where each patch temporal correlation and a moving foreground group corresponds to a fourth-order tensor. Inte- with spatio-temporal continuity within a tensor grating the patch-based modeling idea into (24), representation framework. More specifically, one one can easily get a patch-group-based tensor RPCA can use 3D total variation (3D-TV) to characterize model. It should be noted that solving the noncon- the spatio-temporal continuity underlying the video vex tensor RPCA model (24) as well as its patch- foreground, and low-rank Tucker decomposition to group-based form are computationally difficult. In model the spatio-temporal correlation of the video practice, we can find a local solution using a multi- background, which leads to the following tensor block version of ADMM. For more details, please re- RPCA model: fer to Section V of [90]. Fig. 2 gives an example based on three real videos. min λS + E  It is evident that the proposed tensor models enjoy 3D-TV X , S, E , a superior performance over the other popular ma- G, U , U , U 1 2 3 trix models both in terms of the quality of the recon- structed videos and in terms of the separation of the subject to X = L + E + S, moving objects. This suggests that using a direct ten- L = G × U × U × U , sor modeling technique to deal with practical higher- 1 1 2 2 3 3 order tensor data can utilize more useful structures. y = A(X ), (24) where the factor matrices U and U are orthogonal 1 2 Hyperspectral compressive sensing in columns for two spatial modes, the factor matrix U is orthogonal in columns for the temporal mode, Hyperspectral imaging employs an imaging spec- the core tensor G interacts with these factors and the trometer to collect hundreds of spectral bands Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 763 Ground Truth SparCS ReProCS SpLR H-TenRPCA PG-TenRPCA (a) (b) (c) Figure 2. Visual comparison of two tensor models (i.e. H-TenRPCA and PG-TenRPCA) proposed in [90] and three popular matrix models (i.e. SparCS [86], ReProcs [87] and SpLR [88]) under the sampling ratio 1/30. The first column shows the original video frames from different video volumes (a)–(c); the second to sixth columns correspond to the results produced by all the compared methods, respectively. Here, for each method, the reconstruction result of the original video frame (upper panels) and the detection result of moving objects in the foreground (lower panels) is shown. ranging from ultraviolet to infrared wavelengths for tensively investigated. Similar to other compressive the same area on the surface of the Earth. It has a sensing problems, the main objectives of HCS are wide range of applications including environmen- to design easy hardware encoding implementation tal monitoring, military surveillance and mineral ex- and develop an efficient sparse reconstruction pro- ploration, among numerous others [91,92]. Figura- cedure. In what follows we shall focus primarily tively speaking, a hyperspectral image can be treated on the latter. For hardware implementation, inter- as a 3D (x, y, λ) data cube, where x and y represent ested readers are referred to [93,94] and references two spatial dimensions of the scene, and λ represents therein. the spectral dimension comprising a range of wave- Like other natural images, hyperspectral images lengths. Typically, such hyperspectral cubes are col- can be sparsified by using certain transformations. lected by an airborne sensor or a satellite and sent Traditional sparse recovery methods such as  min- to a ground station on Earth for subsequent process- imization and TV minimization are often used for ing. Noting that the dimension of λ is usually in the such purposes; see e.g. [95,96]. To further exploit hundreds, hyperspectral cubes are of a fairly large the inherent spectral correlation of hyperspectral im- size even for moderate dimensions of x and y.This ages, a series of work based on low-rank modeling makes it necessary to devise effective techniques for has been carried out in recent years. For example, hyperspectral data compression, due to the limited Golbabaee and Vandergheynst proposed in [97]a bandwidth of the link connection between the satel- joint nuclear and  / minimization method to de- 2 1 lite/aerospace and the ground station. scribe the spectral correlation and the joint-sparse In the last few years, a popular hyperspectral com- spatial wavelet representations of hyperspectral im- pressive sensing (HCS) scheme based on the prin- ages. Then, they modeled the spectral correlation ciple of compressive sensing for hyperspectral data together with the spatial piecewise smoothness of compression, as illustrated in Fig. 3, has been ex- hyperspectral images by using a joint nuclear and Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 764 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW Satellites, aerospace Transmit measurements Encoding by compressive sensing Receiving Reconstruction via sparse modeling Ground station Figure 3. The framework of hyperspectral compressive sensing. TV norm minimization method [98]. Similar to the surveillance videos, however, modeling a hyperspec- tral cube as a matrix cannot utilize the finer spatial- and-spectral information, leading to suboptimal re- construction results under relatively low sampling ratios (e.g. 1% of the whole image size). To further exploit the compressibility underlying a hyperspectral cube, one may consider such a sin- gle cube as a tensor with three modes (width, height and band) and then identify the hidden spatial- and-spectral structures using direct tensor modeling techniques. Precisely, all the bands of a hyperspec- tral image have very strong correlation in the spec- tral domain, and each band, if considered as a ma- trix, has relatively strong correlation in the spatial domain; such spatial-and-spectral correlation can be modeled through low-rank Tucker decomposition. In addition, the intensity at each voxel is likely to be similar to its neighbors, which can be characterized by smoothness using the so-called 3D-TV penalty. In summary, [99] considered the following joint ten- sor Tucker decomposition and 3D-TV minimiza- tion model: min λX  + E 3D-TV X , E , G, U , U , U 1 2 3 subject to X = G × U × U × U + E , 1 1 2 2 3 3 y = A(X ). (25) It is clear that the above minimization problem is highly nonconvex. One often looks for good local so- lutions using a multi-block ADMM algorithm. Figure 4. Vision comparison of the tensor method (25) over three other competing Fig. 4 gives an example of the first band of four hy- methods on the rst fi band of four different hyperspectral datasets (a)–(d). Here the perspectral datasets, respectively reconstructed by sampling ratio is 1%. The last column shows the original image bands. The columns different methods, with the sampling ratio at 1%. It from the first to the fourth correspond to the produced results from the KCS method [96], the JNTV method [98], the ST-NCS method [100] and the tensor method (25), is evident that the proposed tensor method could respectively. provide nearly perfect reconstruction. In addition, Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 765 sparse tensor and nonlinear compressive sensing 13. Candes ` E, Romberg J and Tao T. Stable signal recovery from (ST-NCS) performs slightly better than Kronecker incomplete and inaccurate measurements. Comm Pure Appl compressive sensing (KCS) and joint nuclear/TV Math 2006; 59: 1207–23. norm minimization (JNTV) in terms of reconstruc- 14. Candes ` E and Recht B. Exact matrix completion via convex op- tion accuracy, because of the use of a direct ten- timization. Found Comput Math 2009; 9: 717–72. sor sparse representation of a hyperspectral cube. 15. Candes ` E and Tao T. The power of convex relaxation: near- Both these findings demonstrate the power of ten- optimal matrix completion. IEEE Trans Inform Theor 2010; 56: sor modeling techniques. It is also worth noting that, 2053–80. compared with ST-NCS, the images reconstructed 16. Gross D. Recovering low-rank matrices from few coefficients with the method (25) are clearer and sharper. in any basis. IEEE Trans Inform Theor 2011; 57: 1548–66. 17. Candes ` E, Li X and Ma Y et al. Robust principal component analysis? JACM 2011; 58: 1–39. ACKNOWLEDGEMENT 18. Chandrasekaran V, Sanghavi S and Parrilo P et al. Rank- The authors thank the associate editor and three referees for help- sparsity incoherence for matrix decomposition. SIAM J Optim ful comments. 2011; 21: 572–96. 19. Natarajan B. Sparse approximate solutions to linear systems. SIAM J Comput 1995; 24: 227–34. FUNDING 20. Donoho D, Elad M and Temlyakov VN. Stable recovery of This work was supported in part by the National Natural Sci- sparse overcomplete representations in the presence of noise. ence Foundation of China (11501440 and 61273020 to IEEE Trans Inform Theor 2006; 52: 6–18. Y.W., 61373114, 61661166011 and 61721002 to D.Y.M.), 21. Cohen A, Dahmen W and DeVore R. Compressed sensing the National Basic Research Program of China (973 Program) and best k-term approximation. J Am Math Soc 2009; 22: (2013CB329404 to D.Y.M.) and the National Science Founda- 211–31. tion (DMS-1265202 to M.Y.). 22. Eldar Y and Kutyniok G. Compressed Sensing: Theory and Ap- plications. Cambridge: Cambridge University Press, 2012. REFERENCES 23. Foucart S and Rauhut H. A Mathematical Introduction to Com- 1. Liu J, Musialski P and Wonka P et al. Tensor completion for pressive Sensing. Berlin: Springer, 2013. estimating missing values in visual data. In: Proceedings of In- 24. Cai TT, Wang L and Xu G. New bounds for restricted isometry ternational Conference on Computer Vision, 2009. constants. IEEE Trans Inform Theor 2010; 56: 4388–94. 2. Tomioka R, Hayashi K and Kashima H. Estimation of low-rank 25. Cai TT and Zhang A. Compressed sensing and affine rank min- tensors via convex optimization. arXiv:10100789. imization under restricted isometry. IEEE Trans Signal Process 3. Gandy S, Recht B and Yamada I. Tensor completion and low- 2013; 61: 3279–90. n-rank tensor recovery via convex optimization. Inverse Probl 26. Candes ` E. The restricted isometry property and its implications 2011; 27: 025010. for compressed sensing. Compt Rendus Math 2008; 346: 589– 4. Liu J, Musialski P and Wonka P. Tensor completion for esti- mating missing values in visual data. IEEE Trans Pattern Anal 27. Cai TT, Wang L and Xu G. Shifting inequality and recovery of Mach Intell 2013; 34: 208–20. sparse signals. IEEE Trans Signal Process 2010; 58: 1300–8. 5. Yuan M and Zhang CH. On tensor completion via nuclear norm 28. Cai TT and Zhang A. Sparse representation of a polytope and minimization. Found Comput Math 2016; 16: 1031–68. recovery of sparse signals and low-rank matrices. IEEE Trans 6. Chen SS, Donoho DL and Saunders MA. Atomic decomposition Inform Theor 2014; 60: 122–32. by basis pursuit. SIAM J Sci Comput 1998; 20: 33–61. 29. DeVore R. Deterministic constructions of compressed sensing 7. Donoho DL and Huo X. Uncertainty principles and ideal atomic matrices. J Complex 2007; 23: 918–25. decomposition. IEEE Trans Inform Theor 2001; 47: 2845–62. 30. Bourgain J, Dilworth SJ and Ford K et al. Explicit constructions 8. Donoho DL and Elad M. Optimally sparse representation in of RIP matrices and related problems. Duke Math J 2011; 159: general (nonorthogonal) dictionaries via  minimization. Proc 145–85. Natl Acad Sci 2003; 100: 2197–202. 31. Xu Z. Deterministic sampling of sparse trigonometric polyno- 9. Candes ` E and Tao T. Decoding by linear programming. IEEE mials. J Complex 2011; 27: 133–40. Trans Inform Theor 2005; 51: 4203–15. 32. Eldar Y and Mishali M. Robust recovery of signals from a struc- 10. Candes ` E, Romberg J and Tao T. Robust uncertainty principles: tured union of subspaces. IEEE Trans Inform Theor 2009; 55: exact vector reconstruction from highly incomplete frequency 5302–16. information. IEEE Trans Inform Theor 2006; 52: 489–509. 33. Wang Y, Wang J and Xu Z. On recovery of block-sparse signals 11. Donoho D. Compressed sensing. IEEE Trans Inform Theor 2006; via mixed  / (0 < q ≤ 1) norm minimization. EURASIP J Adv 2 q 52: 1289–306. Signal Process 2013; 76: 1–17. 12. Candes ` E and Tao T. Near-optimal signal recovery from random 34. Chartrand R. Exact reconstruction of sparse signals via non- projections: universal encoding strategies. IEEE Trans Inform convex minimization. IEEE Signal Process Lett 2007; 14: Theor 2006; 52: 5406–25. 707–10. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 766 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW 35. Sun Q. Recovery of sparsest signals via  -minimization.ApplComputHarmon 59. Elhamifar E. High-rank matrix completion and clustering under self-expressive Anal 2012; 32: 329–41. models. In: Proceedings of Advances in Neural Information Processing Sys- 36. Song CB and Xia ST. Sparse signal recovery by  minimization under re- tems, 2016. stricted isometry property. IEEE Signal Process Lett 2014; 21: 1154–8. 60. Li CG and Vidal R. A structured sparse plus structured low-rank framework 37. Xu Z, Zhang H and Wang Y et al.  regularization. Sci China Inform Sci for subspace clustering and completion. IEEE Trans Signal Process 2016; 64: 1/2 2010; 53: 1159–69. 6557–70. 38. Xu Z, Chang X and Xu F et al.  regularization: a thresholding representa- 61. Zhou ZH, Li X and Wright J et al. Stable principal component pursuit. In: Pro- 1/2 tion theory and a fast solver. IEEE Trans Neural Network Learn Syst 2012; 23: ceedings of the 2010 IEEE International Symposium on Information Theory, 1013–27. 2010. 39. Zeng J, Lin S and Wang Y et al.  regularization: convergence of it- 62. Ganesh A, Wright J and Li Xetal. Dense error correction for low-rank matrices 1/2 erative half thresholding algorithm. IEEE Trans Signal Process 2014; 62: via principal component pursuit. In: Proceedings of the 2010 IEEE International 2317–29. Symposium on Information Theory, 2010. 40. Fan J and Li R. Variable selection via nonconcave penalized likelihood and its 63. Zhao Q, Meng D and Xu Z et al. Robust principal component analysis with oracle properties. J Am Stat Assoc 2001; 96: 1348–60. complex noise. In: Proceedings of the 31st International Conference on Ma- 41. Zhang CH. Nearly unbiased variable selection under minimax concave penalty. chine Learning, 2014. Ann Stat 2010; 38: 894–942. 64. Netrapalli P, Niranjan U and Sanghavi S et al. Non-convex robust PCA. 42. Rauhut H, Schnass K and Vandergheynst P. Compressed sensing and redun- In: Proceedings of Advances in Neural Information Processing Systems 27, dant dictionaries. IEEE Trans Inform Theor 2013; 29: 1401–12. 2014. 43. Candes ` E, Eldar Y and Needell D et al. Compressed sensing with coherent and 65. Zhang M, Huang ZH and Zhang Y. Restricted p-isometry properties of noncon- redundant dictionaries. Appl Comput Harmon Anal 2010; 31: 59–73. vex matrix recovery. IEEE Trans Inform Theor 2013; 59: 4316–23. 44. Elad M, Milanfar P and Rubinstein R. Analysis versus synthesis in signal pri- 66. Wang J, Wang M and Hu X et al. Visual data denoising with a unified ors. Appl Comput Harmon Anal 2007; 23: 947–68. Schatten-p norm and  norm regularized principal component pursuit. Pat- 45. Lin J, Li S and Shen Y. New bounds for restricted isometry constants with tern Recogn 2015; 48: 3135–44. coherent tight frames. IEEE Trans Signal Process 2013; 61: 611–21. 67. Zhao Q, Meng D and Xu Z et al.  -norm low-rank matrix factorization by 46. Lin J and Li S. Sparse recovery with coherent tight frames via analydsis variational Bayesian method. IEEE Trans Neural Network Learn Syst 2015; 26: Dantzig selector and analysis LASSO. Appl Comput Harmon Anal 2014; 37: 825–39. 126–39. 68. Yue MC and So AMC. A perturbation inequality for concave functions of sin- 47. Liu Y, Mi T and Li S. Compressed sensing with general frames via optimal- gular values and its applications in low-rank matrix recovery. Appl Comput dual-based  -analysis. IEEE Trans Inform Theor 2012; 58: 4201–14. Harmon Anal 2016; 40: 396–416. 48. Li S and Lin J. Compressed sensing with coherent tight frames via  - 69. Korah T and Rasmussen C. Spatio-temporal inpainting for recovering texture minimization for 0 < q ≤ 1. Inverse Probl Imag 2014; 8: 761–77. maps of occluded building facades. IEEE Trans Image Process 2007; 16: 2262– 49. Moravec M, Romberg J and Baraniuk R. Compressive phase retrieval. In: Pro- 71. ceedings of SPIE, the International Society for Optics and Photonics, 2007. 70. Pauly M, Mitra N and Giesen J et al. Example-based 3D scan completion. In: 50. Yang Z, Zhang C and Xie L. Robust compressive phase retrieval via  mini- Proceedings of the Symposium on Geometry Processing, 2005. mization with application to image reconstruction. arXiv:13020081. 71. Acar E, Dunlavy D and Kolda T et al. Scalable tensor factorizations for incom- 51. Voroninski V and Xu Z. A strong restricted isometry property, with an applica- plete data. Chemometr Intell Lab Syst 2001; 106: 41–56. tion to phaseless compressed sensing. Appl Comput Harmon Anal 2016; 40: 72. Xie K, Wang L and X Wetal. Accurate recovery of internet traffic data: a tensor 386–95. completion approach. In: Proceedings of the 35th Annual IEEE International 52. Schniter P and Rangan S. Compressive phase retrieval via general- Conference on Computer Communications, 2016. ized approximate message passing. IEEE Trans Signal Process 2015; 63: 73. Peng Y, Meng D and Xu Z et al. Decomposable nonlocal tensor dictionary 1043–55. learning for multispectral image denoising. In:ProceedingsofIEEEConference 53. Foygel R, Shamir O and Srebro N et al. Learning with the weighted trace-norm on Computer Vision and Pattern Recognition, 2014. under arbitrary sampling distributions. In: Proceedings of Advances in Neural 74. Hillar CJ and Lim LH. Most tensor problems are NP-hard. JACM 2013; 60: Information Processing Systems 24, 2011. 1–39. 54. Chen Y, Bhojanapalli S and Sanghavi S et al. Coherent matrix completion. In: 75. Nie J and Wang L. Semidefinite relaxations for best rank-1 tensor approxima- Proceedings of the 31st International Conference on Machine Learning, 2014. tions. SIAM J Matrix Anal Appl 2014; 35: 1155–79. 55. Cai TT and Zhou WX. Matrix completion via max-norm constrained optimiza- 76. Jiang B, Ma S and Zhang S. Tensor principal component analysis via convex tion. Electron J Stat 2016; 10: 1493–525. optimization. Math Program 2015; 150: 423–57. 56. Recht B, Fazel M and Parrilo P. Guaranteed minimum-rank solutions of linear 77. Yang Y, Feng Y and Suykens J. A rank-one tensor updating algorithm for tensor matrix equations via nuclear norm minimization.SIAMRev 2010; 52: 471–501. completion. IEEE Signal Process Lett 2015; 22: 1633–7. 57. Candes ` E and Plan Y. Tight oracle inequalities for low-rank matrix recovery 78. Zhao Q, Meng D and Kong X et al. A novel sparsity measure for tensor from a minimal number of noisy random measurements. IEEE Trans Inform recovery. In: Proceedings of International Conference on Computer Vision, Theor 2011; 57: 2342–59. 2015. 58. Eriksson B, Balzano L and Nowak R. High-rank matrix completion. In: Proceed- 79. Xie Q, Zhao Q and Meng Detal. Kronecker-Basis-Representation based tensor ings of the 15th International Conference on Artificial Intelligence and Statis- sparsity and its Applications to tensor recovery. IEEE Trans Pattern Anal Mach tics, 2012. Intell 2017; 40: 1888–902. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 767 80. Wang T, Backhouse A and Gu I. Online subspace learning on Grassmann man- 90. Cao W, Wang Y and Sun J et al. Total variation regularized tensor RPCA for ifold for moving object tracking in video. In: Proceedings of IEEE International background subtraction from compressive measurements. IEEE Trans Image Conference on Acoustics, Speech, and Signal Process, 2008. Process 2016; 25: 4075–90. 81. Beleznai C, Fruhstuck B and Bischof H. Multiple object tracking using local 91. Goetz AFH. Three decades of hyperspectral remote sensing of the Earth: a PCA. In: Proceedings of the 18th International Conference on Pattern Recog- personal view. Rem Sens Environ 2009; 113: S5–S6. nition, 2006. 92. Willett R, Duarte M and Davenport M et al. Sparsity and structure in hyper- 82. Wakin M, Laska JN and Duarte MF et al. Compressive imaging for video spectral imaging: sensing, reconstruction, and target detection. IEEE Signal representation and coding. In: Proceedings of Picture Coding Symposium, Process Mag 2014; 31: 116–26. 2006. 93. Arce G, Brady D and Carin L et al. Compressive coded aperture spectral imag- 83. Takhar D, Laska JN and Wakin M et al. A new compressive imaging camera ing: an introduction. IEEE Signal Process Mag 2014; 31: 105–15. architecture using optical-domain compression. In: Proceedings of Computa- 94. Yuan X, Tsai TH and Zhu R et al. Compressive hyperspectral imaging with side tional Imaging IV at SPIE Electronic Imaging, 2006. information. IEEE J Sel Top Signal Process 2015; 9: 964–76. 84. Duarte M, Davenport M and Takhar D et al. Single-pixel imaging via compres- 95. Li C, Sun T and Kelly KF et al. A compressive sensing and unmixing scheme for sive sampling. IEEE Signal Process Mag 2008; 25: 83–91. hyperspectral data processing. IEEE Trans Image Process 2012; 21: 1200–10. 85. Cevher V, Sankaranarayanan A and Duarte M et al. Compressive sensing for 96. Duarte M and Baraniuk R. Kronecker compressive sensing. IEEE Trans Image background subtraction. In: Proceedings of the 10th European Conference on Process 2012; 21: 494–504. Computer Vision, 2008. 97. Golbabaee M and Vandergheynst P. Hyperspectral image compressed sensing 86. Waters A, Sankaranarayanan A and Baraniuk R et al. A new compres- via low-rank and joint-sparse matrix recovery. In: Proceedings of IEEE Interna- sive imaging camera architecture using optical-domain compression. In: tional Conference on Acoustics, Speech and Signal Processing, 2012. Proceedings of Conference on Neural Information Processing Systems 24, 98. Golbabaee M and Vandergheynst P. Joint trace/TV norm minimization: a new 2011. efficient approach for spectral compressive imaging. In: Proceedings of 19th 87. Guo H, Qiu CL and Vaswani N. An online algorithm for separating sparse and IEEE International Conference on Image Processing, 2012. low-dimensional signal sequences from their sum. IEEE Trans Signal Process 99. Wang Y, Lin L and Zhao Q et al. Compressive sensing of hyperspectral images 2014; 62: 4284–97. via joint tensor tucker decomposition and weighted total variation regulariza- 88. Jiang H, Deng W and Shen Z. Surveillance video processing using compres- tion. IEEE Geosci Rem Sens Lett 2017; 14: 2457–61. sive sensing. Inverse Probl Imag 2014; 6: 201–14. 100. Yang S, Wang M and Li P et al. Compressive hyperspectral imaging via sparse 89. Jiang H, Zhao S and Shen Z et al. Surveillance video analysis using compres- tensor and nonlinear compressed sensing. IEEE Trans Geosci Rem Sens 2015; sive sensing with low latency. Bell Labs Tech J 2014; 18: 63–74. 53: 5943–57. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png National Science Review Oxford University Press

Sparse recovery: from vectors to tensors

National Science Review , Volume 5 (5) – Sep 1, 2018

Loading next page...
 
/lp/ou_press/sparse-recovery-from-vectors-to-tensors-9et0j5uEb0

References (132)

Publisher
Oxford University Press
Copyright
Copyright © 2022 China Science Publishing & Media Ltd. (Science Press)
ISSN
2095-5138
eISSN
2053-714X
DOI
10.1093/nsr/nwx069
Publisher site
See Article on Publisher Site

Abstract

Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 National Science Review 5: 756–767, 2018 REVIEW doi: 10.1093/nsr/nwx069 Advance access publication 14 July 2017 INFORMATION SCIENCE 1,2,∗ 1,3,∗ 4,∗ Yao Wang , Deyu Meng and Ming Yuan ABSTRACT Recent advances in various fields such as telecommunications, biomedicine and economics, among others, have created enormous amount of data that are often characterized by their huge size and high dimensionality. It has become evident, from research in the past couple of decades, that sparsity is a flexible and powerful notion when dealing with these data, both from empirical and theoretical viewpoints. In this survey, we review some of the most popular techniques to exploit sparsity, for analyzing high-dimensional vectors, matrices and higher-order tensors. Keywords: high-dimensional data, sparsity, compressive sensing, low-rank matrix recovery, tensors nary. Another common example is the recovery of INTRODUCTION School of n ×n 1 2 low-rank matrices where x ∈ R is assumed to Mathematics and The problem of sparse recovery is ubiquitous in have a rank much smaller than min {n , n }. In many 1 2 Statistics, Xi’an modern science and engineering applications. In practical situations, we are also interested in signals Jiaotong University, these applications, we are interested in inferring a of higher-order multilinear structure. For example, Xi’an 710049, China; high-dimensional object, namely a vector, a ma- it is natural to represent multispectral images by Shenyang Institute of trix or a higher-order tensor, from very few obser- a third-order multilinear array, or tensor, with the Automation, Chinese vations. Notable examples include identifying key Academy of Sciences, third index corresponding to different bandwidths. genes driving a complex disease, and reconstruct- Shenyang 10016, Clearly, vectors and matrices can be viewed as first- ing high-quality images or videos from compressive China; Ministry of order and second-order tensors as well. Despite the measurements, among many others. Education Key Lab of connection, moving from vectors and matrices to More specifically, consider linear measurements Intelligent Networks higher-order tensors could present significant new of an n-dimensional object x of the form: and Network Security, challenges. A common way to address these chal- Xi’an Jiaotong lenges is to unfold tensors to matrices; see e.g. [1–4]. University, Xi’an y =a , x, k = 1, ··· , m, (1) k k However, as recently pointed out in [5], the multi- 710049, China and 4 linear structure is lost in such matricization and, as a Department of where ·, · stands for the usual inner product in result, methods based on these techniques often lead Statistics, Columbia R , and a are a set of prespecified n-dimensional to suboptimal results. University, New York, k vectors. The number of measurements m is typically NY 10027, USA A general approach to sparse recovery is through much smaller than n so that the linear system (1) is solving the following constrained optimization underdetermined whenever m < n. Thus it is impos- Corresponding problem: authors. E-mails: sible to recover x from y in the absence of any ad- min S(z)subjectto y =a , z, yao.s.wang@gmail.com; ditional assumption. The idea behind sparse recov- k k dymeng@mail. ery is to assume that x actually resides in a subspace xjtu.edu.cn; k = 1, ··· , m, (2) whose dimensionality is much smaller than the am- ming.yuan@columbia.edu bient dimension n. where S(·) is an objective function that encourages A canonical example of sparse recovery is the so- sparse solutions. The success of this approach hinges Received 25 called compressive sensing for vectors, where x is as- upon several crucial aspects including, among oth- November 2016; sumed to have only a small number of, albeit un- ers, Revised 21 February known, nonzero coordinates. More generally, we call 2017; Accepted 21 how to choose the object function S(·); avector x ∈ R k-sparse if it can be represented March 2017 how to solve the optimization problem (2); by up to k elements from a predetermined dictio- The Author(s) 2017. Published by Oxford University Press on behalf of China Science Publishing & Media Ltd. All rights reserved. For permissions, plea se e-mail: journals.permissions@oup.com Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 757 how to design the sampling vectors a to facilitate  relaxation, which minimizes the  norm instead, k 1 1 recovery; leading to what is the minimum sample size requirement m min z subject to y = Az. (4) to ensure recovery? z∈R There is, by now, an impressive literature address- Assuming that x is k-sparse and the unique solution ing these issues when x is a vector or matrix; see e.g. to (3), a key question pertaining to the  relaxation [6–18], among numerous others. In this review, we (4) is: under what condition is it also the unique aim to survey some of the key developments, with a solution to (4)? The answer can be characterized particular focus on applications to image and video by various properties of the sensing matrix includ- analysis. ing the mutual incoherence property (MIP [20]), The rest of the paper is organized as follows. In null space property (NSP [21]) and restricted isom- the sections entitled ‘Recovery of sparse vectors’, etry property (RIP [9]), among others. We shall fo- ‘Recovery of low-rank matrices’ and ‘Recovery of cus primarily on RIP here. Interested readers are re- low-rank higher-order tensors’, we discuss the recov- ferred to [22,23] and references therein for further ery of sparse vector, low-rank matrix and low-rank discussions on MIP and NSP. tensor signals, respectively. Finally, a couple of illus- m×n Definition 2.1 [9] A sensing matrix A ∈ R is trative examples in image and video processing are said to satisfy the RIP of order k if there exists a con- given in the section entitled ‘Applications’. stant δ ∈ [0, 1) such that, for every k-sparse vector z ∈ R , RECOVERY OF SPARSE VECTORS 2 2 2 (1 − δ )z ≤Az ≤ (1 + δ )z . k k 2 2 2 We first consider recovering a sparse vector signal, (5) a problem more commonly known as compressive Similarly, A is said to obey the restricted orthogonal- sensing [10–13]. ity property (ROP) of order (k, k ) if there exists a constant θ such that, for every k-sparse vector z k,k and k -sparse vector z with non-overlapping support Compressive sensing of sparse signals sets, With slight abuse of notation, let y be an m- dimensional vector whose coordinates are the mea- |Az, Az | ≤ θ z z  . (6) k ,k 1 2 2 2 surement y , and A be an m × n matrix whose rows The constants δ and θ are called the k-restricted are given by a . It is then not hard to see that (1) can k k,k isometry constant (RIC) and (k, k )-restricted or- be more compactly written as y = Ax, where x is an thogonality constant (ROC), respectively. n-dimensional vector. Following the jargon in com- The concept was first introduced by Cand es ` and pressive sensing, hereafter we shall refer to A as the Tao [9], who showed that, if δ + θ + θ < 1, sensing matrix. k k, k k,2k then x is the unique solution to (4). This condition Obviously, when m < n, there may be infinitely has since been weakened. For example, Candes ` and many z that agree with the measurements in that Tao [13] showed that δ + 3θ < 2 suffices, and y = Az. Since x is known a priori to be sparse, it is 2k k,2k Cai et al. [24] required that δ + θ < 1. then naturally to seek among all these solutions the 1.25k k,1.25k More recently, Cai and Zhang [25] further weak- one that is sparsest. As mentioned before, an obvious ened the condition to δ + θ < 1 and showed that way to measure sparsity of x is its  norm: k k, k the upper bound 1 is sharp in the sense that, for any x =|{i : x = 0}|, > 0, the condition δ + θ < 1 +  is not suf- k k, k where |·| stands for the cardinality of a set, leading to ficient to guarantee exact recovery. Sufficient condi- the following recovering x by asolutionto tions for exact recovery by (4) that only involve RIC have also been investigated in the literature. For ex- min z subject to y = Az. (3) ample, [26]arguedthat δ < 2 − 1 implies that 0 2k z∈R x is the unique solution to (4). This was improved to δ < 0.472 in [27]. More recently, [28] showed 2k Under mild regularity conditions on the sensing ma- that, for any given constant t ≥ 4/3, the condition trix A, it can be shown that the solution to (3) is in- δ < (t − 1)/t guarantees the exact recovery of deed well defined and unique, and thus correctly re- tk all k-sparse signals by  minimization. Moreover, covers x [9]. However, it is also well known [19]that solving (3) is NP-hard in general and thus infeasible for any > 0, δ < (t − 1)/t +  is not suffi- tk to compute for even moderate-size problems. The cient to ensure exact recovery of all k-sparse signals most popular way to overcome this challenge is the for large k. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 758 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW An immediate question following these results is tor z ∈ R , how to design a sensing matrix that satisfies these 2 2 2 (1 − δ )z ≤Az ≤ (1 + δ )z . conditions so that we can use (4) to recover x.It k|I k|I 2 2 2 (9) is now well understood that, for many random en- Similarly, A is said to obey the block-ROP of order sembles, that is, where each entry of A is indepen- (k, k ) if there exists a constant θ such that, for k,k |I dently sampled from a common distribution such every block k-sparse vector z and block k -sparse vec- as a Gaussian, Rademacher or other sub-Gaussian tor z with disjoint supports, distribution, δ < with overwhelming probabil- −2 ity, provided that m ≥ C k log (n/k), for some |Az, Az | ≤ θ z z  . (10) k ,k |I 1 2 2 2 constant C > 0. There has also been some re- cent progress in constructing deterministic sensing The constants δ and θ are referred to as block k|I k,k |I matrices that satisfy these RIP conditions; see e.g. k-RIC and block (k, k )-ROC, respectively. [29–31]. Clearly, any sufficient RIP conditions of standard minimization can be naturally extended to the set- ting of block-sparse recovery via mixed  / min- 2 1 Compressive sensing of block-sparse imization so that, for example, θ + δ < 1is k,k|I k|I signals also a sufficient condition for x to be the unique so- In many applications, the signal of interest may have lution to (8). more structured sparsity patterns. The most com- mon example is so-called block-sparsity where spar- Nonconvex methods sity occurs in a blockwise fashion rather than at the individual coordinate level. More specifically, let x ∈ In addition to the  -minimization-based approach, R be the concatenation of b signal ‘blocks’: there is also an extensive literature on nonconvex methods for sparse recovery where instead of the norm of z one minimizes a nonconvex objective x =[x ··· x x ··· x ··· x ··· x ] , 1 n n +1 n +n n−n +1 n 1 1 1 2 b function in z. The most notable example is the x[1] x[2] x[b] (0 < q < 1) (quasi-)norm, leading to (7) where each signal ‘block’ x[i] is of length n .Weas- min z subject to y = Az. (11) z∈R sume that x is block k-sparse in that there are at most k nonzero blocks among x[i]. As before, we are in- Some recent studies, e.g. [34–36], have shown that terested in the most block-sparse signal that satis- the solution of (11) can recover a sparse signal fies y = Az. To circumvent the potential computa- based on much fewer measurements when com- tional challenge, the following relaxation is often em- pared with the  minimization (4). In particular, the ployed: case of q = 1/2 has been treated extensively in [37– 39]. Other notable examples of nonconvex objective min z subject to y = Az, (8) functions include smoothly clipped absolute devia- 2 1 z∈R tion (SCAD) [40] and the minimax concave plus function [41], among others. where the mixed  / norm is defined as 2 1 Compressive sensing with general z = z[i ] . 2 1 2 dictionaries i =1 Thus far, we have focused on sparsity with respect It is not hard to see that, when each block has to the canonical basis ofR . In many applications, it size n = 1, (8) reduces to the  minimization given might be more appropriate to have sparsity with re- i 1 by (4). More generally, the optimization problem in spect to more general dictionaries; see e.g. [42–46]. (8) is convex and can be recast as a second-order More specifically, a signal x is represented by x = Dα n×n cone program, and thus can be solved efficiently. with respect to a dictionary D ∈ R , where α ∈ One can also extend the notion of RIP and ROP to R is the coordinate in the dictionary and is known the block-sparse setting; see e.g. [ 32,33]. For brevity, a priori to be sparse or nearly sparse. One can ob- write I ={n ,..., n }. viously treat A = AD as the new sensing matrix 1 b Definition 2.2 . A sensing matrix A is said to sat- and apply any of the aforementioned methods to ex- isfy the block-RIP of order k if there exists a constant ploit the sparsity of α. One of the drawbacks, how- δ ∈ [0, 1) such that, for every block k-sparse vec- ever, is that nice properties of the original sensing k|I Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 759 matrix A may not be inherited by A . In other words, fact and distinguish from the vector case treated in even if one carefully designs a sensing matrix A,ex- the last section, we shall write the underlying signal n ×n 1 2 act recovery of x may not be guaranteed despite its as a capital letter, X ∈ R , throughout this sec- sparsity with respect to D. Alternatively, some have tion. In these applications, we often observe a small advocated reconstructing x by the solution to the fol- fraction of the entries of X. The task of matrix com- lowing optimization problem: pletion is then to ‘complete’ the remaining entries. Formally, let  be a subset of [n ] × [n ] where 1 2 min D z subject to y = Az; (12) 1 [n] = {1, ..., n}. The goal of matrix completion is z∈R to recover X based on {X :(i, j) ∈ }, particularly ij see e.g. [43,44,47,48]. with the sample size || much smaller than the total number n n of entries. To fix ideas, we shall assume 1 2 that  is a uniformly sampled subset of [n ] × [n ], 1 2 Compressive phase retrieval although other sampling schemes have also been in- vestigated in the literature, e.g. [53–55]. In the previous subsections, we have mainly dis- Obviously, we cannot complete an arbitrary ma- cussed the problem of recovering a sparse signal trix from a subset of its entries. But it is possible from a small number of linear measurements. How- for low-rank matrices as their degrees of freedom ever, in some practical scenarios one can only ob- are much smaller than n n . But low rankness alone 1 2 serve some nonlinear measurements of the original is not sufficient. Consider a matrix with a single signal. The typical example is the so-called compres- nonzero entry; it is of rank one but it is impossible sive phase retrieval problem. In such a scenario, we ob- to complete it unless the nonzero entry is observed. serve the magnitude of the Fourier coefficients in- A formal way to characterize low-rank matrices that stead of their phase, which can be modeled as the can be completed from {X :(i, j) ∈ } was first in- ij form troduced in [14]. Definition 3.1 .Let U be a subspace of R of di- y =|a , x|, k = 1, ··· , m. (13) k k mension r and P be the orthogonal projection For recovering x, several studies [49–51]havecon- onto U. Then the coherence of U (with respect to the sidered the following  minimization: 1 standard basis (e )) is defined to be μ(U ) ≡ max P e  . min z subject to y =|a , z|, U i 1 k k n r 1≤i ≤n z∈R It is clear that the smallest possible value for k = 1, ··· , m. (14) μ(U) is 1 and the largest possible value for μ(U) is n/r.Let M be an n × n matrix of rank r and 1 2 By introducing a strong notion of RIP, Voroninski with column and row spaces denoted by U and V, and Xu [51] built a parallel result for compressive respectively. We shall say that M satisfies the inco- phase retrieval with the classical compressive sens- herence condition with parameter μ if max (μ(U), ing. Specifically, they proved that a k-sparse signal x μ(V)) ≤ μ . can be recovered from m = O(k log (n/k)) random Now let X be an incoherent matrix of rank r.Ina Gaussian phaseless measurements by solving (14). similar spirit as the vector case, a natural way to re- Unlike the standard convex  minimization (4), the construct it from {X :(i, j) ∈ } is to seek, among ij problem (14) is a nonconvex problem, but some ef- all matrices whose entries indexed by  agree with ficient algorithms have been developed to compute our observations, the one with the smallest rank: it; see e.g. [50,52]. min rank(Z)subjectto Z = X , ij ij n ×n 1 2 Z ∈R RECOVERY OF LOW-RANK MATRICES ∀(i , j ) ∈ . (15) We now consider recovering a low-rank matrix, a problem often referred to as matrix completion. Again, to overcome the computational challenges in directly minimizing matrix ranks, the following con- vex program is commonly suggested: Matrix completion via nuclear norm min Z  subject to Z = X , (i , j ) ∈ , minimization ∗ ij ij n ×n 1 2 Z ∈R In many practical situations such as collaborative fil- (16) tering, system identification and remote sensing, to where the nuclear norm · is the sum of all name a few, the signal that we aim to recover often- singular values. As before, we are interested in times is a matrix rather than a vector. To signify this when the solution to (16) is unique and correctly Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 760 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW recovers x. Candes and Recht [14] were the first to Mixed sparsity show that this is indeed the case, for almost all  that In some applications, the matrix that we want to re- are large enough. These results are probabilistic in cover is not necessarily of low rank but differs from nature due to the randomness of , that is, one can a low-rank matrix only by a small number of en- correctly recover x using (16) with high probability tries; see e.g. [17,18]. In other words, we may write regarding min {n , n }. The sample size requirement 1 2 X = S + L, where S is a sparse matrix with only a to ensure exact recovery of X by the solution of (16) small number of nonzero entries while L is a ma- was later improved in [16]to ||≥ Cr(n + n ) · 1 2 trix of low rank. It is clear that, even if we observe X polylog(n + n ), where C is a constant that de- 1 2 entirely, the decomposition of X into a sparse com- pends on the coherence coefficients only. It is clear ponent S and a low-rank component L may not be to see that this requirement is (nearly) optimal in uniquely defined. General conditions under which that there are O(r(n + n )) free parameters in spec- 1 2 such a decomposition is indeed unique are provided ifying a rank-r matrix. in, for example, [18]. In the light of the previous dis- cussions, it is natural to consider reconstructing X from observations {X :(i, j) ∈ } by the solution ij Matrix completion from affine to measurements More generally, one may consider recovering a low- min Z  + λvec(Z ) rank matrix based on affine measurements. More 1 ∗ 2 n ×n 1 2 Z ,Z ∈R 1 2 n ×n m 1 2 specifically, let A : R → R be a linear map subject to (Z + Z ) = X , ∀(i , j ) ∈ . 1 2 ij ij such that A(X) = y. We aim to recover X based on the information that A(X) = y.Itisclear (19) that the canonical matrix completion problem dis- cussed in the previous subsection corresponds to the case when A(X) ={X :(i , j ) ∈ }. Similarly, ij This strategy has been investigated extensively in we can proceed to reconstruct X by the solution to the literature; see e.g. [17]. Further developments in this direction can also be found in [61–64], among others. min Z  subject to y = A(Z). n ×n 1 2 Z ∈R (17) Nonconvex methods It is of interest to know under which kind of sensing operator A can X be recovered in exactly this way. Just as  norm is a convex relaxation of the  norm, 1 0 An answer is given in [56], which extends the con- the nuclear norm is also a convex relaxation of the cept of RIP to general linear operator for matrices. rank for a matrix. In addition to these convex ap- n ×n 1 2 Definition 3.2 . A linear operator A : R → proaches, nonconvex methods have also been pro- R is said to satisfy the matrix RIP of order r if there posed by numerous authors. The most common ex- exists a constant δ such that ample is the Schatten- q (0 < q < 1) (quasi-)norm defined by Z Z 1 − δ Z  ≤A(Z) ≤ 1 + δ Z F  F r 2 r 1/q (18) min{n ,n } 1 2 n ×n 1 2 holds for all matrices Z ∈ R of rank at most r. X  = σ , (20) q i Recht et al. [56] further proved that, if A i =1 satisfies the matrix RIP (18) with δ < 1/10, 5r then one can recover a rank-r matrix X from where σ , σ , ··· are the singular values of X.Itis 1 2 n = O(r(n + n ) log (n n )) measurements by the 1 2 1 2 clear that, when q → 0, X → rank(X)while the sq solution to (17). Note that the condition δ < 5r nuclear norm corresponds to the case when q = 1. 1/10 has been dramatically weakened; see e.g. One may now consider reconstructing X by the so- [28,57]. lution to Besides the aforementioned low-rank matrix completion problems, some recent studies, e.g. [58– 60], have drawn attention to the so-called high-rank min Z  subject to y = A(Z); matrix completion, in which the columns of the ma- n ×n 1 2 Z ∈R trix belong to a union of subspaces and, as a result, (21) the rank can be high or even full. see e.g. [65–68], among others. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 761 matrix nuclear norm, computing the tensor nuclear RECOVERY OF LOW-RANK norm, and thereby the problem (23), is NP-hard. HIGHER-ORDER TENSORS Hence, various relaxations and approximate algo- In an increasing number of modern applications, rithms have been introduced in the literature; see e.g. the object to be estimated has a higher-order tensor [75–79], and references therein. This is a research structure. Typical examples include video inpaint- area in its infancy and many interesting issues need ing [69], scan completion [70], multichannel EEG to be addressed. (electroencephalogram) compression [71], traffic data analysis [72] and hyperspectral image restora- tion [73], among many others. Similar to matrices, APPLICATIONS in many of these applications, we are interested in In the previous sections, we have given an overview recovering a low-rank tensor either from observing a subset of its entries or a collection of affine mea- of some basic ideas and techniques in dealing with surements. sparsity in vectors and low rankness in matrices and Despite the apparent similarities between matri- tensors. We now give a couple of examples to illus- ces and higher-order tensors, it is delicate to extend trate how they can be used in action. the idea behind nuclear norm minimization to the latter because matrix-style singular value decompo- sition does not exist for higher-order tensors. A com- Background subtraction with mon approach is to first unfold a higher-order tensor compressive imaging to a matrix and then apply a matrix-based approach Background subtraction in image and video has to recover a tensor. Consider, for example, a third- attracted a lot of attention in the past couple of d ×d ×d 1 2 3 order tensorX ∈ R . We can collapse its sec- decades. It aims at simultaneously separating video ond and third indices, leading to a d × (d d )ma- 1 2 3 background and extracting the moving objects from trix X .If X is of low rank, then so is X .Wecan (1) (1) a video stream, and can provide important clues for exploit the low rankness of X by minimizing its nu- (1) various applications such as moving object detection clear norm. Clearly we can also collapse the first and [80] and object tracking in surveillance [81], among third indices of X , leading to a matrix X , and the (2) numerous others. first and second, leading to X . If we want to com- (3) Conventional background subtraction tech- plete a low-rank tensor X based on its entries X (ω) niques usually consist of four steps: video acqui- for ω ∈ ⊂[d ] × [d ] × [d ], we can then con- 1 2 3 sition, encoding, decoding and separating the sider recovering X by the solution to the following moving objects from the background. This scheme convex program: needs to fully sample the video frames with large computational and storage requirements, followed by well-designed video coding and background min Z ( j ) ∗ d ×d ×d 1 2 3 Z∈R subtraction algorithms. To alleviate the burden of j =1 computation and storage, a newly-developed com- subject to Z(ω) = X (ω), ∀ω ∈ . (22) pressive imaging scheme [82–84] has been used for background subtraction by combining the video Efficient algorithms for solving matrix nuclear norm acquisition, coding and background subtraction minimization (16) such as the alternating direc- into a single framework, as illustrated in Fig. 1.It tion method of multipliers (ADMM) and Douglas– simultaneously achieves background subtraction Rachford operator splitting methods can then be and video reconstruction. In this setting, the main readily adapted to solve (22); see e.g. [3,4]. objective is then to maximize the reconstruction As pointed out in [5], however, such an approach and separation accuracies using as few compressive fails to exploit fully the multilinear structure of a ten- measurements as possible. sor and is thus suboptimal. Instead, directly minimiz- Several studies have been carried out for back- ing the tensor nuclear norm was suggested: ground subtraction from the perspective of com- pressive imaging. In the seminal work of Cevher et al. min Z d ×d ×d 1 2 3 Z∈R [85], the background subtraction problem is formu- lated as a sparse recovery problem. They showed that subject to Z(ω) = X (ω), ∀ω ∈ , (23) the moving objects can be recovered by learning a where the tensor nuclear norm · is defined as low-dimensional compressed representation of the the dual norm of the tensor spectral norm ·.For background image. More recently, the robust princi- more discussions on the tensor nuclear and spec- ple component analysis (RPCA) approach has also tral norms, please see [74]. Unfortunately, unlike the been used to deal with the problem of background Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 762 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW Compressive Reconstruction with foreground- measurements background separation form Transmission Figure 1. The framework of background subtraction with compressive imaging. subtraction with compressive imaging, in which they 3D-TV term · is defined as 3D-TV commonly model the video as a matrix with columns X  : =X (i , j, k) of vectorized video frames and then decompose the 3D-TV h 1 matrix into a low-rank matrix L and a sparse matrix +X (i , j, k) +X (i , j, k) , v 1 t 1 S; see e.g. [86–89]. Although methods based on RPCA have where achieved satisfactory performance, they fail to exploit the finer structures of background and fore- X (i , j, k): = X (i , j + 1, k) − X (i , j, k), ground after vectorizing the video frames. It appears X (i , j, k): = X (i + 1, j, k) − X (i , j, k), more advantageous to model the spatio-temporal information of background and foreground using X (i , j, k): = X (i , j, k + 1) − X (i , j, k). direct tensor representation of a video. To this end, a novel tensor RPCA approach has been Because a 3D patch in a video background is simi- proposed in [90] for background subtraction lar to many other 3D patches over the video frames, from compressive measurements by decomposing one can model the video background using several the video into a static background with spatio- groups of similar video 3D patches, where each patch temporal correlation and a moving foreground group corresponds to a fourth-order tensor. Inte- with spatio-temporal continuity within a tensor grating the patch-based modeling idea into (24), representation framework. More specifically, one one can easily get a patch-group-based tensor RPCA can use 3D total variation (3D-TV) to characterize model. It should be noted that solving the noncon- the spatio-temporal continuity underlying the video vex tensor RPCA model (24) as well as its patch- foreground, and low-rank Tucker decomposition to group-based form are computationally difficult. In model the spatio-temporal correlation of the video practice, we can find a local solution using a multi- background, which leads to the following tensor block version of ADMM. For more details, please re- RPCA model: fer to Section V of [90]. Fig. 2 gives an example based on three real videos. min λS + E  It is evident that the proposed tensor models enjoy 3D-TV X , S, E , a superior performance over the other popular ma- G, U , U , U 1 2 3 trix models both in terms of the quality of the recon- structed videos and in terms of the separation of the subject to X = L + E + S, moving objects. This suggests that using a direct ten- L = G × U × U × U , sor modeling technique to deal with practical higher- 1 1 2 2 3 3 order tensor data can utilize more useful structures. y = A(X ), (24) where the factor matrices U and U are orthogonal 1 2 Hyperspectral compressive sensing in columns for two spatial modes, the factor matrix U is orthogonal in columns for the temporal mode, Hyperspectral imaging employs an imaging spec- the core tensor G interacts with these factors and the trometer to collect hundreds of spectral bands Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 763 Ground Truth SparCS ReProCS SpLR H-TenRPCA PG-TenRPCA (a) (b) (c) Figure 2. Visual comparison of two tensor models (i.e. H-TenRPCA and PG-TenRPCA) proposed in [90] and three popular matrix models (i.e. SparCS [86], ReProcs [87] and SpLR [88]) under the sampling ratio 1/30. The first column shows the original video frames from different video volumes (a)–(c); the second to sixth columns correspond to the results produced by all the compared methods, respectively. Here, for each method, the reconstruction result of the original video frame (upper panels) and the detection result of moving objects in the foreground (lower panels) is shown. ranging from ultraviolet to infrared wavelengths for tensively investigated. Similar to other compressive the same area on the surface of the Earth. It has a sensing problems, the main objectives of HCS are wide range of applications including environmen- to design easy hardware encoding implementation tal monitoring, military surveillance and mineral ex- and develop an efficient sparse reconstruction pro- ploration, among numerous others [91,92]. Figura- cedure. In what follows we shall focus primarily tively speaking, a hyperspectral image can be treated on the latter. For hardware implementation, inter- as a 3D (x, y, λ) data cube, where x and y represent ested readers are referred to [93,94] and references two spatial dimensions of the scene, and λ represents therein. the spectral dimension comprising a range of wave- Like other natural images, hyperspectral images lengths. Typically, such hyperspectral cubes are col- can be sparsified by using certain transformations. lected by an airborne sensor or a satellite and sent Traditional sparse recovery methods such as  min- to a ground station on Earth for subsequent process- imization and TV minimization are often used for ing. Noting that the dimension of λ is usually in the such purposes; see e.g. [95,96]. To further exploit hundreds, hyperspectral cubes are of a fairly large the inherent spectral correlation of hyperspectral im- size even for moderate dimensions of x and y.This ages, a series of work based on low-rank modeling makes it necessary to devise effective techniques for has been carried out in recent years. For example, hyperspectral data compression, due to the limited Golbabaee and Vandergheynst proposed in [97]a bandwidth of the link connection between the satel- joint nuclear and  / minimization method to de- 2 1 lite/aerospace and the ground station. scribe the spectral correlation and the joint-sparse In the last few years, a popular hyperspectral com- spatial wavelet representations of hyperspectral im- pressive sensing (HCS) scheme based on the prin- ages. Then, they modeled the spectral correlation ciple of compressive sensing for hyperspectral data together with the spatial piecewise smoothness of compression, as illustrated in Fig. 3, has been ex- hyperspectral images by using a joint nuclear and Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 764 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW Satellites, aerospace Transmit measurements Encoding by compressive sensing Receiving Reconstruction via sparse modeling Ground station Figure 3. The framework of hyperspectral compressive sensing. TV norm minimization method [98]. Similar to the surveillance videos, however, modeling a hyperspec- tral cube as a matrix cannot utilize the finer spatial- and-spectral information, leading to suboptimal re- construction results under relatively low sampling ratios (e.g. 1% of the whole image size). To further exploit the compressibility underlying a hyperspectral cube, one may consider such a sin- gle cube as a tensor with three modes (width, height and band) and then identify the hidden spatial- and-spectral structures using direct tensor modeling techniques. Precisely, all the bands of a hyperspec- tral image have very strong correlation in the spec- tral domain, and each band, if considered as a ma- trix, has relatively strong correlation in the spatial domain; such spatial-and-spectral correlation can be modeled through low-rank Tucker decomposition. In addition, the intensity at each voxel is likely to be similar to its neighbors, which can be characterized by smoothness using the so-called 3D-TV penalty. In summary, [99] considered the following joint ten- sor Tucker decomposition and 3D-TV minimiza- tion model: min λX  + E 3D-TV X , E , G, U , U , U 1 2 3 subject to X = G × U × U × U + E , 1 1 2 2 3 3 y = A(X ). (25) It is clear that the above minimization problem is highly nonconvex. One often looks for good local so- lutions using a multi-block ADMM algorithm. Figure 4. Vision comparison of the tensor method (25) over three other competing Fig. 4 gives an example of the first band of four hy- methods on the rst fi band of four different hyperspectral datasets (a)–(d). Here the perspectral datasets, respectively reconstructed by sampling ratio is 1%. The last column shows the original image bands. The columns different methods, with the sampling ratio at 1%. It from the first to the fourth correspond to the produced results from the KCS method [96], the JNTV method [98], the ST-NCS method [100] and the tensor method (25), is evident that the proposed tensor method could respectively. provide nearly perfect reconstruction. In addition, Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 765 sparse tensor and nonlinear compressive sensing 13. Candes ` E, Romberg J and Tao T. Stable signal recovery from (ST-NCS) performs slightly better than Kronecker incomplete and inaccurate measurements. Comm Pure Appl compressive sensing (KCS) and joint nuclear/TV Math 2006; 59: 1207–23. norm minimization (JNTV) in terms of reconstruc- 14. Candes ` E and Recht B. Exact matrix completion via convex op- tion accuracy, because of the use of a direct ten- timization. Found Comput Math 2009; 9: 717–72. sor sparse representation of a hyperspectral cube. 15. Candes ` E and Tao T. The power of convex relaxation: near- Both these findings demonstrate the power of ten- optimal matrix completion. IEEE Trans Inform Theor 2010; 56: sor modeling techniques. It is also worth noting that, 2053–80. compared with ST-NCS, the images reconstructed 16. Gross D. Recovering low-rank matrices from few coefficients with the method (25) are clearer and sharper. in any basis. IEEE Trans Inform Theor 2011; 57: 1548–66. 17. Candes ` E, Li X and Ma Y et al. Robust principal component analysis? JACM 2011; 58: 1–39. ACKNOWLEDGEMENT 18. Chandrasekaran V, Sanghavi S and Parrilo P et al. Rank- The authors thank the associate editor and three referees for help- sparsity incoherence for matrix decomposition. SIAM J Optim ful comments. 2011; 21: 572–96. 19. Natarajan B. Sparse approximate solutions to linear systems. SIAM J Comput 1995; 24: 227–34. FUNDING 20. Donoho D, Elad M and Temlyakov VN. Stable recovery of This work was supported in part by the National Natural Sci- sparse overcomplete representations in the presence of noise. ence Foundation of China (11501440 and 61273020 to IEEE Trans Inform Theor 2006; 52: 6–18. Y.W., 61373114, 61661166011 and 61721002 to D.Y.M.), 21. Cohen A, Dahmen W and DeVore R. Compressed sensing the National Basic Research Program of China (973 Program) and best k-term approximation. J Am Math Soc 2009; 22: (2013CB329404 to D.Y.M.) and the National Science Founda- 211–31. tion (DMS-1265202 to M.Y.). 22. Eldar Y and Kutyniok G. Compressed Sensing: Theory and Ap- plications. Cambridge: Cambridge University Press, 2012. REFERENCES 23. Foucart S and Rauhut H. A Mathematical Introduction to Com- 1. Liu J, Musialski P and Wonka P et al. Tensor completion for pressive Sensing. Berlin: Springer, 2013. estimating missing values in visual data. In: Proceedings of In- 24. Cai TT, Wang L and Xu G. New bounds for restricted isometry ternational Conference on Computer Vision, 2009. constants. IEEE Trans Inform Theor 2010; 56: 4388–94. 2. Tomioka R, Hayashi K and Kashima H. Estimation of low-rank 25. Cai TT and Zhang A. Compressed sensing and affine rank min- tensors via convex optimization. arXiv:10100789. imization under restricted isometry. IEEE Trans Signal Process 3. Gandy S, Recht B and Yamada I. Tensor completion and low- 2013; 61: 3279–90. n-rank tensor recovery via convex optimization. Inverse Probl 26. Candes ` E. The restricted isometry property and its implications 2011; 27: 025010. for compressed sensing. Compt Rendus Math 2008; 346: 589– 4. Liu J, Musialski P and Wonka P. Tensor completion for esti- mating missing values in visual data. IEEE Trans Pattern Anal 27. Cai TT, Wang L and Xu G. Shifting inequality and recovery of Mach Intell 2013; 34: 208–20. sparse signals. IEEE Trans Signal Process 2010; 58: 1300–8. 5. Yuan M and Zhang CH. On tensor completion via nuclear norm 28. Cai TT and Zhang A. Sparse representation of a polytope and minimization. Found Comput Math 2016; 16: 1031–68. recovery of sparse signals and low-rank matrices. IEEE Trans 6. Chen SS, Donoho DL and Saunders MA. Atomic decomposition Inform Theor 2014; 60: 122–32. by basis pursuit. SIAM J Sci Comput 1998; 20: 33–61. 29. DeVore R. Deterministic constructions of compressed sensing 7. Donoho DL and Huo X. Uncertainty principles and ideal atomic matrices. J Complex 2007; 23: 918–25. decomposition. IEEE Trans Inform Theor 2001; 47: 2845–62. 30. Bourgain J, Dilworth SJ and Ford K et al. Explicit constructions 8. Donoho DL and Elad M. Optimally sparse representation in of RIP matrices and related problems. Duke Math J 2011; 159: general (nonorthogonal) dictionaries via  minimization. Proc 145–85. Natl Acad Sci 2003; 100: 2197–202. 31. Xu Z. Deterministic sampling of sparse trigonometric polyno- 9. Candes ` E and Tao T. Decoding by linear programming. IEEE mials. J Complex 2011; 27: 133–40. Trans Inform Theor 2005; 51: 4203–15. 32. Eldar Y and Mishali M. Robust recovery of signals from a struc- 10. Candes ` E, Romberg J and Tao T. Robust uncertainty principles: tured union of subspaces. IEEE Trans Inform Theor 2009; 55: exact vector reconstruction from highly incomplete frequency 5302–16. information. IEEE Trans Inform Theor 2006; 52: 489–509. 33. Wang Y, Wang J and Xu Z. On recovery of block-sparse signals 11. Donoho D. Compressed sensing. IEEE Trans Inform Theor 2006; via mixed  / (0 < q ≤ 1) norm minimization. EURASIP J Adv 2 q 52: 1289–306. Signal Process 2013; 76: 1–17. 12. Candes ` E and Tao T. Near-optimal signal recovery from random 34. Chartrand R. Exact reconstruction of sparse signals via non- projections: universal encoding strategies. IEEE Trans Inform convex minimization. IEEE Signal Process Lett 2007; 14: Theor 2006; 52: 5406–25. 707–10. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 766 Natl Sci Rev, 2018, Vol. 5, No. 5 REVIEW 35. Sun Q. Recovery of sparsest signals via  -minimization.ApplComputHarmon 59. Elhamifar E. High-rank matrix completion and clustering under self-expressive Anal 2012; 32: 329–41. models. In: Proceedings of Advances in Neural Information Processing Sys- 36. Song CB and Xia ST. Sparse signal recovery by  minimization under re- tems, 2016. stricted isometry property. IEEE Signal Process Lett 2014; 21: 1154–8. 60. Li CG and Vidal R. A structured sparse plus structured low-rank framework 37. Xu Z, Zhang H and Wang Y et al.  regularization. Sci China Inform Sci for subspace clustering and completion. IEEE Trans Signal Process 2016; 64: 1/2 2010; 53: 1159–69. 6557–70. 38. Xu Z, Chang X and Xu F et al.  regularization: a thresholding representa- 61. Zhou ZH, Li X and Wright J et al. Stable principal component pursuit. In: Pro- 1/2 tion theory and a fast solver. IEEE Trans Neural Network Learn Syst 2012; 23: ceedings of the 2010 IEEE International Symposium on Information Theory, 1013–27. 2010. 39. Zeng J, Lin S and Wang Y et al.  regularization: convergence of it- 62. Ganesh A, Wright J and Li Xetal. Dense error correction for low-rank matrices 1/2 erative half thresholding algorithm. IEEE Trans Signal Process 2014; 62: via principal component pursuit. In: Proceedings of the 2010 IEEE International 2317–29. Symposium on Information Theory, 2010. 40. Fan J and Li R. Variable selection via nonconcave penalized likelihood and its 63. Zhao Q, Meng D and Xu Z et al. Robust principal component analysis with oracle properties. J Am Stat Assoc 2001; 96: 1348–60. complex noise. In: Proceedings of the 31st International Conference on Ma- 41. Zhang CH. Nearly unbiased variable selection under minimax concave penalty. chine Learning, 2014. Ann Stat 2010; 38: 894–942. 64. Netrapalli P, Niranjan U and Sanghavi S et al. Non-convex robust PCA. 42. Rauhut H, Schnass K and Vandergheynst P. Compressed sensing and redun- In: Proceedings of Advances in Neural Information Processing Systems 27, dant dictionaries. IEEE Trans Inform Theor 2013; 29: 1401–12. 2014. 43. Candes ` E, Eldar Y and Needell D et al. Compressed sensing with coherent and 65. Zhang M, Huang ZH and Zhang Y. Restricted p-isometry properties of noncon- redundant dictionaries. Appl Comput Harmon Anal 2010; 31: 59–73. vex matrix recovery. IEEE Trans Inform Theor 2013; 59: 4316–23. 44. Elad M, Milanfar P and Rubinstein R. Analysis versus synthesis in signal pri- 66. Wang J, Wang M and Hu X et al. Visual data denoising with a unified ors. Appl Comput Harmon Anal 2007; 23: 947–68. Schatten-p norm and  norm regularized principal component pursuit. Pat- 45. Lin J, Li S and Shen Y. New bounds for restricted isometry constants with tern Recogn 2015; 48: 3135–44. coherent tight frames. IEEE Trans Signal Process 2013; 61: 611–21. 67. Zhao Q, Meng D and Xu Z et al.  -norm low-rank matrix factorization by 46. Lin J and Li S. Sparse recovery with coherent tight frames via analydsis variational Bayesian method. IEEE Trans Neural Network Learn Syst 2015; 26: Dantzig selector and analysis LASSO. Appl Comput Harmon Anal 2014; 37: 825–39. 126–39. 68. Yue MC and So AMC. A perturbation inequality for concave functions of sin- 47. Liu Y, Mi T and Li S. Compressed sensing with general frames via optimal- gular values and its applications in low-rank matrix recovery. Appl Comput dual-based  -analysis. IEEE Trans Inform Theor 2012; 58: 4201–14. Harmon Anal 2016; 40: 396–416. 48. Li S and Lin J. Compressed sensing with coherent tight frames via  - 69. Korah T and Rasmussen C. Spatio-temporal inpainting for recovering texture minimization for 0 < q ≤ 1. Inverse Probl Imag 2014; 8: 761–77. maps of occluded building facades. IEEE Trans Image Process 2007; 16: 2262– 49. Moravec M, Romberg J and Baraniuk R. Compressive phase retrieval. In: Pro- 71. ceedings of SPIE, the International Society for Optics and Photonics, 2007. 70. Pauly M, Mitra N and Giesen J et al. Example-based 3D scan completion. In: 50. Yang Z, Zhang C and Xie L. Robust compressive phase retrieval via  mini- Proceedings of the Symposium on Geometry Processing, 2005. mization with application to image reconstruction. arXiv:13020081. 71. Acar E, Dunlavy D and Kolda T et al. Scalable tensor factorizations for incom- 51. Voroninski V and Xu Z. A strong restricted isometry property, with an applica- plete data. Chemometr Intell Lab Syst 2001; 106: 41–56. tion to phaseless compressed sensing. Appl Comput Harmon Anal 2016; 40: 72. Xie K, Wang L and X Wetal. Accurate recovery of internet traffic data: a tensor 386–95. completion approach. In: Proceedings of the 35th Annual IEEE International 52. Schniter P and Rangan S. Compressive phase retrieval via general- Conference on Computer Communications, 2016. ized approximate message passing. IEEE Trans Signal Process 2015; 63: 73. Peng Y, Meng D and Xu Z et al. Decomposable nonlocal tensor dictionary 1043–55. learning for multispectral image denoising. In:ProceedingsofIEEEConference 53. Foygel R, Shamir O and Srebro N et al. Learning with the weighted trace-norm on Computer Vision and Pattern Recognition, 2014. under arbitrary sampling distributions. In: Proceedings of Advances in Neural 74. Hillar CJ and Lim LH. Most tensor problems are NP-hard. JACM 2013; 60: Information Processing Systems 24, 2011. 1–39. 54. Chen Y, Bhojanapalli S and Sanghavi S et al. Coherent matrix completion. In: 75. Nie J and Wang L. Semidefinite relaxations for best rank-1 tensor approxima- Proceedings of the 31st International Conference on Machine Learning, 2014. tions. SIAM J Matrix Anal Appl 2014; 35: 1155–79. 55. Cai TT and Zhou WX. Matrix completion via max-norm constrained optimiza- 76. Jiang B, Ma S and Zhang S. Tensor principal component analysis via convex tion. Electron J Stat 2016; 10: 1493–525. optimization. Math Program 2015; 150: 423–57. 56. Recht B, Fazel M and Parrilo P. Guaranteed minimum-rank solutions of linear 77. Yang Y, Feng Y and Suykens J. A rank-one tensor updating algorithm for tensor matrix equations via nuclear norm minimization.SIAMRev 2010; 52: 471–501. completion. IEEE Signal Process Lett 2015; 22: 1633–7. 57. Candes ` E and Plan Y. Tight oracle inequalities for low-rank matrix recovery 78. Zhao Q, Meng D and Kong X et al. A novel sparsity measure for tensor from a minimal number of noisy random measurements. IEEE Trans Inform recovery. In: Proceedings of International Conference on Computer Vision, Theor 2011; 57: 2342–59. 2015. 58. Eriksson B, Balzano L and Nowak R. High-rank matrix completion. In: Proceed- 79. Xie Q, Zhao Q and Meng Detal. Kronecker-Basis-Representation based tensor ings of the 15th International Conference on Artificial Intelligence and Statis- sparsity and its Applications to tensor recovery. IEEE Trans Pattern Anal Mach tics, 2012. Intell 2017; 40: 1888–902. Downloaded from https://academic.oup.com/nsr/article/5/5/756/3966714 by DeepDyve user on 20 July 2022 REVIEW Wang et al. 767 80. Wang T, Backhouse A and Gu I. Online subspace learning on Grassmann man- 90. Cao W, Wang Y and Sun J et al. Total variation regularized tensor RPCA for ifold for moving object tracking in video. In: Proceedings of IEEE International background subtraction from compressive measurements. IEEE Trans Image Conference on Acoustics, Speech, and Signal Process, 2008. Process 2016; 25: 4075–90. 81. Beleznai C, Fruhstuck B and Bischof H. Multiple object tracking using local 91. Goetz AFH. Three decades of hyperspectral remote sensing of the Earth: a PCA. In: Proceedings of the 18th International Conference on Pattern Recog- personal view. Rem Sens Environ 2009; 113: S5–S6. nition, 2006. 92. Willett R, Duarte M and Davenport M et al. Sparsity and structure in hyper- 82. Wakin M, Laska JN and Duarte MF et al. Compressive imaging for video spectral imaging: sensing, reconstruction, and target detection. IEEE Signal representation and coding. In: Proceedings of Picture Coding Symposium, Process Mag 2014; 31: 116–26. 2006. 93. Arce G, Brady D and Carin L et al. Compressive coded aperture spectral imag- 83. Takhar D, Laska JN and Wakin M et al. A new compressive imaging camera ing: an introduction. IEEE Signal Process Mag 2014; 31: 105–15. architecture using optical-domain compression. In: Proceedings of Computa- 94. Yuan X, Tsai TH and Zhu R et al. Compressive hyperspectral imaging with side tional Imaging IV at SPIE Electronic Imaging, 2006. information. IEEE J Sel Top Signal Process 2015; 9: 964–76. 84. Duarte M, Davenport M and Takhar D et al. Single-pixel imaging via compres- 95. Li C, Sun T and Kelly KF et al. A compressive sensing and unmixing scheme for sive sampling. IEEE Signal Process Mag 2008; 25: 83–91. hyperspectral data processing. IEEE Trans Image Process 2012; 21: 1200–10. 85. Cevher V, Sankaranarayanan A and Duarte M et al. Compressive sensing for 96. Duarte M and Baraniuk R. Kronecker compressive sensing. IEEE Trans Image background subtraction. In: Proceedings of the 10th European Conference on Process 2012; 21: 494–504. Computer Vision, 2008. 97. Golbabaee M and Vandergheynst P. Hyperspectral image compressed sensing 86. Waters A, Sankaranarayanan A and Baraniuk R et al. A new compres- via low-rank and joint-sparse matrix recovery. In: Proceedings of IEEE Interna- sive imaging camera architecture using optical-domain compression. In: tional Conference on Acoustics, Speech and Signal Processing, 2012. Proceedings of Conference on Neural Information Processing Systems 24, 98. Golbabaee M and Vandergheynst P. Joint trace/TV norm minimization: a new 2011. efficient approach for spectral compressive imaging. In: Proceedings of 19th 87. Guo H, Qiu CL and Vaswani N. An online algorithm for separating sparse and IEEE International Conference on Image Processing, 2012. low-dimensional signal sequences from their sum. IEEE Trans Signal Process 99. Wang Y, Lin L and Zhao Q et al. Compressive sensing of hyperspectral images 2014; 62: 4284–97. via joint tensor tucker decomposition and weighted total variation regulariza- 88. Jiang H, Deng W and Shen Z. Surveillance video processing using compres- tion. IEEE Geosci Rem Sens Lett 2017; 14: 2457–61. sive sensing. Inverse Probl Imag 2014; 6: 201–14. 100. Yang S, Wang M and Li P et al. Compressive hyperspectral imaging via sparse 89. Jiang H, Zhao S and Shen Z et al. Surveillance video analysis using compres- tensor and nonlinear compressed sensing. IEEE Trans Geosci Rem Sens 2015; sive sensing with low latency. Bell Labs Tech J 2014; 18: 63–74. 53: 5943–57.

Journal

National Science ReviewOxford University Press

Published: Sep 1, 2018

There are no references for this article.