ISSN 0032-9460, Problems of Information Transmission, 2015, Vol. 51, No. 4, pp. 335–348.
Pleiades Publishing, Inc., 2015.
Original Russian Text
O. Klopp, A.B. Tsybakov, 2015, published in Problemy Peredachi Informatsii, 2015, Vol. 51, No. 4, pp. 32–46.
METHODS OF SIGNAL PROCESSING
Estimation of Matrices with Row Sparsity
CREST and MODAL’X, University Paris Ouest, Nanterre, France
UMR CNRS 9194, ENSAE, Malakoﬀ, France
Received March 15, 2015; in ﬁnal form, September 8, 2015
Abstract—An increasing number of applications is concerned with recovering a sparse matrix
from noisy observations. In this paper, we consider the setting where each row of an unknown
matrix is sparse. We establish minimax optimal rates of convergence for estimating matrices
with row sparsity. A major focus in the present paper is on the derivation of lower bounds.
In recent years, there has been a great interest to the theory of estimation in high-dimensional
statistical models under diﬀerent sparsity scenarii. The main motivation behind sparse estima-
tion is based on the observation that, in several practical applications, the number of variables is
much larger than the number of observations, but the degree of freedom of the underlying model is
relatively small. One example of such sparse estimation is the problem of estimating of a sparse re-
gression vector from a set of linear measurements (see, e.g., [1–4]). Another example is the problem
of matrix recovery under the assumption that the unknown matrix has low rank (see, e.g., [5–8]).
In some recent papers dealing with covariance matrix estimation, a diﬀerent notion of sparsity
was considered (see, for example, [9,10]). This notion is based on sparsity assumptions on the rows
(or columns) M
of matrix M. One can consider the hard sparsity assumption meaning that each
of M contains at most s nonzero elements, or soft sparsity assumption, based on imposing
a certain decay rate on ordered entries of M
. These notions of sparsity can be deﬁned in terms of
-balls for q ∈ [0, 2), deﬁned as
) ∈ R
where s<∞ is a given constant. The case q =0
) ∈ R
corresponds to the set of vectors v with at most s nonzero elements. Here I(·) denotes the indicator
function and s ≥ 1 is an integer.
Conducted as a part of the project Labex MME-DII, grant no. ANR11-LBX-0023-01.
Supported by the French National Research Agency (ANR), grant nos. ANR-13-BSH1-0004-02 and ANR-
11-LABEX-0047, and by GENES. Also supported by the “Chaire Economie et Gestion des Nouvelles
Donn´ees,” under the auspices of Institut Louis Bachelier, Havas Media, and Paris-Dauphine.