On the use of KPCA pre-filtering for KCCA method
Received:24July2016 /Accepted:23January2017 /Published online:13February2017
Abstract In this paper, we propose a kernel method to build
RKHS models of nonlinear system. This method entitled
prefiltered kernel canonical correlation analysis (PKCCA)
performs a prefiltering prior to the use of KCCA in order to
avoid low variances between canonical coefficients and the
learning data set observations. The prefilter phase is based
on KPCA. The method is used to identify a benchmark non-
linear system and is compared to KCCA and SVM.
Keywords Kernel method
Canonical correlation coordinates
Canonical coefficients in linear case
Canonical coefficients in kernel case
Canonical correlation patterns
Canonical correlation analysis (CCA)  is a statistical tech-
nique for finding the linear relationship between two sets of
variables. It aims to find a linear dependency between trans-
formations for each transformation of the two sets of variables
such that the projected variables in the transformed space are
maximally correlated. Kernel canonical correlation analysis
(KCCA) [2, 4, 14] is the kernel extension of CCA with posi-
tive definite kernels.
When the feature space of the input and output contains
many variables, it is common to use kernel principal compo-
nent analysis (KPCA) to prefilter the data to reduce the dimen-
sions of the data sets, that is to say, apply KPCA to the feature
space of the input and output separately, then apply KCCA to
the resulting data.
Another reason for using the KPCA prefiltering is that
when the number of variables is not small relative to the sam-
ple size, the KCCA method may become unstable .
In this paper, we are interested in the kernel version of CCA
to identify systems with models developed on reproducing
kernel Hilbert space (RKHS) .
The direct application of KCCA method in system identi-
fication problem may lead to instability problem. This is due
to the high (possibly infinite) dimension of the feature space.
This instability could be avoided by prefiltering using KPCA.
Prefiltered kernel canonical correlation analysis (PKCCA)
can be used for reducing sample complexity of prediction
problems using unlabeled data. The applications range broad-
ly across a number of fields, including medicine, meteorology,
chemometrics, biology and neurology, natural language pro-
cessing, speech processing, computer vision, and multimodal
In order to construct a RKHS model of a nonlinear system
using the canonical correlation analysis, we first prefilter the
Laboratory of Automatic Signal and Image Processing, National
School of Engineers of Monastir, University of Monastir,
5019 Monastir, Tunisia
Int J Adv Manuf Technol(2017)91:4331–4340