ISSN 0032-9460, Problems of Information Transmission, 2012, Vol. 48, No. 4, pp. 376–394.
Pleiades Publishing, Inc., 2012.
Original Russian Text
C. Gioran, I. Kontoyiannis, 2012, published in Problemy Peredachi Informatsii, 2012, Vol. 48, No. 4, pp. 88–108.
Complexity-Compression Tradeoﬀs in Lossy
Compression via Eﬃcient Random Codebooks
C. Gioran and I. Kontoyiannis
Department of Informatics, Athens University of Economics and Business, Athens, Greece
Received January 18, 2011; in ﬁnal form, January 31, 2012
Abstract—The compression-complexity trade-oﬀ of lossy compression algorithms that are
based on a random codebook or a random database is examined. Motivated, in part, by recent
results of Gupta–Verd´u–Weissman (GVW) and their underlying connections with the pattern-
matching scheme of Kontoyiannis’ lossy Lempel–Ziv algorithm, we introduce a nonuniversal
version of the lossy Lempel–Ziv method (termed LLZ). The optimality of LLZ for memory-
less sources is established, and its performance is compared to that of the GVW divide-and-
conquer approach. Experimental results indicate that the GVW approach often yields better
compression than LLZ, but at the price of much higher memory requirements. To combine the
advantages of both, we introduce a hybrid algorithm (HYB) that utilizes both the divide-and-
conquer idea of GVW and the single-database structure of LLZ. It is proved that HYB shares
with GVW the exact same rate-distortion performance and implementation complexity, while,
like LLZ, requiring less memory, by a factor which may become unbounded, depending on the
choice of the relevant design parameters. Experimental results are also presented, illustrating
the performance of all three methods on data generated by simple discrete memoryless sources.
In particular, the HYB algorithm is shown to outperform existing schemes for the compression
of some simple discrete sources with respect to the Hamming distortion criterion.
One of the last major outstanding classical problems of information theory is the development
of general-purpose, practical, eﬃciently implementable lossy compression algorithms. The corre-
sponding problem for lossless data compression was essentially settled in the late 1970s by the
advance of the Lempel–Ziv (LZ) family of algorithms [1–3] and arithmetic coding [4–6]; see also
the texts [7, 8]. Similarly, from the early- to mid-1990s on, eﬃcient channel coding strategies
emerged that perform close to capacity, primarily using sparse graph codes, turbo codes, and local
message-passing decoding algorithms; see, e.g., [9–12], the texts [13–15], and references therein.
For lossy data compression, although there is a rich and varied literature on both theoretical
results and practical compression schemes, near-optimal eﬃciently implementable algorithms are
yet to be discovered. From rate-distortion theory [16, 17] we know that it is possible to achieve a
sometimes dramatic improvement in compression performance by allowing for a certain amount of
distortion in the reconstructed data. But the majority of existing algorithms are either compression-
suboptimal or they involve exhaustive searches of exponential complexity at the encoder, making
them unsuitable for realistic practical implementation.
Supported in part by the Marie Curie International Outgoing Fellowship, grant no. PIOF-GA-2009-235837.