Context Sensitive Rewriting Codes for Flash Memory

Context Sensitive Rewriting Codes for Flash Memory Abstract Writing data on flash memory is asymmetric in the sense that it is possible to change a 0 into a 1-bit, but erasing a 1 back to value 0 is much more expensive and can only be done in blocks. This has triggered the development of rewriting codes in which new data can overwrite the old one, subject to the constraint of never changing a 1 into a 0. The notion of context-sensitive rewriting codes is introduced and we bring here the analysis of the compression performance of a family of such codes, based on generalizations of the Fibonacci sequence. This is then compared with experimental results. 1. INTRODUCTION Flash memory [1, 2] is nowadays omnipresent in our personal computers, mobile phones, digital cameras and many other devices. There are, however, several features that are significantly different for flash memory, when compared to previously known storage media. Contrarily to conventional storage, writing a 0-bit or a 1-bit on flash are not considered to be symmetrical tasks. If a block contains only zeros, individual bits can be changed to 1. However, once a bit is set to 1, it can be changed back to value 0 only by erasing entire blocks (of size 0.5MB or more). Therefore, while one can randomly access and read any data in flash memory, overwriting or erasing it cannot be performed in random access, only blockwise. Consider then the problem of reusing a piece of flash memory, after a block of r bits has already been used to encode some data in what we shall call a first round of encoding. Now some new data are given to be encoded in a second round, and the challenge is to reuse the same r bits, or a subset thereof, without incurring the expensive overhead of erasing the entire block before rewriting. This can be generalized to three, or even more, rounds of writing, all with the same constraint of changing only 0s to 1s. Actually, the problem of devising such special codes has been treated long before flash memory became popular, under the name of write-once memory (WOM). Rivest and Shamir [3] suggested a simple way to use three bits of memory to encode two rounds of the four possible values of two bits. This work has been extended over the years, see [4–7] to mention just a few, and the corresponding codes are called rewriting codes. We shall refer to the Rivest-Shamir code below as RS-WOM. As a baseline for measuring the compression efficiency, we use a compression ratio defined as the number of provided information bits divided by the number of actually used storage bits. The number of information bits is in fact the information content of the data, whereas the number of storage bits depends on the way the data are encoded. For example, consider a 3-digit decimal number, with each digit being encoded in a 4-bit binary encoded decimal, that is, the digits 0,1,…,9 are encoded as 0000,0001,…,1001, respectively. The information content of the three digits is ⌈log21000⌉=10 and the number of storage bits is 12, which yields the ratio 10/12=0.833. For a standard binary encoding, information and storage bits are equivalent, giving a baseline of 1. For rewriting codes, we use the combined number of storage bits of all (two or more) writing rounds, thus the above mentioned RS-WOM-code yields a ratio of 43=1.333. In the rewriting codes literature, this ratio is called the sum-rate. This effectively increases the storage capacity of the flash memory between erasing cycles. For two rounds, the theoretical best possible ratio is log3=1.585, see [8], and the best ratio achieved so far is 1.493 [9]. For the RS-WOM-code, every bit-triplet is coded individually, independently of the preceding ones. The code is thus not context sensitive, and this is true also for many of its extensions. We suggest rewriting codes with two rounds, based on exploiting context sensitivity by using a special encoding in the first round that might be more wasteful than the standard encoding, but has the advantage of allowing the unambiguous reuse of a part of the data bits in the second round; the overall number of bits used in both rounds together is thereby increased. These encodings are based on generalizations of Fibonacci Codes [10], whose main relevant feature is that 1-bits never appear in adjacent positions. The aim of this work is to study how to apply Fibonacci representations to devise rewriting codes. The expected performance is analyzed and compared with empirical results on extended experiments. In the next section, we describe the basic form of the new coding procedure. Section 3 brings an extension based on a generalization of Fibonacci codes, and analytical estimates of the compression ratios are derived in Section 4. For comparison, Section 5 describes experimental results on input files of various sizes and nature. One of the advantages of using context sensitivity is that it enables the development of a compression booster, turning certain rewriting codes with k writing rounds into a code with k+1 writing rounds, for k≥1. The details are described in Section 6. 2. DESCRIPTION OF THE CODING PROCEDURE Consider an unbounded stream of data bits to be stored. It does not matter what these input bits represent and several interpretations might be possible. For example, the binary string 010011100101000101111000 could represent the ascii encoding of the character string NQx, as well as the standard binary encoding of the integer 5 132 664. By working directly at the level of data bits, the following method is most general and could be applied to any kind of input data. For technical reasons, it is convenient to break the input stream into successive blocks of n bits, for some constant n. This may help limiting the propagation of errors and setting an upper bound to the numbers that are manipulated. In any case, this does not limit the scope of the method, as the processed blocks can be concatenated to restore the original input. To continue the above example, if n=8, the input blocks are 01001110, 01010001 and 01111000, the first of which represents the character N or the number 78. The description below concentrates on the encoding of a single block of length n. A block of n bits can be used to store numbers between 0 and 2n−1 in what is commonly called the standard binary representation, based on a sum of different powers of 2. Any number x in this range can be uniquely represented by the string bn−1bn−2⋯b1b0, with bi∈{0,1}, such that x=∑i=0n−1bi2i. But this is not the only possibility. Actually, there are infinitely many binary representations for a given integer, each based on a different numeration system [11]. The numeration system used for the standard representation is the sequence of powers of 2: {1,2,4,8,…}. Another popular and useful numeration system in this context is based on the Fibonacci sequence: {1,2,3,5,8,13,…}. Fibonacci numbers are defined by the following recurrence relation: Fi=Fi−1+Fi−2fori≥1, and the boundary conditions F0=1andF−1=0. Any integer x can be decomposed into a sum of distinct Fibonacci numbers; it can, therefore, be represented by a binary string crcr−1⋯c2c1 of length r, called its Fibonacci or Zeckendorf representation [12], such that x=∑i=1rciFi. This can be seen from the following procedure producing such a representation: given the integer x, find the largest Fibonacci number Fr smaller or equal to x; then continue recursively with x−Fr. For example, 49=34+13+2=F8+F6+F2, so its binary Fibonacci representation would be 10100010. Moreover, the use of the largest possible Fibonacci number in each iteration implies the uniqueness of this representation. Note that as a result of this encoding procedure, there are never consecutive Fibonacci numbers in any of these sums, implying that in the corresponding binary representation, there are no adjacent 1s. This property of the appearance of a 1-bit implying that the following bit, if it exists, must be a zero, has been exploited in several useful applications: robustness to errors [13], the design of Fibonacci codes [14], direct access [15], fast decoding and compressed search [10, 16], compressed matching in dictionaries [17], faster modular exponentiation [18], etc. The present work is yet another application of this idea. Extensions of the Fibonacci sequence have already been used in other coding applications, like in the error-correcting code suggested in [19], supporting multiple insertions and deletions. The repeated encoding will be performed in three steps: Encoding the data of the first round; Preparing the data block for a possible second encoding; Encoding the (new) data of the second round, overwriting the previous data. In the first step, the n bits of the block are transformed into a block of size r by recoding the integer represented in the input block into its Fibonacci representation. The resulting block will be longer, since more bits are needed, but also sparser, because of the property prohibiting adjacent 1s. To get an estimate of the increase in the number of bits, note that the largest number that can be represented is y=2n−1. The index r of the largest Fibonacci number Fr≈ϕr+1/5 needed to represent y is r=⌊logϕ(5y)−1⌋≃⌊(logϕ2)n+logϕ5−1⌋≃⌊1.44n−0.67⌋, where ϕ=1+52 is the well-known golden ratio. The storage penalty incurred by passing from the standard binary representation to the Fibonacci representation is thus 44%, for any block size n. Note that since n is fixed in our discussion, this is also true for r≃(logϕ2)n≃1.44n; the blocks in both representations may thus contain leading zeroes. The second step is supposed to be performed after the data written in the first round has finished its life cycle and is not needed any more, but instead of overwriting it by first erasing the entire block, we wish to be able to reuse the block subject to the update constraints of flash memory. The step is not needed for the correctness of the procedure and therefore seems optional, with a potential to increase the number of data bits that can be stored in the second round. Yet we show at the end of this section that the step is essential. In the second step, a maximal number of 1-bits is added without violating the non-adjacency property of the Fibonacci encoding. This means that short runs of zeros limited by 1-bits, like 101 and 1001, are not touched, but the longer ones, like 100001 or 1000001, are changed to 101001 and 1010101, where the added bits are boldfaced. In general, in a run of zeros of odd length 2i+1, every second zero is turned on, and this is true also for a run of zeros of even length 2i, except that for the even length the last bit is left as zero, since it is followed by a 1. A similar strategy is used for a run of leading zeros in the block: a run of length 1 is left untouched, but longer runs, like 001, 0001 or 00001, are changed to 101, 1001 and 10101, respectively. As a result of this filling strategy, the data block still does not have any adjacent 1s, but the lengths of the 1-limited zero-runs are now either 1 or 2, and the length of the leading run is either 0 or 1. In the third step, new data are encoded in the bits immediately to the right of every 1-bit. Since it is known that these positions contained only zeros at the end of Step 2, they can be used at this stage to record new data, and their location can be identified. The data block at the end of the third step thus contains bits of three kinds: separator bits (S), data bits (D) and extension bits (E). An E-bit is the second zero in a 0-run of length two at the end of the second step. The first bit of the blocks is either an S-bit, if it is 1, or an E-bit, if it is 0 (which can only occur if the leading zero-run was of length 1). S-bits have value 1 and are followed by D-bits; D-bits have value 0 or 1 and are followed by an S-bit (1) or by an E-bit (0); E-bits have value 0 and are followed by S-bits. Figure 1 continues the running example, showing the data block at the end of each of the steps. The strings are partitioned into blocks of 8 bits just for visual convenience. The input is the character string NQx, and the 24-bit numerical value 5 132 664 of its ascii encoding (with leading zeros), is given in Fibonacci encoded form with ⌊1.44×24−0.67⌋=33bits in the first line. The second line displays the block at the end of Step 2, after having added some 1-bits which are boldfaced. The data bits that can be used in the next step are those immediately to the right of the 1-bits and are currently all zero. In this example, there are 14 such bits. For the last step, suppose the new data to be stored are the number 7777, whose standard 14-bit binary representation is 01111001100001. These bits are interspersed into the data bits of the block, resulting in the string appearing in the third line, in which these data bits are boxed. The fourth line classifies the bits of the resulting string into S-, D- and E-bits. In this example, the combined number of information bits is 24 for the string NQx, plus 14 for the number 7777, that is 38 bits, but using only 33 bits of storage, yielding a compression ratio of 3833=1.152. Note also that all the changes from one step to another are consistent with the flash memory constraints, namely that only changes from 0 to 1 are allowed, but not from 1 to 0. FIGURE 1. View largeDownload slide Encoding example. FIGURE 1. View largeDownload slide Encoding example. Decoding at the end of the first step is according to Fibonacci codes, as in [10], and decoding of the data of the second round at the end of the third step can be done using the decoding automaton appearing in Fig. 2. An initial state I is used to decide whether to start in state S if the first bit is a 1, or in state E if it is a zero. The states S, D and E are entered after having read an S-bit, D-bit and E-bit, respectively. Only bits leading to state D are considered to carry information. There is no edge labeled 0 emanating from state E, because E-bits are always followed by 1s. FIGURE 2. View largeDownload slide Decoding automaton for data in the second round. FIGURE 2. View largeDownload slide Decoding automaton for data in the second round. Since at the end of the second step, no run of zeros can be longer than 2, the worst-case scenario is when every third bit is a separator. Any block is then of the form SDESDE⋯, and one-third of the bits are data-bits. The number of data bits in the third step is thus 1.44n/3=0.48n, which together with the n bits encoded in the first step, yield 1.48n, 2.76% more than the 1.44n storage bits used. Thus even in the worst case, there is a gain, albeit a small one. The maximal possible benefit will be in the case whenthere are no E-bits at all, that is, the block is of the form SDSDSD⋯. In this case, half of the bits are D-bits, and the compression ratio will be n+(logϕ2)n2(logϕ2)n=1logϕ2+12≃11.44+12=1.194. The analysis of the average gain is deferred to the more general case in the next section. A closer look at the second step, that of inserting additional 1-bits, which at first sight seemed merely as a technical improvement to increase the potential number of data bits, reveals that it is in fact critical to assure any gain at all. Theorem If the second step of the encoding procedure is omitted, there will be no improvement in the compression ratio. Proof If the second step is omitted, and we use as data bits of the second round only the bits following the 1-bits of the original Fibonacci encoding, then the following loop shows how to restore the data of both writing rounds: while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y A and B are initially empty strings, and ∥ stands for concatenation. After applying the loop to an encoded string, A contains the Fibonacci encoding of the first round, and B is the binary data of the second round. Yet, the total information conveyed by n input bits, no matter of how they have been partitioned, cannot possibly exceed the theoretical information content of n bits, or, in other words, if all the data are accessible, the compression ratio, as defined above, cannot be larger than 1.□ In other words, if at the end of the second round of a rewriting method with ratio >1, we are still able to recover all the data of the first writing round, this scheme could have been used a priori to produce a code being able to encode more messages than given by the information theoretic bound. This means that to get a real gain by rewriting codes, it is necessary to destroy at least a part of the data of the first writing round. We conclude that the insertion of additional 1s in the second step, after which the code used in the first step is not accessible any more, is not only a correcting step to improve the compression but is essential to enable any possible gain. 3. GENERALIZATION BASED ON EXTENSIONS OF FIBONACCI CODES One can extend the idea above to a code in which there are at least m 0-bits, for m≥1, between any two 1-bits. Such a code for m=2 has been designed for the encoding of data on CD-ROMs [20] and is known as Eight-to-Fourteen-Modulation: every byte of 8 bits is mapped to a bit-string of length 14 in which there are at least two zeros between any two 1s. In fact, the Fibonacci code and its extensions are special cases of a larger class known as Run-Length Limited codes [21], which impose lower and upper bounds on the number of zeros separating 1s. The lower bound on the number of zeros is obtained by representing numbers according to the basis elements of numeration systems which are extensions of the Fibonacci sequence, based on the following recurrences: Ak(m)=Ak−1(m)+Ak−m(m)fork>m+1, and the boundary conditions Ak(m)=k−1for1<k≤m+1. In particular, Ak(2)=Fk−1 are the standard Fibonacci numbers. The first few elements of the sequences A(m)≡{1=A2(m),A3(m),A4(m),…} for 2≤m≤6 are listed in the right part of Table 1. Table 1. Generalization of Fibonacci-based numeration systems. m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 View Large Table 1. Generalization of Fibonacci-based numeration systems. m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 View Large A closed form expression of the elements of the sequence A(m) can be obtained by considering the characteristic polynomial xm−xm−1−1=0, and finding its m roots φm,1,φm,2,…,φm,m. The element Ak(m) is then a linear combination of the kth power of these roots. For these particular polynomials, when m>2, there is only one root, say φm,1≡φm, which is real and is larger than 1, all the other roots are complex numbers a+ib with b≠0 and with norm strictly smaller than 1. For m=2, the second root 1−52=−0.6180 is also real, but its absolute value is <1. It follows that with increasing k, all the terms φm,jk, 1<j≤m, quickly vanish, so that the elements Ak(m) can be accurately approximated by powers of the dominant root φm alone, with appropriate coefficients, Ak(m)≈amφmk−1. The constants am and φm are listed in Table 1. For a given m, any integer x can be decomposed into a sum of distinct elements of the sequence A(m); it can therefore be uniquely represented by a binary string crcr−1⋯c3c2 of length r−1, such that x=∑i=2rciAi(m), using the recursive encoding method presented above, based on finding in each iteration the largest element of the sequence fitting into the remainder. For example, 36=28+6+2=A10(3)+A6(3)+A3(3), so its representation according to A(3) would be 100010010. As a result of the encoding procedure, the indices i1,i2,… of the elements in the sum x=∑i=2rciAi(m) for which ci=1 satisfy that ik+1≥ik+m, for k>2. In the above example x=36, these indices are 3, 6 and 10. This means that in the corresponding binary representation, there are at least m−1 zeros between any two 1s. Extending the calculation of the increase in the number of bits to the general case, recall that the largest number that can be represented is y=2n−1. The largest A(m) number Ar(m)≈amφmr−1 needed to represent y is r=⌊logφm(y/am)+1⌋=⌊(logφm2)n−logφmam+1⌋. The storage penalty incurred by passing from the standard to the A(m) representation is thus at most a factor of (logφm2), for any block size n. This will be 44% for the Fibonacci numbers ( m=2), and 81% and 115% for their extensions, with m=3 and m=4, respectively, as listed in the fourth column of Table 1. The encoding according to A(m) will be performed in three steps similar to those described for m=2 in the previous section. In the first step, the n bits of the block are transformed into a block of size r≃(logφm2)n by recoding the integer represented in the input block into its representation according to A(m). In the second step, a maximal number 1-bits is added without violating the property of having at least m−1 zeros between 1s. This means that in a run of zeros of length j, limited on both sides by 1s, with j≥2m−1, the zeros in positions m,2m,…,⌊j−m+1m⌋m are turned on. For a run of leading zeros of length j (limited by a 1-bit only at its right end), for j≥m, the zeros in positions 1,m+1,2m+1,…,⌊j−mm⌋m+1 are turned on. Another special case is the run of zeros following the rightmost 1-bit. For example, for A(3), 100000000001 is turned into 100100100001, 00000001 is turned into 10010001, and 10000 is turned into 10010. As a result of this filling strategy, the data block still does have at least m−1 zeros between 1s, but the lengths of the 1-limited zero-runs are now between m−1 and 2m−2, and the lengths of the leftmost and rightmost runs are between 0 and m−1. In the third step, new data are encoded in the m−1 bits immediately to the right of every 1-bit. Since it is known that these positions contained only zeros at the end of Step 2, they can be used at this stage to record new data, and their location can be identified. For the rightmost 0-run, the number of data-bits may be smaller than m−1, for example, for A(3) if the last run was 10000, it was transformed to 10010 and only one data bit may be added after the rightmost 1. To continue the analogy with the case m=2, there are now data bits of different kinds 𝖣1 to 𝖣m−1, and similarly for extension bits 𝖤1 to 𝖤m−1. The decoding of the data of the second round at the end of the third step for A(3) can be done using the decoding automaton appearing in Fig. 3. An initial state I is used to decide whether to start in state S if the first bit is a 1, or in state 𝖤1 if it is a zero. Only bits leading to states 𝖣1 or 𝖣2 are considered to carry information. There is no edge labeled 0 emanating from state 𝖤2, because a second E-bit is always followed by 1s. Similar decoding automata, with states I, S, 𝖣1 to 𝖣m−1 and 𝖤1 to 𝖤m−1 can be designed for all m≥2. Figure 3. View largeDownload slide Decoding automaton for data in the second round with A(3). Figure 3. View largeDownload slide Decoding automaton for data in the second round with A(3). 4. SPACE ANALYSIS 4.1. Worst case—lower bound of performance Since at the end of the second step, no run of zeros can be longer than 2m−2, the worst-case scenario is when every (2m−1) th bit is a 1-bit. Any block is then of the form 𝖤𝖤⋯𝖤𝖲𝖣𝖣⋯𝖣𝖤𝖤⋯𝖤𝖲𝖣𝖣⋯𝖣𝖤𝖤⋯𝖤𝖲, where all the runs of Ds and Es are of length m−1 and, in the limit, when the block size tends to infinity so that the initial run of E-bits and the final S-bit may be neglected, (m−1)/(2m−1) of the bits are data-bits. The worst-case compression factor is thus n+(m−1)(logφm2)n2m−1(logφm2)n=1logφm2+m−12m−1, (1) where the first and second summands represent, respectively, the contribution of the first and second writing rounds. This worst-case scenario corresponds to a block size r after the first step, which is a multiple of 2m−1, plus a final S-bit and m−1 initial E-bits, that is, r≡m(mod(2m−1)); for each such r, there is exactly one possible input string yielding this worst case, e.g. for m=3, 0010000100001 of length r=13, which corresponds to the value 70 written in A(3), for an initial input string 01000110 of length n=8. 4.2. Best case—upper bound of performance The maximal possible benefit will be in the case when there are no E-bits at all, that is, the block is of the form 𝖲𝖣𝖣⋯𝖣𝖲𝖣𝖣⋯𝖣𝖲𝖣⋯, where all the runs of Ds are of length m−1 and the number of data-bits is (m−1)/m. In this case, the compression ratio will be 1logφm2+m−1m. (2) For example, for m=2, the best case for r=10 is X=1010101010, which corresponds to the value 143 of an initial input string 10001111 of length n=8. The best case for a given length r, however, is not unique; in fact, it is shared by all the input strings which after adding the 1-bits of the second step will be of the above form. To continue the last example, some of the strings in A(2), other than X, of length r=10 yielding five data bits are 1000101010, 1010000000 or even 0000000000, corresponding, respectively, to input strings with values 109, 123 and 0; all these strings turn into X when the 1-bits of the second step are added. 4.3. Average case Similarly to the above, the compression ratio will be 1logφm2+m−1Dm, where Dm is a random variable designing the distance in bits between consecutive 1ʼs at the end of the second step. Thus D2 is either two or three, and generally, Dm assumes values between m and 2m−1, inclusive. We are interested in the expected value, which is 1logφm2+Em−1Dm=1logφm2+(m−1)∑j=0m−1Prob(Dm=m+j)m+j. (3) To evaluate Equation (3), we proceed by the following steps. First, the probability of a 1-bit in the given context is established. We then specify the probabilistic model on the basis of which the average is defined, and evaluate the probability distribution of the distance between 1-bits according to this model. 4.3.1. Probability of a 1-bit The constraint of the A(m) encoding implies that the probabilities of occurrence of 0s and 1s are not the same, as would be the case in the standard binary encoding, when all possible inputs are supposed to be equi-probable. To calculate the average compression ratio, one needs first to know the probability of occurrence of a 1-bit in the binary representation according to A(m). Let us refer to binary strings representing integers according to A(m) as A(m)-strings. Thus 10010 is an A(3)-string, but 01010 is not. Define Tr(m) as the number of different A(m)-strings of length r, and Qr(m) as the total number of 1s in all the A(m)-strings of length r, r≥0. For example, the sets of A(3)-strings of lengths up to 4 are {0,1}, {00,01,10}, {000,001,010,100} and {0000,0001,0010,0100,1000,1001}. In general, one gets that T0(m)=0,Tr(m)=r−1for0<r≤m,andTr(m)=Tr−1(m)+Tr−m(m)forr>m, since the A(m)-strings of length r can be generated by either prefixing an A(m)-string of length r−1 with 0, or by prefixing an A(m)-string of length r−m, with 10⋯0 (1 followed by m−1 zeros). In fact, one gets that Tr(m)=Ar+1(m) for r≥1. Similarly, for Qr(m), one gets the recurrence Q0(m)=0,Qr(m)=rfor0<r≤m,andQr(m)=Qr−1(m)+Qr−m(m)+Tr−m(m)forr>m. We again split the set of A(m)-strings of length r into those with leading 0 and those with leading 10⋯0 (1 followed by m−1 zeros); removing these leading bits, we are left in the former set with A(m)-strings of length r−1, which contribute Qr−1(m) 1-bits, and in the latter set with Tr−m(m) A(m)-strings of length r−m, which contribute Qr−m(m) 1-bits, to which the Tr−m(m) 1s in the removed prefixes of the form 10⋯0 have to be added. To evaluate the probability of a 1-bit in the representation according to A(m), which we shall denote by pm′, the number of 1-bits has to be divided by the total number of bits, both limited to A(m)-strings of length r, that is Qr(m)/(rTr(m)). The limiting values of these ratios, when r→∞, appear in the second column of Table 2, for 2≤m≤6. Table 2. Compression ratios with A(m). m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 View Large Table 2. Compression ratios with A(m). m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 View Large 4.3.2. Probabilistic model For the current calculation, we assume that the bit-string at the end of the first step has been created by a stochastic process in which the bits are generated from left to right, independently from each other, with some probability pm for generating a 1-bit, every 1-bit being immediately followed by m−1 zeros, and with probability hm=1−pm for a zero-bit in any position which is not within the m−1 immediately following a 1-bit. The probabilities pm will be chosen so that this process shall produce bit strings with at least m−1 0s between 1s and with probability of occurrence of a 1-bit in the entire string being pm′ and the corresponding probability of a 0-bit being 1−pm′. Consider a block of n bits. The number of 1-bits is expected to be npm′, and thus npm′(m−1) 0-bits are expected to be forced and not generated by the process. The expected number of generated bits is therefore n−npm′(m−1)=n(1−(m−1)pm′), of which npm′ bits should be 1s. One can thus derive the probabilities of a 1-bit and a 0-bit in the generating process as: pm=pm′1−(m−1)pm′ and hm=1−pm=1−mpm′1−(m−1)pm′. Both the output of the encoding process and the stochastic process assumed in the model thus produce the same set of binary strings, and according to the same probabilities. 4.3.3. Probability distribution of distance between 1-bits Using these assumptions, let us calculate the probability distribution of the distance between consecutive 1-bits, which is the expected length of a zero-run including the terminating 1-bit in the data block at the end of the second step. The distance between consecutive 1s is mostly m, but sometimes it is m+1, m+2, etc., up to at most 2m−1. It will be m+j, for 0≤j<m, if there is a run of 0s of length mk+j−1, with k≥1, between the 1-bits at the end of the first step, as, for m=3, in 10001 or 10000001 for j=1, or in 100001 and 100000001 for j=2. Recall that the first m−1 zeros are forced by the leading 1, so that the probability of a run of km+j−1 zeros, for k≥1, is hkm+j−mp (we have shortened hm and pm to h and p for clarity). After inserting the additional 1-bits in the second step, such a first-step-run of km+j−1 zeros followed by a 1 will be transformed into a sequence of k elements: k−1 m-tuples 00⋯01 ( m−1 zeros followed by 1) and a single m+j-tuple of m+j−1 zeros followed by a 1, for 0≤j<m. The distance between consecutive 1s will thus increase from m to m+j, for j>0, only if the original run (at the end of the first step) was of length mk+j, for k≥1, and for such a run, only one of the k elements will be a m+j tuple. Assuming an infinitely long input block, the probability of getting a distance between 1s larger than m is thus Prob(Dm>m)=∑j=1m−1∑k=1∞1khmk−m+jp=∑j=1m−1phm−j∑k=1∞hmkk=∑j=1m−1phm−jln11−hm, where we have used the identity ln11−x=∑k=1∞xkk [22, Section 1.2.9, Equation (25)]. In all other cases, that is, for j=0 and for the k−1 first elements when j>0, the distance between consecutive 1s will be m, and the probability for this event will be Prob(Dm=m)=1−Prob(Dm>m). To clarify this point, consider m=3 and the following two runs of zeros at the end of Step 1, i.e. after the A(3) encoding, corresponding to distances 9=3·3+0 and 17=3·5+2 between consecutive 1s, respectively: 1000000001 100000000000000001. At the end of the second step, after having inserted the boldfaced 1s, these strings are transformed into 100𝟭00𝟭001 100𝟭00𝟭00𝟭00𝟭00001, and only the last block of the second string is of length 5=m+2, whereas the first four blocks of the second string, and all the blocks of the first are of length 3=m. The expected value E1Dm needed in Equation (3) will therefore tend to Prob(Dm=m)m+∑j=1m−1p(m+j)hm−jln11−hm. 4.4. Analytical results The worst, best and average compression ratios, according to Equations (1–3), as well as the probability of a 1-bit, from which the average case can be derived, are listed in Table 2, for 2≤m≤6. The best values for each case are emphasized in bold. For the best and worst cases, the columns headed ratio show the proportion of data-bits used in the second round relative to the total number of bits. As can be seen, for the best and average cases there is always a gain relative to the baseline, at least for the given values of m, and in the worst case only for m=2. Interestingly, while m=2 is best in the worst and average cases, the highest value in the best case is obtained for m=3. If one extends the definition of the sequence A(m) also to m=1, one gets the sequence of powers of 2, that is, the standard binary numeration system, with no restrictions on the appearance of 1-bits. The compression ratio in that case will be 1. For higher values of m, the combined compression ratio may be higher, but the proportion of the first round data will be smaller. Table 3 brings these proportions for 1≤m≤6. One way to look at these results is thus to choose the order m of the encoding according to the partition between first and second round data one may be interested in. This is an advantage over previously known rewriting systems, in which the contribution of each round of writing is rigidly fixed in advance, like for the RS-WOM-scheme with (0.5, 0.5). Table 3. Proportions of first and second round data bits. m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 View Large Table 3. Proportions of first and second round data bits. m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 View Large 5. EXPERIMENTAL RESULTS All the numbers mentioned above have been derived analytically. To confirm a similar performance on real life applications, we decided to run the following tests on a variety of input files of different size and nature. This also serves to double-check that the assumptions of our probabilistic model were realistic. The first file (ran) consists of a sequence of 100000 random numbers between 0 and 255. The second file (bib) is a cleaned copy of the King James version of the Bible in English, from which all punctuation signs have been stripped. The third file (mis) is the Ebook version of Les Misérables by Victor Hugo in French, downloaded from the Gutenberg Project.1 The fourth file (zip) is the gzip compressed version of the third file. All the files were processed in blocks of n=400bits, that is, 50 bytes, and for each such block, the represented n-bit number was first transformed into the corresponding A(m)-string of length r=(logφm2)n, and the 1-bits of Step 2 have been added. The number db of possible data-bits for the second round was then evaluated, with the appropriate adaptations at the extremities of the block: every 1-bit contributes m−1 data-bits, except possibly the last one if there are less than m−1 0-bits after the rightmost 1. The compression ratio for each block is then (n+db)/r, and these values were averaged. The results appear in Table 4. The second column gives the size of the input files in Mbytes, and the other columns, the averaged compression ratio for 2≤m≤6. The last line repeats the analytic result of the theoretical average from the last column of Table 2, and the other lines correspond to the different input files. Table 4. Empirical compression ratios on test files. File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 View Large Table 4. Empirical compression ratios on test files. File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 View Large These results suggest that the rewriting compression ratio is practically independent of the nature of the input file, be it pure ascii in English or French, a compressed file or even a randomly generated one: the fluctuations of the ratio for a given value of m were below one-tenth of a percent. The results also confirm that the assumed probabilistic model is realistic, as the theoretical ratios differ from the observed ones by about 1% or less. The fact that the empirical ratios are consistently lower can be explained by the approximations in the theory, in particular, letting the block size n tend to infinity in the derivation of the probability pm′ of a 1-bit and when calculating E(1/Dm). 6. USAGE AS COMPRESSION BOOSTER The above ideas can be used to build a compression booster in the following way. Suppose we are given a rewriting system S allowing k rounds. This can be turned into a system with k+1 rounds by using, in a first round, the new encoding as described earlier, which identifies a subset of the bits in which the new data can be recorded. These bits are then used in k additional rounds according to S. Note that only context-insensitive systems, like the RS-WOM-code, can be extended in that way. Certain codes, like those suggested in [7], may be used with any number k of writing rounds, so for them, this boosting scheme is not relevant. But for other codes, the number of rounds is fixed and cannot be changed, and then this is a method to increment the number of rounds. Since the first round recodes the data using more bits, the extension with an additional round of rewriting will not always improve the compression. For example, for the A(m) codes, even if the second term of the numerator of the left hand side of Equation (1), representing the number of bits used in the second round, is multiplied by 43, the compression factor of the RS-WOM-code, one still gets only at most 1.319 on the average, for A(3), about 1.1% less than the RS-WOM-code used alone. However, for certain instances, using some A(m) code in the first round, followed by two rounds of RS-WOM, may yield better codes than RS-WOM, as can be seen in the example in Fig. 4. More generally, suppose some rewriting code is given, yielding an overall compression ratio of α in k rounds. The worst-case ratio for the boosted code with k+1 rounds is then 1logφm2+αm−12m−1, and we seek values of m, if they exist at all, for which this ratio is larger than α itself. This condition is equivalent to 2m−1mlog2φm>α. Similar constraints can be derived for the average and best cases. FIGURE 4. View largeDownload slide Encoding example with A(3). FIGURE 4. View largeDownload slide Encoding example with A(3). Figure 4 brings the same running example as above, this time for A(3), and coupled with the RS-WOM-code. The same input character string NQx is used, and the 24-bit numerical value 5 132 664 of its ascii encoding (with leading zeros), is given in A(3) encoded form with 40 bits in the first line. The second line displays the block at the end of Step 2, after having added, for this particular example, a single 1-bit, which is boldfaced. The data bits that can be used in the next step are the pairs immediately to the right of the 1-bits and are currently all zero. In this example, there are 24 such bits. There are thus 24+24=48 data bits in the two rounds together, using only 40 storage bit, which gives a ratio of 1.2. For the next step, suppose the new data to be stored in the second round are the number 55,555, and in the third round the number 44,444, whose standard 16-bit binary representations are 11 011001 00 0000 11 and 10 10 11 01 10 01 11 00, respectively. The RS-WOM-code considers these numbers as a sequence of pairs (the spaces have only been added for clarity), each of which is translated into a triplet, yielding two 24-bit strings 001 100 010 100 000 000 000 001 and 101101110100101011110111. These bits are interspersed into the data bits of the block, resulting in the strings appearing in the third and fourth lines, in which these data bits are boxed in pairs. The fifth line shows again the classification of the resulting string into S-, D- and E-bits. In this example, the combined number of information bits is 24 for the string NQx, plus 16 for each of the numbers 55,555 and 44,444, that is 56 bits, but using only 40 bits of storage, yielding a compression ratio of 1.4. Using the RS-WOM-code alone with 40 bits would only be able to store 53.3 bits of information. Note also that as before, all the changes from one step to another are consistent with the flash memory constraints, namely that only changes from 0 to 1 are allowed, but not from 1 to 0. 7. CONCLUDING REMARKS We have shown how the Fibonacci code and some of its generalizations can be used to devise a rewriting code suitable for flash memory. Our analysis shows that some gain may be achieved, though the performance cannot compete with state of the art rewriting codes, as in [8], or even the simple RS-WOM-code. The novelty of our approach and its main contribution is in the introduction of the new paradigm of context-sensitive rewriting codes, of which the Fibonacci codes are just an example. The new challenge is, therefore, to find other codes in which the appearance of the bits obeys some rules that can be exploited to encode more data in subsequent writing rounds, with the additional constraint that the overall compression ratio should be greater than 1, and if possible, better than the ratio of the currently known best rewriting codes. Examples are abundant: other RLL codes [21] with various parameters, or the set defined in [23] of all strings of the form y=xπ, where π is some fixed bit pattern of length at least one bit, x is any bit-string, and π occurs in y only as suffix. The Fibonacci code is the special case π=11. A more recent example can be found in [24], which extends the set of code words by allowing multiple delimiters. As the authors report better compression performance than for the Fibonacci codes, one may also expect an improvement for the corresponding rewriting codes. Many of these codes and similar ones, or some combination thereof, may be adapted to form a rewriting code with two or more rounds, following the basic techniques presented in this paper. References 1 Assar , M. , Nemazie , S. and Estakhri , P. ( 1995 ) Flash memory mass storage architecture. US Patent 5,388,083, issued Feb. 7, 1995. 2 Gal , E. and Toledo , S. ( 2005 ) Algorithms and data structures for flash memories . ACM Comput. Surv. , 37 , 138 – 163 . Google Scholar CrossRef Search ADS 3 Rivest , R.L. and Shamir , A. ( 1982 ) How to reuse a ‘write-once’ memory . Inf. Control , 55 , 1 – 19 . Google Scholar CrossRef Search ADS 4 Fiat , A. and Shamir , A. ( 1984 ) Generalized ‘write-once’ memories . IEEE Trans. Inf. Theory , 30 , 470 – 479 . Google Scholar CrossRef Search ADS 5 Jiang , A. , Bohossian , V. and Bruck , J. ( 2010 ) Rewriting codes for joint information storage in flash memories . IEEE Trans. Inf. Theory , 56 , 5300 – 5313 . Google Scholar CrossRef Search ADS 6 Kurkoski , B.M. ( 2010 ) Rewriting codes for flash memories based upon lattices, and an example using the E8 lattice. IEEE Globecom Workshop on Applications of Communication Theory to Emerging Memory Technologies, ACTEMT 2010, Miami, Florida, 6–10 December 2010, pp. 1861–1865. IEEE. 7 Burshtein , D. and Strugatski , A. ( 2013 ) Polar write once memory codes . IEEE Trans. Inf. Theory , 59 , 5088 – 5101 . Google Scholar CrossRef Search ADS 8 Shpilka , A. ( 2013 ) New constructions of WOM codes using the Wozencraft ensemble . IEEE Trans. Inf. Theory , 59 , 4520 – 4529 . Google Scholar CrossRef Search ADS 9 Yaakobi , E. , Kayser , S. , Siegel , P.H. , Vardy , A. and Wolf , J.K. ( 2012 ) Codes for write-once memories . IEEE Trans. Inf. Theory , 58 , 5985 – 5999 . Google Scholar CrossRef Search ADS 10 Klein , S.T. and Ben-Nissan , M.K. ( 2010 ) On the usefulness of Fibonacci compression codes . Comput. J. , 53 , 701 – 716 . Google Scholar CrossRef Search ADS 11 Fraenkel , A.S. ( 1985 ) Systems of numeration . Amer. Math. Mon. , 92 , 105 – 114 . Google Scholar CrossRef Search ADS 12 Zeckendorf , E. ( 1972 ) Représentation des nombres naturels par une somme des nombres de Fibonacci ou de nombres de Lucas . Bull. Soc. Roy. Sci. Liège , 41 , 179 – 182 . 13 Apostolico , A. and Fraenkel , A.S. ( 1987 ) Robust transmission of unbounded strings using Fibonacci representations . IEEE Trans. Inf. Theory , 33 , 238 – 245 . Google Scholar CrossRef Search ADS 14 Fraenkel , A.S. and Klein , S.T. ( 1996 ) Robust universal complete codes for transmission and compression . Discrete Appl. Math. , 64 , 31 – 55 . Google Scholar CrossRef Search ADS 15 Klein , S.T. and Shapira , D. ( 2016 ) Random access to Fibonacci encoded files . Discrete Appl. Math. , 212 , 115 – 128 . Google Scholar CrossRef Search ADS 16 Klein , S.T. and Shapira , D. ( 2016 ) Compressed matching for feature vectors . Theor. Comput. Sci. , 638 , 52 – 62 . Google Scholar CrossRef Search ADS 17 Klein , S.T. and Shapira , D. ( 2011 ) Compressed matching in dictionaries . Algorithms , 4 , 61 – 74 . Google Scholar CrossRef Search ADS 18 Klein , S.T. ( 2008 ) Should one always use repeated squaring for modular exponentiation? Inf. Process. Lett. , 106 , 232 – 237 . Google Scholar CrossRef Search ADS 19 Helberg , A.S.J. and Ferreira , H.C. ( 2002 ) On multiple insertion/deletion correcting codes . IEEE Trans. Inf. Theory , 48 , 305 – 308 . Google Scholar CrossRef Search ADS 20 Immink , K.A. , Nijboer , J.G. , Ogawa , H. and Odaka , K. ( 1985 ) Method of coding binary data. US Patent 4,501,000, issued Feb. 19, 1985. 21 Schouhamer Immink , K.A. ( 1990 ) Runlength-limited sequences . Proc. IEEE , 78 , 1745 – 1759 . Google Scholar CrossRef Search ADS 22 Knuth , D.E. ( 1973 ) The Art of Computer Programming, Volume I: Fundamental Algorithms ( 2nd edn ). Addison-Wesley , Boston . 23 Lakshmanan , K.B. ( 1981 ) On universal codeword sets . IEEE Trans. Inf. Theory , 27 , 659 – 662 . Google Scholar CrossRef Search ADS 24 Anisimov , A.V. and Zavadskyi , I.O. ( 2017 ) Variable-length prefix codes with multiple delimiters . IEEE Trans. Inf. Theory , 63 , 2885 – 2895 . Footnote 1 //www.gutenberg.org/files/17489/17489-h/17489-h.htm Author notes Handling editor: Prudence Wong © The British Computer Society 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The Computer Journal Oxford University Press

Context Sensitive Rewriting Codes for Flash Memory

Loading next page...
 
/lp/ou_press/context-sensitive-rewriting-codes-for-flash-memory-8ydfGkRbjc
Publisher
Oxford University Press
Copyright
© The British Computer Society 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
0010-4620
eISSN
1460-2067
D.O.I.
10.1093/comjnl/bxy020
Publisher site
See Article on Publisher Site

Abstract

Abstract Writing data on flash memory is asymmetric in the sense that it is possible to change a 0 into a 1-bit, but erasing a 1 back to value 0 is much more expensive and can only be done in blocks. This has triggered the development of rewriting codes in which new data can overwrite the old one, subject to the constraint of never changing a 1 into a 0. The notion of context-sensitive rewriting codes is introduced and we bring here the analysis of the compression performance of a family of such codes, based on generalizations of the Fibonacci sequence. This is then compared with experimental results. 1. INTRODUCTION Flash memory [1, 2] is nowadays omnipresent in our personal computers, mobile phones, digital cameras and many other devices. There are, however, several features that are significantly different for flash memory, when compared to previously known storage media. Contrarily to conventional storage, writing a 0-bit or a 1-bit on flash are not considered to be symmetrical tasks. If a block contains only zeros, individual bits can be changed to 1. However, once a bit is set to 1, it can be changed back to value 0 only by erasing entire blocks (of size 0.5MB or more). Therefore, while one can randomly access and read any data in flash memory, overwriting or erasing it cannot be performed in random access, only blockwise. Consider then the problem of reusing a piece of flash memory, after a block of r bits has already been used to encode some data in what we shall call a first round of encoding. Now some new data are given to be encoded in a second round, and the challenge is to reuse the same r bits, or a subset thereof, without incurring the expensive overhead of erasing the entire block before rewriting. This can be generalized to three, or even more, rounds of writing, all with the same constraint of changing only 0s to 1s. Actually, the problem of devising such special codes has been treated long before flash memory became popular, under the name of write-once memory (WOM). Rivest and Shamir [3] suggested a simple way to use three bits of memory to encode two rounds of the four possible values of two bits. This work has been extended over the years, see [4–7] to mention just a few, and the corresponding codes are called rewriting codes. We shall refer to the Rivest-Shamir code below as RS-WOM. As a baseline for measuring the compression efficiency, we use a compression ratio defined as the number of provided information bits divided by the number of actually used storage bits. The number of information bits is in fact the information content of the data, whereas the number of storage bits depends on the way the data are encoded. For example, consider a 3-digit decimal number, with each digit being encoded in a 4-bit binary encoded decimal, that is, the digits 0,1,…,9 are encoded as 0000,0001,…,1001, respectively. The information content of the three digits is ⌈log21000⌉=10 and the number of storage bits is 12, which yields the ratio 10/12=0.833. For a standard binary encoding, information and storage bits are equivalent, giving a baseline of 1. For rewriting codes, we use the combined number of storage bits of all (two or more) writing rounds, thus the above mentioned RS-WOM-code yields a ratio of 43=1.333. In the rewriting codes literature, this ratio is called the sum-rate. This effectively increases the storage capacity of the flash memory between erasing cycles. For two rounds, the theoretical best possible ratio is log3=1.585, see [8], and the best ratio achieved so far is 1.493 [9]. For the RS-WOM-code, every bit-triplet is coded individually, independently of the preceding ones. The code is thus not context sensitive, and this is true also for many of its extensions. We suggest rewriting codes with two rounds, based on exploiting context sensitivity by using a special encoding in the first round that might be more wasteful than the standard encoding, but has the advantage of allowing the unambiguous reuse of a part of the data bits in the second round; the overall number of bits used in both rounds together is thereby increased. These encodings are based on generalizations of Fibonacci Codes [10], whose main relevant feature is that 1-bits never appear in adjacent positions. The aim of this work is to study how to apply Fibonacci representations to devise rewriting codes. The expected performance is analyzed and compared with empirical results on extended experiments. In the next section, we describe the basic form of the new coding procedure. Section 3 brings an extension based on a generalization of Fibonacci codes, and analytical estimates of the compression ratios are derived in Section 4. For comparison, Section 5 describes experimental results on input files of various sizes and nature. One of the advantages of using context sensitivity is that it enables the development of a compression booster, turning certain rewriting codes with k writing rounds into a code with k+1 writing rounds, for k≥1. The details are described in Section 6. 2. DESCRIPTION OF THE CODING PROCEDURE Consider an unbounded stream of data bits to be stored. It does not matter what these input bits represent and several interpretations might be possible. For example, the binary string 010011100101000101111000 could represent the ascii encoding of the character string NQx, as well as the standard binary encoding of the integer 5 132 664. By working directly at the level of data bits, the following method is most general and could be applied to any kind of input data. For technical reasons, it is convenient to break the input stream into successive blocks of n bits, for some constant n. This may help limiting the propagation of errors and setting an upper bound to the numbers that are manipulated. In any case, this does not limit the scope of the method, as the processed blocks can be concatenated to restore the original input. To continue the above example, if n=8, the input blocks are 01001110, 01010001 and 01111000, the first of which represents the character N or the number 78. The description below concentrates on the encoding of a single block of length n. A block of n bits can be used to store numbers between 0 and 2n−1 in what is commonly called the standard binary representation, based on a sum of different powers of 2. Any number x in this range can be uniquely represented by the string bn−1bn−2⋯b1b0, with bi∈{0,1}, such that x=∑i=0n−1bi2i. But this is not the only possibility. Actually, there are infinitely many binary representations for a given integer, each based on a different numeration system [11]. The numeration system used for the standard representation is the sequence of powers of 2: {1,2,4,8,…}. Another popular and useful numeration system in this context is based on the Fibonacci sequence: {1,2,3,5,8,13,…}. Fibonacci numbers are defined by the following recurrence relation: Fi=Fi−1+Fi−2fori≥1, and the boundary conditions F0=1andF−1=0. Any integer x can be decomposed into a sum of distinct Fibonacci numbers; it can, therefore, be represented by a binary string crcr−1⋯c2c1 of length r, called its Fibonacci or Zeckendorf representation [12], such that x=∑i=1rciFi. This can be seen from the following procedure producing such a representation: given the integer x, find the largest Fibonacci number Fr smaller or equal to x; then continue recursively with x−Fr. For example, 49=34+13+2=F8+F6+F2, so its binary Fibonacci representation would be 10100010. Moreover, the use of the largest possible Fibonacci number in each iteration implies the uniqueness of this representation. Note that as a result of this encoding procedure, there are never consecutive Fibonacci numbers in any of these sums, implying that in the corresponding binary representation, there are no adjacent 1s. This property of the appearance of a 1-bit implying that the following bit, if it exists, must be a zero, has been exploited in several useful applications: robustness to errors [13], the design of Fibonacci codes [14], direct access [15], fast decoding and compressed search [10, 16], compressed matching in dictionaries [17], faster modular exponentiation [18], etc. The present work is yet another application of this idea. Extensions of the Fibonacci sequence have already been used in other coding applications, like in the error-correcting code suggested in [19], supporting multiple insertions and deletions. The repeated encoding will be performed in three steps: Encoding the data of the first round; Preparing the data block for a possible second encoding; Encoding the (new) data of the second round, overwriting the previous data. In the first step, the n bits of the block are transformed into a block of size r by recoding the integer represented in the input block into its Fibonacci representation. The resulting block will be longer, since more bits are needed, but also sparser, because of the property prohibiting adjacent 1s. To get an estimate of the increase in the number of bits, note that the largest number that can be represented is y=2n−1. The index r of the largest Fibonacci number Fr≈ϕr+1/5 needed to represent y is r=⌊logϕ(5y)−1⌋≃⌊(logϕ2)n+logϕ5−1⌋≃⌊1.44n−0.67⌋, where ϕ=1+52 is the well-known golden ratio. The storage penalty incurred by passing from the standard binary representation to the Fibonacci representation is thus 44%, for any block size n. Note that since n is fixed in our discussion, this is also true for r≃(logϕ2)n≃1.44n; the blocks in both representations may thus contain leading zeroes. The second step is supposed to be performed after the data written in the first round has finished its life cycle and is not needed any more, but instead of overwriting it by first erasing the entire block, we wish to be able to reuse the block subject to the update constraints of flash memory. The step is not needed for the correctness of the procedure and therefore seems optional, with a potential to increase the number of data bits that can be stored in the second round. Yet we show at the end of this section that the step is essential. In the second step, a maximal number of 1-bits is added without violating the non-adjacency property of the Fibonacci encoding. This means that short runs of zeros limited by 1-bits, like 101 and 1001, are not touched, but the longer ones, like 100001 or 1000001, are changed to 101001 and 1010101, where the added bits are boldfaced. In general, in a run of zeros of odd length 2i+1, every second zero is turned on, and this is true also for a run of zeros of even length 2i, except that for the even length the last bit is left as zero, since it is followed by a 1. A similar strategy is used for a run of leading zeros in the block: a run of length 1 is left untouched, but longer runs, like 001, 0001 or 00001, are changed to 101, 1001 and 10101, respectively. As a result of this filling strategy, the data block still does not have any adjacent 1s, but the lengths of the 1-limited zero-runs are now either 1 or 2, and the length of the leading run is either 0 or 1. In the third step, new data are encoded in the bits immediately to the right of every 1-bit. Since it is known that these positions contained only zeros at the end of Step 2, they can be used at this stage to record new data, and their location can be identified. The data block at the end of the third step thus contains bits of three kinds: separator bits (S), data bits (D) and extension bits (E). An E-bit is the second zero in a 0-run of length two at the end of the second step. The first bit of the blocks is either an S-bit, if it is 1, or an E-bit, if it is 0 (which can only occur if the leading zero-run was of length 1). S-bits have value 1 and are followed by D-bits; D-bits have value 0 or 1 and are followed by an S-bit (1) or by an E-bit (0); E-bits have value 0 and are followed by S-bits. Figure 1 continues the running example, showing the data block at the end of each of the steps. The strings are partitioned into blocks of 8 bits just for visual convenience. The input is the character string NQx, and the 24-bit numerical value 5 132 664 of its ascii encoding (with leading zeros), is given in Fibonacci encoded form with ⌊1.44×24−0.67⌋=33bits in the first line. The second line displays the block at the end of Step 2, after having added some 1-bits which are boldfaced. The data bits that can be used in the next step are those immediately to the right of the 1-bits and are currently all zero. In this example, there are 14 such bits. For the last step, suppose the new data to be stored are the number 7777, whose standard 14-bit binary representation is 01111001100001. These bits are interspersed into the data bits of the block, resulting in the string appearing in the third line, in which these data bits are boxed. The fourth line classifies the bits of the resulting string into S-, D- and E-bits. In this example, the combined number of information bits is 24 for the string NQx, plus 14 for the number 7777, that is 38 bits, but using only 33 bits of storage, yielding a compression ratio of 3833=1.152. Note also that all the changes from one step to another are consistent with the flash memory constraints, namely that only changes from 0 to 1 are allowed, but not from 1 to 0. FIGURE 1. View largeDownload slide Encoding example. FIGURE 1. View largeDownload slide Encoding example. Decoding at the end of the first step is according to Fibonacci codes, as in [10], and decoding of the data of the second round at the end of the third step can be done using the decoding automaton appearing in Fig. 2. An initial state I is used to decide whether to start in state S if the first bit is a 1, or in state E if it is a zero. The states S, D and E are entered after having read an S-bit, D-bit and E-bit, respectively. Only bits leading to state D are considered to carry information. There is no edge labeled 0 emanating from state E, because E-bits are always followed by 1s. FIGURE 2. View largeDownload slide Decoding automaton for data in the second round. FIGURE 2. View largeDownload slide Decoding automaton for data in the second round. Since at the end of the second step, no run of zeros can be longer than 2, the worst-case scenario is when every third bit is a separator. Any block is then of the form SDESDE⋯, and one-third of the bits are data-bits. The number of data bits in the third step is thus 1.44n/3=0.48n, which together with the n bits encoded in the first step, yield 1.48n, 2.76% more than the 1.44n storage bits used. Thus even in the worst case, there is a gain, albeit a small one. The maximal possible benefit will be in the case whenthere are no E-bits at all, that is, the block is of the form SDSDSD⋯. In this case, half of the bits are D-bits, and the compression ratio will be n+(logϕ2)n2(logϕ2)n=1logϕ2+12≃11.44+12=1.194. The analysis of the average gain is deferred to the more general case in the next section. A closer look at the second step, that of inserting additional 1-bits, which at first sight seemed merely as a technical improvement to increase the potential number of data bits, reveals that it is in fact critical to assure any gain at all. Theorem If the second step of the encoding procedure is omitted, there will be no improvement in the compression ratio. Proof If the second step is omitted, and we use as data bits of the second round only the bits following the 1-bits of the original Fibonacci encoding, then the following loop shows how to restore the data of both writing rounds: while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y while not end of string do x← next bit A←A∥x if x=1 and not end of string then y← next bit A←A∥0 B←B∥y A and B are initially empty strings, and ∥ stands for concatenation. After applying the loop to an encoded string, A contains the Fibonacci encoding of the first round, and B is the binary data of the second round. Yet, the total information conveyed by n input bits, no matter of how they have been partitioned, cannot possibly exceed the theoretical information content of n bits, or, in other words, if all the data are accessible, the compression ratio, as defined above, cannot be larger than 1.□ In other words, if at the end of the second round of a rewriting method with ratio >1, we are still able to recover all the data of the first writing round, this scheme could have been used a priori to produce a code being able to encode more messages than given by the information theoretic bound. This means that to get a real gain by rewriting codes, it is necessary to destroy at least a part of the data of the first writing round. We conclude that the insertion of additional 1s in the second step, after which the code used in the first step is not accessible any more, is not only a correcting step to improve the compression but is essential to enable any possible gain. 3. GENERALIZATION BASED ON EXTENSIONS OF FIBONACCI CODES One can extend the idea above to a code in which there are at least m 0-bits, for m≥1, between any two 1-bits. Such a code for m=2 has been designed for the encoding of data on CD-ROMs [20] and is known as Eight-to-Fourteen-Modulation: every byte of 8 bits is mapped to a bit-string of length 14 in which there are at least two zeros between any two 1s. In fact, the Fibonacci code and its extensions are special cases of a larger class known as Run-Length Limited codes [21], which impose lower and upper bounds on the number of zeros separating 1s. The lower bound on the number of zeros is obtained by representing numbers according to the basis elements of numeration systems which are extensions of the Fibonacci sequence, based on the following recurrences: Ak(m)=Ak−1(m)+Ak−m(m)fork>m+1, and the boundary conditions Ak(m)=k−1for1<k≤m+1. In particular, Ak(2)=Fk−1 are the standard Fibonacci numbers. The first few elements of the sequences A(m)≡{1=A2(m),A3(m),A4(m),…} for 2≤m≤6 are listed in the right part of Table 1. Table 1. Generalization of Fibonacci-based numeration systems. m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 View Large Table 1. Generalization of Fibonacci-based numeration systems. m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 m φm am lnφm2 A2(m),A3(m),A4(m),… 2 1.6180 0.8541 1.4404 1 2 3 5 8 13 21 34 55 89 144 3 1.4656 0.7614 1.8133 1 2 3 4 6 9 13 19 28 41 60 88 4 1.3803 0.6946 2.1507 1 2 3 4 5 7 10 14 19 26 36 50 5 1.3247 0.6430 2.4650 1 2 3 4 5 6 8 11 15 20 26 34 6 1.2852 0.6016 2.7625 1 2 3 4 5 6 7 9 12 16 21 27 34 View Large A closed form expression of the elements of the sequence A(m) can be obtained by considering the characteristic polynomial xm−xm−1−1=0, and finding its m roots φm,1,φm,2,…,φm,m. The element Ak(m) is then a linear combination of the kth power of these roots. For these particular polynomials, when m>2, there is only one root, say φm,1≡φm, which is real and is larger than 1, all the other roots are complex numbers a+ib with b≠0 and with norm strictly smaller than 1. For m=2, the second root 1−52=−0.6180 is also real, but its absolute value is <1. It follows that with increasing k, all the terms φm,jk, 1<j≤m, quickly vanish, so that the elements Ak(m) can be accurately approximated by powers of the dominant root φm alone, with appropriate coefficients, Ak(m)≈amφmk−1. The constants am and φm are listed in Table 1. For a given m, any integer x can be decomposed into a sum of distinct elements of the sequence A(m); it can therefore be uniquely represented by a binary string crcr−1⋯c3c2 of length r−1, such that x=∑i=2rciAi(m), using the recursive encoding method presented above, based on finding in each iteration the largest element of the sequence fitting into the remainder. For example, 36=28+6+2=A10(3)+A6(3)+A3(3), so its representation according to A(3) would be 100010010. As a result of the encoding procedure, the indices i1,i2,… of the elements in the sum x=∑i=2rciAi(m) for which ci=1 satisfy that ik+1≥ik+m, for k>2. In the above example x=36, these indices are 3, 6 and 10. This means that in the corresponding binary representation, there are at least m−1 zeros between any two 1s. Extending the calculation of the increase in the number of bits to the general case, recall that the largest number that can be represented is y=2n−1. The largest A(m) number Ar(m)≈amφmr−1 needed to represent y is r=⌊logφm(y/am)+1⌋=⌊(logφm2)n−logφmam+1⌋. The storage penalty incurred by passing from the standard to the A(m) representation is thus at most a factor of (logφm2), for any block size n. This will be 44% for the Fibonacci numbers ( m=2), and 81% and 115% for their extensions, with m=3 and m=4, respectively, as listed in the fourth column of Table 1. The encoding according to A(m) will be performed in three steps similar to those described for m=2 in the previous section. In the first step, the n bits of the block are transformed into a block of size r≃(logφm2)n by recoding the integer represented in the input block into its representation according to A(m). In the second step, a maximal number 1-bits is added without violating the property of having at least m−1 zeros between 1s. This means that in a run of zeros of length j, limited on both sides by 1s, with j≥2m−1, the zeros in positions m,2m,…,⌊j−m+1m⌋m are turned on. For a run of leading zeros of length j (limited by a 1-bit only at its right end), for j≥m, the zeros in positions 1,m+1,2m+1,…,⌊j−mm⌋m+1 are turned on. Another special case is the run of zeros following the rightmost 1-bit. For example, for A(3), 100000000001 is turned into 100100100001, 00000001 is turned into 10010001, and 10000 is turned into 10010. As a result of this filling strategy, the data block still does have at least m−1 zeros between 1s, but the lengths of the 1-limited zero-runs are now between m−1 and 2m−2, and the lengths of the leftmost and rightmost runs are between 0 and m−1. In the third step, new data are encoded in the m−1 bits immediately to the right of every 1-bit. Since it is known that these positions contained only zeros at the end of Step 2, they can be used at this stage to record new data, and their location can be identified. For the rightmost 0-run, the number of data-bits may be smaller than m−1, for example, for A(3) if the last run was 10000, it was transformed to 10010 and only one data bit may be added after the rightmost 1. To continue the analogy with the case m=2, there are now data bits of different kinds 𝖣1 to 𝖣m−1, and similarly for extension bits 𝖤1 to 𝖤m−1. The decoding of the data of the second round at the end of the third step for A(3) can be done using the decoding automaton appearing in Fig. 3. An initial state I is used to decide whether to start in state S if the first bit is a 1, or in state 𝖤1 if it is a zero. Only bits leading to states 𝖣1 or 𝖣2 are considered to carry information. There is no edge labeled 0 emanating from state 𝖤2, because a second E-bit is always followed by 1s. Similar decoding automata, with states I, S, 𝖣1 to 𝖣m−1 and 𝖤1 to 𝖤m−1 can be designed for all m≥2. Figure 3. View largeDownload slide Decoding automaton for data in the second round with A(3). Figure 3. View largeDownload slide Decoding automaton for data in the second round with A(3). 4. SPACE ANALYSIS 4.1. Worst case—lower bound of performance Since at the end of the second step, no run of zeros can be longer than 2m−2, the worst-case scenario is when every (2m−1) th bit is a 1-bit. Any block is then of the form 𝖤𝖤⋯𝖤𝖲𝖣𝖣⋯𝖣𝖤𝖤⋯𝖤𝖲𝖣𝖣⋯𝖣𝖤𝖤⋯𝖤𝖲, where all the runs of Ds and Es are of length m−1 and, in the limit, when the block size tends to infinity so that the initial run of E-bits and the final S-bit may be neglected, (m−1)/(2m−1) of the bits are data-bits. The worst-case compression factor is thus n+(m−1)(logφm2)n2m−1(logφm2)n=1logφm2+m−12m−1, (1) where the first and second summands represent, respectively, the contribution of the first and second writing rounds. This worst-case scenario corresponds to a block size r after the first step, which is a multiple of 2m−1, plus a final S-bit and m−1 initial E-bits, that is, r≡m(mod(2m−1)); for each such r, there is exactly one possible input string yielding this worst case, e.g. for m=3, 0010000100001 of length r=13, which corresponds to the value 70 written in A(3), for an initial input string 01000110 of length n=8. 4.2. Best case—upper bound of performance The maximal possible benefit will be in the case when there are no E-bits at all, that is, the block is of the form 𝖲𝖣𝖣⋯𝖣𝖲𝖣𝖣⋯𝖣𝖲𝖣⋯, where all the runs of Ds are of length m−1 and the number of data-bits is (m−1)/m. In this case, the compression ratio will be 1logφm2+m−1m. (2) For example, for m=2, the best case for r=10 is X=1010101010, which corresponds to the value 143 of an initial input string 10001111 of length n=8. The best case for a given length r, however, is not unique; in fact, it is shared by all the input strings which after adding the 1-bits of the second step will be of the above form. To continue the last example, some of the strings in A(2), other than X, of length r=10 yielding five data bits are 1000101010, 1010000000 or even 0000000000, corresponding, respectively, to input strings with values 109, 123 and 0; all these strings turn into X when the 1-bits of the second step are added. 4.3. Average case Similarly to the above, the compression ratio will be 1logφm2+m−1Dm, where Dm is a random variable designing the distance in bits between consecutive 1ʼs at the end of the second step. Thus D2 is either two or three, and generally, Dm assumes values between m and 2m−1, inclusive. We are interested in the expected value, which is 1logφm2+Em−1Dm=1logφm2+(m−1)∑j=0m−1Prob(Dm=m+j)m+j. (3) To evaluate Equation (3), we proceed by the following steps. First, the probability of a 1-bit in the given context is established. We then specify the probabilistic model on the basis of which the average is defined, and evaluate the probability distribution of the distance between 1-bits according to this model. 4.3.1. Probability of a 1-bit The constraint of the A(m) encoding implies that the probabilities of occurrence of 0s and 1s are not the same, as would be the case in the standard binary encoding, when all possible inputs are supposed to be equi-probable. To calculate the average compression ratio, one needs first to know the probability of occurrence of a 1-bit in the binary representation according to A(m). Let us refer to binary strings representing integers according to A(m) as A(m)-strings. Thus 10010 is an A(3)-string, but 01010 is not. Define Tr(m) as the number of different A(m)-strings of length r, and Qr(m) as the total number of 1s in all the A(m)-strings of length r, r≥0. For example, the sets of A(3)-strings of lengths up to 4 are {0,1}, {00,01,10}, {000,001,010,100} and {0000,0001,0010,0100,1000,1001}. In general, one gets that T0(m)=0,Tr(m)=r−1for0<r≤m,andTr(m)=Tr−1(m)+Tr−m(m)forr>m, since the A(m)-strings of length r can be generated by either prefixing an A(m)-string of length r−1 with 0, or by prefixing an A(m)-string of length r−m, with 10⋯0 (1 followed by m−1 zeros). In fact, one gets that Tr(m)=Ar+1(m) for r≥1. Similarly, for Qr(m), one gets the recurrence Q0(m)=0,Qr(m)=rfor0<r≤m,andQr(m)=Qr−1(m)+Qr−m(m)+Tr−m(m)forr>m. We again split the set of A(m)-strings of length r into those with leading 0 and those with leading 10⋯0 (1 followed by m−1 zeros); removing these leading bits, we are left in the former set with A(m)-strings of length r−1, which contribute Qr−1(m) 1-bits, and in the latter set with Tr−m(m) A(m)-strings of length r−m, which contribute Qr−m(m) 1-bits, to which the Tr−m(m) 1s in the removed prefixes of the form 10⋯0 have to be added. To evaluate the probability of a 1-bit in the representation according to A(m), which we shall denote by pm′, the number of 1-bits has to be divided by the total number of bits, both limited to A(m)-strings of length r, that is Qr(m)/(rTr(m)). The limiting values of these ratios, when r→∞, appear in the second column of Table 2, for 2≤m≤6. Table 2. Compression ratios with A(m). m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 View Large Table 2. Compression ratios with A(m). m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 m pm′ –prob of 1-bit Best case Worst case Average case Ratio Compr Ratio Compr Compr 2 0.2763 1/2 1.194 1/3 1.028 1.145 3 0.1945 2/3 1.218 2/5 0.952 1.127 4 0.1511 3/4 1.215 3/7 0.894 1.098 5 0.1240 4/5 1.206 4/9 0.850 1.072 6 0.1055 5/6 1.195 5/11 0.817 1.045 View Large 4.3.2. Probabilistic model For the current calculation, we assume that the bit-string at the end of the first step has been created by a stochastic process in which the bits are generated from left to right, independently from each other, with some probability pm for generating a 1-bit, every 1-bit being immediately followed by m−1 zeros, and with probability hm=1−pm for a zero-bit in any position which is not within the m−1 immediately following a 1-bit. The probabilities pm will be chosen so that this process shall produce bit strings with at least m−1 0s between 1s and with probability of occurrence of a 1-bit in the entire string being pm′ and the corresponding probability of a 0-bit being 1−pm′. Consider a block of n bits. The number of 1-bits is expected to be npm′, and thus npm′(m−1) 0-bits are expected to be forced and not generated by the process. The expected number of generated bits is therefore n−npm′(m−1)=n(1−(m−1)pm′), of which npm′ bits should be 1s. One can thus derive the probabilities of a 1-bit and a 0-bit in the generating process as: pm=pm′1−(m−1)pm′ and hm=1−pm=1−mpm′1−(m−1)pm′. Both the output of the encoding process and the stochastic process assumed in the model thus produce the same set of binary strings, and according to the same probabilities. 4.3.3. Probability distribution of distance between 1-bits Using these assumptions, let us calculate the probability distribution of the distance between consecutive 1-bits, which is the expected length of a zero-run including the terminating 1-bit in the data block at the end of the second step. The distance between consecutive 1s is mostly m, but sometimes it is m+1, m+2, etc., up to at most 2m−1. It will be m+j, for 0≤j<m, if there is a run of 0s of length mk+j−1, with k≥1, between the 1-bits at the end of the first step, as, for m=3, in 10001 or 10000001 for j=1, or in 100001 and 100000001 for j=2. Recall that the first m−1 zeros are forced by the leading 1, so that the probability of a run of km+j−1 zeros, for k≥1, is hkm+j−mp (we have shortened hm and pm to h and p for clarity). After inserting the additional 1-bits in the second step, such a first-step-run of km+j−1 zeros followed by a 1 will be transformed into a sequence of k elements: k−1 m-tuples 00⋯01 ( m−1 zeros followed by 1) and a single m+j-tuple of m+j−1 zeros followed by a 1, for 0≤j<m. The distance between consecutive 1s will thus increase from m to m+j, for j>0, only if the original run (at the end of the first step) was of length mk+j, for k≥1, and for such a run, only one of the k elements will be a m+j tuple. Assuming an infinitely long input block, the probability of getting a distance between 1s larger than m is thus Prob(Dm>m)=∑j=1m−1∑k=1∞1khmk−m+jp=∑j=1m−1phm−j∑k=1∞hmkk=∑j=1m−1phm−jln11−hm, where we have used the identity ln11−x=∑k=1∞xkk [22, Section 1.2.9, Equation (25)]. In all other cases, that is, for j=0 and for the k−1 first elements when j>0, the distance between consecutive 1s will be m, and the probability for this event will be Prob(Dm=m)=1−Prob(Dm>m). To clarify this point, consider m=3 and the following two runs of zeros at the end of Step 1, i.e. after the A(3) encoding, corresponding to distances 9=3·3+0 and 17=3·5+2 between consecutive 1s, respectively: 1000000001 100000000000000001. At the end of the second step, after having inserted the boldfaced 1s, these strings are transformed into 100𝟭00𝟭001 100𝟭00𝟭00𝟭00𝟭00001, and only the last block of the second string is of length 5=m+2, whereas the first four blocks of the second string, and all the blocks of the first are of length 3=m. The expected value E1Dm needed in Equation (3) will therefore tend to Prob(Dm=m)m+∑j=1m−1p(m+j)hm−jln11−hm. 4.4. Analytical results The worst, best and average compression ratios, according to Equations (1–3), as well as the probability of a 1-bit, from which the average case can be derived, are listed in Table 2, for 2≤m≤6. The best values for each case are emphasized in bold. For the best and worst cases, the columns headed ratio show the proportion of data-bits used in the second round relative to the total number of bits. As can be seen, for the best and average cases there is always a gain relative to the baseline, at least for the given values of m, and in the worst case only for m=2. Interestingly, while m=2 is best in the worst and average cases, the highest value in the best case is obtained for m=3. If one extends the definition of the sequence A(m) also to m=1, one gets the sequence of powers of 2, that is, the standard binary numeration system, with no restrictions on the appearance of 1-bits. The compression ratio in that case will be 1. For higher values of m, the combined compression ratio may be higher, but the proportion of the first round data will be smaller. Table 3 brings these proportions for 1≤m≤6. One way to look at these results is thus to choose the order m of the encoding according to the partition between first and second round data one may be interested in. This is an advantage over previously known rewriting systems, in which the contribution of each round of writing is rigidly fixed in advance, like for the RS-WOM-scheme with (0.5, 0.5). Table 3. Proportions of first and second round data bits. m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 View Large Table 3. Proportions of first and second round data bits. m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 m 1 2 3 4 5 6 First round 1.00 0.61 0.49 0.42 0.38 0.34 Second round 0.00 0.39 0.51 0.58 0.62 0.66 View Large 5. EXPERIMENTAL RESULTS All the numbers mentioned above have been derived analytically. To confirm a similar performance on real life applications, we decided to run the following tests on a variety of input files of different size and nature. This also serves to double-check that the assumptions of our probabilistic model were realistic. The first file (ran) consists of a sequence of 100000 random numbers between 0 and 255. The second file (bib) is a cleaned copy of the King James version of the Bible in English, from which all punctuation signs have been stripped. The third file (mis) is the Ebook version of Les Misérables by Victor Hugo in French, downloaded from the Gutenberg Project.1 The fourth file (zip) is the gzip compressed version of the third file. All the files were processed in blocks of n=400bits, that is, 50 bytes, and for each such block, the represented n-bit number was first transformed into the corresponding A(m)-string of length r=(logφm2)n, and the 1-bits of Step 2 have been added. The number db of possible data-bits for the second round was then evaluated, with the appropriate adaptations at the extremities of the block: every 1-bit contributes m−1 data-bits, except possibly the last one if there are less than m−1 0-bits after the rightmost 1. The compression ratio for each block is then (n+db)/r, and these values were averaged. The results appear in Table 4. The second column gives the size of the input files in Mbytes, and the other columns, the averaged compression ratio for 2≤m≤6. The last line repeats the analytic result of the theoretical average from the last column of Table 2, and the other lines correspond to the different input files. Table 4. Empirical compression ratios on test files. File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 View Large Table 4. Empirical compression ratios on test files. File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 File Size Compression ratio m=2 m=3 m=4 m=5 m=6 ran 0.10 1.1415 1.1207 1.0891 1.0605 1.0374 bib 3.10 1.1411 1.1201 1.0883 1.0593 1.0363 mis 0.68 1.1410 1.1197 1.0881 1.0596 1.0365 zip 0.26 1.1413 1.1202 1.0887 1.0603 1.0369 theor. avg 1.1446 1.1272 1.0983 1.0720 1.0449 View Large These results suggest that the rewriting compression ratio is practically independent of the nature of the input file, be it pure ascii in English or French, a compressed file or even a randomly generated one: the fluctuations of the ratio for a given value of m were below one-tenth of a percent. The results also confirm that the assumed probabilistic model is realistic, as the theoretical ratios differ from the observed ones by about 1% or less. The fact that the empirical ratios are consistently lower can be explained by the approximations in the theory, in particular, letting the block size n tend to infinity in the derivation of the probability pm′ of a 1-bit and when calculating E(1/Dm). 6. USAGE AS COMPRESSION BOOSTER The above ideas can be used to build a compression booster in the following way. Suppose we are given a rewriting system S allowing k rounds. This can be turned into a system with k+1 rounds by using, in a first round, the new encoding as described earlier, which identifies a subset of the bits in which the new data can be recorded. These bits are then used in k additional rounds according to S. Note that only context-insensitive systems, like the RS-WOM-code, can be extended in that way. Certain codes, like those suggested in [7], may be used with any number k of writing rounds, so for them, this boosting scheme is not relevant. But for other codes, the number of rounds is fixed and cannot be changed, and then this is a method to increment the number of rounds. Since the first round recodes the data using more bits, the extension with an additional round of rewriting will not always improve the compression. For example, for the A(m) codes, even if the second term of the numerator of the left hand side of Equation (1), representing the number of bits used in the second round, is multiplied by 43, the compression factor of the RS-WOM-code, one still gets only at most 1.319 on the average, for A(3), about 1.1% less than the RS-WOM-code used alone. However, for certain instances, using some A(m) code in the first round, followed by two rounds of RS-WOM, may yield better codes than RS-WOM, as can be seen in the example in Fig. 4. More generally, suppose some rewriting code is given, yielding an overall compression ratio of α in k rounds. The worst-case ratio for the boosted code with k+1 rounds is then 1logφm2+αm−12m−1, and we seek values of m, if they exist at all, for which this ratio is larger than α itself. This condition is equivalent to 2m−1mlog2φm>α. Similar constraints can be derived for the average and best cases. FIGURE 4. View largeDownload slide Encoding example with A(3). FIGURE 4. View largeDownload slide Encoding example with A(3). Figure 4 brings the same running example as above, this time for A(3), and coupled with the RS-WOM-code. The same input character string NQx is used, and the 24-bit numerical value 5 132 664 of its ascii encoding (with leading zeros), is given in A(3) encoded form with 40 bits in the first line. The second line displays the block at the end of Step 2, after having added, for this particular example, a single 1-bit, which is boldfaced. The data bits that can be used in the next step are the pairs immediately to the right of the 1-bits and are currently all zero. In this example, there are 24 such bits. There are thus 24+24=48 data bits in the two rounds together, using only 40 storage bit, which gives a ratio of 1.2. For the next step, suppose the new data to be stored in the second round are the number 55,555, and in the third round the number 44,444, whose standard 16-bit binary representations are 11 011001 00 0000 11 and 10 10 11 01 10 01 11 00, respectively. The RS-WOM-code considers these numbers as a sequence of pairs (the spaces have only been added for clarity), each of which is translated into a triplet, yielding two 24-bit strings 001 100 010 100 000 000 000 001 and 101101110100101011110111. These bits are interspersed into the data bits of the block, resulting in the strings appearing in the third and fourth lines, in which these data bits are boxed in pairs. The fifth line shows again the classification of the resulting string into S-, D- and E-bits. In this example, the combined number of information bits is 24 for the string NQx, plus 16 for each of the numbers 55,555 and 44,444, that is 56 bits, but using only 40 bits of storage, yielding a compression ratio of 1.4. Using the RS-WOM-code alone with 40 bits would only be able to store 53.3 bits of information. Note also that as before, all the changes from one step to another are consistent with the flash memory constraints, namely that only changes from 0 to 1 are allowed, but not from 1 to 0. 7. CONCLUDING REMARKS We have shown how the Fibonacci code and some of its generalizations can be used to devise a rewriting code suitable for flash memory. Our analysis shows that some gain may be achieved, though the performance cannot compete with state of the art rewriting codes, as in [8], or even the simple RS-WOM-code. The novelty of our approach and its main contribution is in the introduction of the new paradigm of context-sensitive rewriting codes, of which the Fibonacci codes are just an example. The new challenge is, therefore, to find other codes in which the appearance of the bits obeys some rules that can be exploited to encode more data in subsequent writing rounds, with the additional constraint that the overall compression ratio should be greater than 1, and if possible, better than the ratio of the currently known best rewriting codes. Examples are abundant: other RLL codes [21] with various parameters, or the set defined in [23] of all strings of the form y=xπ, where π is some fixed bit pattern of length at least one bit, x is any bit-string, and π occurs in y only as suffix. The Fibonacci code is the special case π=11. A more recent example can be found in [24], which extends the set of code words by allowing multiple delimiters. As the authors report better compression performance than for the Fibonacci codes, one may also expect an improvement for the corresponding rewriting codes. Many of these codes and similar ones, or some combination thereof, may be adapted to form a rewriting code with two or more rounds, following the basic techniques presented in this paper. References 1 Assar , M. , Nemazie , S. and Estakhri , P. ( 1995 ) Flash memory mass storage architecture. US Patent 5,388,083, issued Feb. 7, 1995. 2 Gal , E. and Toledo , S. ( 2005 ) Algorithms and data structures for flash memories . ACM Comput. Surv. , 37 , 138 – 163 . Google Scholar CrossRef Search ADS 3 Rivest , R.L. and Shamir , A. ( 1982 ) How to reuse a ‘write-once’ memory . Inf. Control , 55 , 1 – 19 . Google Scholar CrossRef Search ADS 4 Fiat , A. and Shamir , A. ( 1984 ) Generalized ‘write-once’ memories . IEEE Trans. Inf. Theory , 30 , 470 – 479 . Google Scholar CrossRef Search ADS 5 Jiang , A. , Bohossian , V. and Bruck , J. ( 2010 ) Rewriting codes for joint information storage in flash memories . IEEE Trans. Inf. Theory , 56 , 5300 – 5313 . Google Scholar CrossRef Search ADS 6 Kurkoski , B.M. ( 2010 ) Rewriting codes for flash memories based upon lattices, and an example using the E8 lattice. IEEE Globecom Workshop on Applications of Communication Theory to Emerging Memory Technologies, ACTEMT 2010, Miami, Florida, 6–10 December 2010, pp. 1861–1865. IEEE. 7 Burshtein , D. and Strugatski , A. ( 2013 ) Polar write once memory codes . IEEE Trans. Inf. Theory , 59 , 5088 – 5101 . Google Scholar CrossRef Search ADS 8 Shpilka , A. ( 2013 ) New constructions of WOM codes using the Wozencraft ensemble . IEEE Trans. Inf. Theory , 59 , 4520 – 4529 . Google Scholar CrossRef Search ADS 9 Yaakobi , E. , Kayser , S. , Siegel , P.H. , Vardy , A. and Wolf , J.K. ( 2012 ) Codes for write-once memories . IEEE Trans. Inf. Theory , 58 , 5985 – 5999 . Google Scholar CrossRef Search ADS 10 Klein , S.T. and Ben-Nissan , M.K. ( 2010 ) On the usefulness of Fibonacci compression codes . Comput. J. , 53 , 701 – 716 . Google Scholar CrossRef Search ADS 11 Fraenkel , A.S. ( 1985 ) Systems of numeration . Amer. Math. Mon. , 92 , 105 – 114 . Google Scholar CrossRef Search ADS 12 Zeckendorf , E. ( 1972 ) Représentation des nombres naturels par une somme des nombres de Fibonacci ou de nombres de Lucas . Bull. Soc. Roy. Sci. Liège , 41 , 179 – 182 . 13 Apostolico , A. and Fraenkel , A.S. ( 1987 ) Robust transmission of unbounded strings using Fibonacci representations . IEEE Trans. Inf. Theory , 33 , 238 – 245 . Google Scholar CrossRef Search ADS 14 Fraenkel , A.S. and Klein , S.T. ( 1996 ) Robust universal complete codes for transmission and compression . Discrete Appl. Math. , 64 , 31 – 55 . Google Scholar CrossRef Search ADS 15 Klein , S.T. and Shapira , D. ( 2016 ) Random access to Fibonacci encoded files . Discrete Appl. Math. , 212 , 115 – 128 . Google Scholar CrossRef Search ADS 16 Klein , S.T. and Shapira , D. ( 2016 ) Compressed matching for feature vectors . Theor. Comput. Sci. , 638 , 52 – 62 . Google Scholar CrossRef Search ADS 17 Klein , S.T. and Shapira , D. ( 2011 ) Compressed matching in dictionaries . Algorithms , 4 , 61 – 74 . Google Scholar CrossRef Search ADS 18 Klein , S.T. ( 2008 ) Should one always use repeated squaring for modular exponentiation? Inf. Process. Lett. , 106 , 232 – 237 . Google Scholar CrossRef Search ADS 19 Helberg , A.S.J. and Ferreira , H.C. ( 2002 ) On multiple insertion/deletion correcting codes . IEEE Trans. Inf. Theory , 48 , 305 – 308 . Google Scholar CrossRef Search ADS 20 Immink , K.A. , Nijboer , J.G. , Ogawa , H. and Odaka , K. ( 1985 ) Method of coding binary data. US Patent 4,501,000, issued Feb. 19, 1985. 21 Schouhamer Immink , K.A. ( 1990 ) Runlength-limited sequences . Proc. IEEE , 78 , 1745 – 1759 . Google Scholar CrossRef Search ADS 22 Knuth , D.E. ( 1973 ) The Art of Computer Programming, Volume I: Fundamental Algorithms ( 2nd edn ). Addison-Wesley , Boston . 23 Lakshmanan , K.B. ( 1981 ) On universal codeword sets . IEEE Trans. Inf. Theory , 27 , 659 – 662 . Google Scholar CrossRef Search ADS 24 Anisimov , A.V. and Zavadskyi , I.O. ( 2017 ) Variable-length prefix codes with multiple delimiters . IEEE Trans. Inf. Theory , 63 , 2885 – 2895 . Footnote 1 //www.gutenberg.org/files/17489/17489-h/17489-h.htm Author notes Handling editor: Prudence Wong © The British Computer Society 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

The Computer JournalOxford University Press

Published: Mar 6, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off