Augustana logo

COMPUTING SCIENCE 350

Computer Organization and Architecture II


Error Control Codes



Error-Detecting and -Correcting Codes

In order to detect or correct transmission or storage errors, some additional, redundant bits, called "check bits", can be added to a number of data bits to construct a codeword.

Hamming Distance

The number of corresponding bits that differ between two codewords is the Hamming distance of those two codewords. (See R.W. Hamming, "Error Detecting and Error Correcting Codes," Bell System Technical Journal, vol. 29, pp. 147-160, April 1950.) For example, the Hamming distance between the codewords 1001 and 0101 is 2. The Hamming distance of two codewords can be calculated as the number of 1 bits in the bitwise exclusive-or of the two codewords: 1001 xor 0101 = 1100.

A code is the set of all codewords of a given length that are constructed by adding a specified number of check digits in a specified way to a specified number of data bits. The minimum Hamming distance of a code is the minimum of the Hamming distance between all possible pairs of codewords of that code. The following table indicates the Hamming distance between all pairs of a simple 4-bit binary code:

The Hamming distances between codewords of a simple 4-bit code.
   0000   0011   0101   0110   1001   1010   1100   1111 
 0000  - 2 2 2 2 2 2 4
0011 2 - 2 2 2 2 4 2
0101 2 2 - 2 2 4 2 2
0110 2 2 2 - 4 2 2 2
1001 2 2 2 4 - 2 2 2
1010 2 2 4 2 2 - 2 2
1100 2 4 2 2 2 2 - 2
1111 4 2 2 2 2 2 2 -

Because the minimum Hamming distance between any two codewords is 2, the Hamming distance of the code is 2.

Error Detection

If a code has a minimum Hamming distance of 2, then at least two bits of any codeword would have to be transmitted in error or changed in memory in order to yield another codeword. If only one bit changed, the result would not be a codeword, so the single-bit error could be detected.

Similarly, if a code has a minimum Hamming distance of 3, any fewer than 3 single-bit errors could be detected, since at least 3 bits must be flipped to change any codeword into another codeword of the same code.

To detect d single-bit errors, a code must have a minimum Hamming distance of at least d + 1.
Error Correction

In order to correct transmission or storage errors, the data block that results from erroneous changes to a codeword must be "closer" (in Hamming distance) to a particular codeword of the code than to any other codeword. For example, given the 4-bit code above, the data block 0100 could have resulted from a single-bit error in any of the codewords 0000, 0101, 0110, or 1100. According to the Hamming distance metric, 0100 is "halfway" between any pair of these codewords, so we can't tell which codeword it was originally. (Of course, if multi-bit errors occurred, it could originally have been any codeword.)

If a data block is closer to one particular codeword than to any of the other codewords of a code, and if we assume that only the minimum possible number of bit errors occurred, then we can correct these errors by using the closest codeword in place of the erroneous data block. Thus, error correction is possible only if the number of bit errors is less than half the minimum Hamming distance of the code.

The error correction capability of a code is floor(( Dmin - 1 ) / 2, where Dmin is the minimum Hamming distance of the code.
To correct d single-bit errors, a code must have a minimum Hamming distance of at least 2d + 1.

Parity

A parity bit is a check bit added to a data block such that the number of 1 bits in the resulting codeword is even (for even parity) or odd (for odd parity). The simple code presented above uses a single parity bit following the 3 data bits.

The use of a single parity bit yields a code with a minimum Hamming distance of 2, which can thus be used for single error detection.


Hamming Coding

Richard Hamming proposed an algorithm for constructing a single-bit error-correcting code for any size data word by using multiple parity bits to check overlapping subsets of the bits of the data word.

If the bits of a codeword are numbered from 1 to n, parity bits are located (at least conceptually) at all power-of-two bit positions, with the data bits interspersed in the remaining bit positions. For example, given an 8-bit data word, a Hamming codeword would be constructed by inserting parity-checking bits in bit positions 1, 2, 4, and 8, with the data bits occupying bit positions 3, 5, 6, 7, 9, ..., 12, as follows:

P1 P2 D3 P4 D5 D6 D7 P8 D9 D10 D11 D12

The parity bit in position k checks all bits with a 1 in bit position k of the binary representation of its bit position. To continue the 8-bit data word example:

P1 checks bits 1, 3, 5, 7, 9, 11.
P2 checks bits 2, 3, 6, 7, 10, 11.
P4 checks bits 4, 5, 6, 7, 12.
P8 checks bits 8, 9, 10, 11, 12.

This code has a minimum Hamming distance of 3, so it is capable of correcting single-bit errors. If a single-bit error occurs, the erroneous bit's position can be determined by adding together the bit positions of the parity bits that are now in error. For example, if the data block 110110010001 is received or retrieved, then parity bits 1 and 8 are correct if we assume even parity, but parity bits 2 and 4 are incorrect, as illustrated by the following, in which bit values are shown only in the bit positions checked by each parity bit:

1-0-1-0-0-0- is even
-10--00--00- is odd
---1100----1 is odd
-------10001 is even

Since parity bits 2 and 4 are incorrect, the single-bit error must have occurred in bit position 2 + 4 = 6, since only bit position 6 is checked by both parity bits 2 and 4 and only those parity bits. Thus, the correct code word must have been 110111010001, representing the data block 01100001 (which, if it represents an ASCII character, is the letter 'a').

The 12-bit Hamming code illustrated above added 4 check bits to the original 8 data bits, an overhead of 33%. It should be noted that single-error correction of up to 11 data bits could be accomplished with the same 4 parity bits, resulting in 15-bit codewords, an overhead of 27%. Furthermore, the overhead due to Hamming coding decreases as the size of the data word increases: it is only 2% for a 512-byte data block, which requires only 10 check bits according to Hamming's scheme.

Single Error Correction, Double Error Detection (SECDED)

Although Hamming coding has a minimum Hamming distance of 3, it is not always able to detect 2-bit errors. (Exception: If the sum of the parity bit positions that are in error yields a position beyond the end of the codeword, then it is clear that a multi-bit error has occurred.) However, by adding an additional whole-word parity bit, which might conceptually be placed in the leading bit position 0, a Hamming code can detect double-bit errors: if one or more of the parity bits P1, P2, P4, ... is in error but P0 is correct, then a double-bit (or greater) error has occurred.


Copyright © 2005 Jonathan Mohr

 
University of Alberta logo