Chapter 10 1. Calculate the first order entropy, H1, of the following passage from A Clockwork Orange:
“Our pockets were full of deng, so there was no real need from the point of crasting any more pretty polly to tolchock some old veck in an alley and viddy him swim in his blood while we counted the takings and divided by four, nor to do the ultraviolent on some shivering starry grey haired ptitsa in a shop and go off with the till’s guts. But, as they say, money isn’t everything.”
As this novel was set in the future, the value ought to be a bit higher than it is now. If it isn’t, try to explain why!
2. Have someone else select a sentence from a book so that you can guess at each letter until correct. Keep track of the number of guesses and use the result to estimate the entropy of the text. To be fair, the sentence selected should not be one you are already familiar with! If you recognize the line, start over with a new sentence.
3. What happens to the maximum possible entropy of a language when more characters are introduced?
4. Calculations for the first order entropy of English may be made using the standard 26 letter alphabet or based on a 27 symbol alphabet that includes the blank space between words as a character. If we use this larger character set, should the value we obtain for H1 be larger or smaller?
5. Will the first order entropy of a plaintext message change if it is enciphered with a monoalphabetic substitution cipher? How about the second order entropy?
6. Suppose a long message has been enciphered using matrix encryption. How could we use entropy calculations to determine the size of the matrix that was used?
7. What is the unicity point for 2x2 matrix encryption? What is it for 3x3 matrix encryption? How about the 4x4 case?
8. Bearing in mind that entropy measures disorder in a language, why do estimates of it using larger groups of characters (i.e., digraphs and trigraphs, as opposed to single characters) yield lower values?
9. How many possible meaningful keys should you expect there to be in English for a message 10 letters long that was enciphered using a running key cipher?
10. Shannon estimated the redundancy of English to be 50%. One method he referred to that yields this value is the ability of people to restore English sentences with half of the letters removed. Which of the following can you reconstruct?
a. OU ETTE WTC WHA YOU AY ABUT MY AR (26% removed)
b. THS SNTNC HS N VWLS ( 35% removed)
c. OE F Y AOIE ORR OES S ILD H HPY A (50% removed)
d. I RAIG S AE UH AIR Y H PEEC O RDNAC (50% removed)
e. BTA WS OE TE OT OUA CMC OK HRCE O
TE WNIT CNUY (50% removed)
f. SIIT WO A H TM T IE OK EPAN TR
OK H GRL UC R IG GE EVC TI I TN
HR E ON N TE NPRO T PRU SNE HEVS (60% removed)
11. Suppose the value of H1 for English continues to increase and that at some point in the future it reaches 4.5. At that point, what percentage of letters may be deleted from a typical English sentence without preventing the message from being recovered?
12. What effect does doubling the number of keys in a system have on the unicity point of a cipher?