*** Attacks on simple ciphers. *** Recall modes of attack (ciphertext only, known plaintext, chosen plaintext) Note that for all simple substitution ciphers, the preservation of original word boundaries significantly simplifies attack. Small words Puzzle-solving skills Easy to get rid of, only slight additional decryption difficulty so assume they are gone. Common to write characters in groups of 4-6 for readability. Caesar shift Ciphertext only: * Brute force works pretty well, even by hand, because of small number of keys. * Try each shift, and see if what emerges is intelligible. * Note that only first few characters generally need to be decrypted for each key. Known plaintext: Trivial. Take difference between single encrypted/plaintext pair. Chosen plaintext: Even more trivial. Feed in "x = 0". The result is the key. Affine cipher E_ab(x) = (ax + b) % 26 Ciphertext only: * Brute force reasonable (only) with computer help. * Use more general statistical approach we will develop shortly. Known plaintext: * Harder than Caesar, but not much. * Need two character equivalences E_ab(x), E_ab(y) to solve for two unknowns. E(x) - E(y) = a(x-y) % 26 solvable for a iff x-y has inverse mod 26, which is true iff it is not even or 13. b can then be recovered from b = (E_ab(x) - ax) % 26 * First example of a probabilistic approach. Works if we are "lucky" (if not, try again). Chosen plaintext: * Still pretty easy. Feeding in x = 0 yields b. Then feeding in x = 1 = a + b yields b. ** Whole situation is example of use of mathematical analysis to break a cipher based on mathematical formula. Substitution cipher: Ciphertext only: * Brute force not reasonable, even with computer help (=~ 26! =~ 4 x 10^26 keys) * Doable with hints from word boundaries, otherwise we need more help. Known Plaintext: * Compile (partial) permutation table from sample of encrypted plaintext. * Use puzzle-solving to fill in remainder of table from samples of untranslated ciphertext. Chosen Plaintext: * Still trivial. Give system the string abc...xyz and read off permutation table. *** Help from Probability and Statistics *** English prose (or any natural language) is highly structured (non-random). * In fact, it is structured at so many different levels, that it is very difficult to conceal them all. * This structure can be exploited to crack cryptographic systems. * Quantitative knowledge of measureable structure in a language one of the most potent levers the cryptoanalyst has. * Certain simple statistical measures of frequency are relatively simple to exploit. Important basic concept is counting of various abstract objects * There are n! = n * n-1 * n-2 * ... * 1 ways of ordering n objects. Read "n factorial" * There are (n choose k) = n! / ((n-k)! * k!) different possible subsets of k items drawn from a set of n items. This number is called the "binomial coefficient". * Other examples in the book. You should be reasonably comfortable with this sort of "combinatorics" Notions of probability * Basic notion of a "sample space" that has several possible (disjoint) outcomes (w_1, w_2, ... , w_n) each characterized by a "probability" 0.0 <= P(w_i) <= 1.0 such that Sum over all possible i of P(w_i) = 1.0 * A "Random Variable" X associates a value x_i with each outcome w_i. The distinction between the outcomes and their associated values is sometimes muddled by notaion, as often a convenieant way of naming or referring to the various outcomes is via the value of an associated random variable. * Various notations: P(X = x_i) = P(X_i) = p_i * Can be generalized to infinite sets of outcomes * Some philosophical debate over how or whether probability should be defined in more basic terms. * One useful mental model is the idea of a repeatable "trial" or "observation" of a random variable, something like flipping a coin with probability related to relative frequency of an outcome as the number of trials becomes large. * Possible outcomes x_i are called (atomic) "events" * Trials are said to be "independent" if there is no correlation between them. (difficult to show) * Notion of event can be generalized to subsets of the set of all atomic events, and probabilities assigned by summing atomic probabilities. * Such "compound events" give rise to notions of boolean combinations such as (A and B), (A or B), or ~A, defined in terms of set intersection, union, and complement. * We also get the notion of "conditional probability" P(A|B) = P(A and B) / P(B) * From which can be proven Bayes theorem P(A|B) = P(B|A) * P(A) / P(B) A couple of useful formulas * If an event A occurs with probability p, then the probability that A occurs exactly k times in n independent trials is (N choose k) * p^k * (1-p)^(N-k) * If we have N equiprobable events, then the probability P that n trials will be all different satisfies the inequality ln(P) <= -n^2/2N P = (1 - 1/N)(1 - 2/N) ... (1 - (n-1)/N) ==> ln(P) = ln(1 - 1/N) + ln(1 - 2/N) ... + ln(1 - (n-1)/N) ==> ln(P) <= -(1/N + 2/N + ... + (n-1) / N) since ln(1-x) <= -x in range ==> ln(P) <= -n^2 / 2N from sum 1 to k = k(k-1)/2 This ends up being used in so-called "birthday attacks" from question how many people do you need before you have a better than even chance that two have the same birthday Answer: Find smallest n so that P <= -n^2/2N < ln(1/2) ==> -n^2 / 2*365 < -.69 ==> n > sqrt(.69 * 2 * 265) ~= sqrt(506) = 22.5 Statistics of English useful in breaking ciphers * Single letter frequencies: ETAOIN SHRDLU... e = 12.7, t = 9.1, a = 8.2, o = 7.5, i = 7.0, n = 6.7, ..., z = 0.1 % Compare p for random occurance = 1/26 = 3.8% * Common word frequencies: the, to, of, i, a, and, is, that... the = 4.65, and = 1.82, that = 1.62, for = 1.17, you = 1.06 Some of these are almost certain to occur in any fragment of text exceeding, say 50 words (birthday effect). * Letter pair frequencies, "bigrams": th, in, he, er, re, on, es, ... % th = 2.63, in = 2.08, he = 1.75, er = 1.67, on = 1.33, es = 1.32, ... % Compare p for random sequences = 1 / (26^2) = 1/676 = .14% Top 54 (out of 676) account for 50% Spaces removed above, but similar tables can be compiled for other cases. * Adjacent triples, "trigrams" the = 1.49, ing = .77, tha = .52, and = .50, hat = .47, ion = .45, ... % p for random sequences = 1 / (26^3) = 1/17567 = .006% top 241 (out of 17567) gives 50% Statistical attack on affine cipher 1. Compile tables of frequencies of individual ciphertext characters 2. Use these to guess at ciphertext characters that represent two common plaintext characters, say, e and t 3. Use these guesses in formula for known plaintext attack (since in our encoding, t - e = 19 - 4 = 15, relatively prime to 26, with inverse 7, this should work) 4. Check tentative decoding. If gibberish, make another guess. * Note that if t - e were even or 13, than attack would not work. Suggests that encryption could be made robust against this attack by re-organizing the alphabet so that all common letters have even indices. Statistical attack on mono-alphabetic subsitution cipher. * Somewhat less structured, with more trial and error, but generally can be made to work, even without word boundaries, for long enough messages (above 100-200 characters, or even longer than about 25 if you are really good says the book) 1. Compile tables of frequences 2. Guess a ciphertext characters for few most common chars, say e, t, and a. 3. Use these guesses to look for common trigrams with two of the letters (the, tha, hat, ... and check consistancy of guesses) 4. Once this looks right, use similar approach to find encodings of next few most common letters (o,i,n -> ing, and, ion, ...) 5. At this point, enough partial english words should start to pop out, that remainder of decoding process is relatively simple word puzzle. * Numerous variations on 2-4 can be employed, some using bigram instead of, or in addition to trigrams. * Attack known to Arabs by 9th century AD, rediscovered in Europe in 15 century. * One fairly effective counter to above attack is "Homophonic subsitution" where several symbols are used for each letter, with more common letters getting more codes in proportion to frequency. Removes single-character information, and forces attack to start at level of bigrams. Of course key is also longer, and harder to change ------------------------------------------------- Reading Assigment: Garret, Chapters 2 and 3.
Problems: 2.1.10, 2.1.11, 2.1.12, 2.2.02, 2.2.04, 2.2.08, 2.3.02, 2.4.02
Due: Tuesday, September 10, 2002.
Homework: Write a program (or adapt material found on the web) to perform monoalphabetic encryption and decryption using a permutation derived from a specified key phrase (Singh page 13). For encryption, remove all non-alphabetic characters, and print the output in groups of 5. For decryption, just print one long string of lower case letters, as that is easier to read than groups.
Encrypt two samples of English prose with at least 200 characters using different keys. Prose can be anything that is not offensive, or engineered to be difficult to crack. For the next assignment, you will crack each other's encryptions, so you might consider what you need to do this as you are doing this program. Disallowed will be any tool that is fully automated or that uses a dictionary, (unless you write it yourself).
Hand in short writeup, with encryption and decryption runs. Also attach copies of the encryptions on separate sheets of paper without keys or plaintext (provide these in the main writeup). The program should not take long to write. If you adapt material you find on the web, you must document your source. Disallowed sources are other people associated with the class (students, TA, prof).
Due: Tuesday, September 10, 2002 (you don't have a week on this one).