GitList
Browse code

Add draft notebook about Markov chains

Joseph Weston authored on 29/03/2021 05:27:12
Showing 2 changed files
content/posts/coding/markov-chain-decrypter.md
History View file @ 8b9d952
1 1 
deleted file mode 100644
... ... 
@@ -1,38 +0,0 @@
1 
-title: Markov Chain Monte Carlo for decryption
2 
-date: 2018-11-20
3 
-tags:
4 
-  - coding
5 
-  - haskell
6 
-  - markov-chain
7 
-draft: true
8 
-
9 
-Each year I teach part of the Python programming course at the
10 
-Casimir research school, and each year I try and think of more
11 
-short projects to offer the participants during the latter half
12 
-of the course. While fishing for ideas I came across an incredibly
13 
-cool idea: using Markov chains to break classic cryptographic ciphers.
14 
-
15 
-+ Found this paper
16 
-+ Idea is:
17 
-  - Analyze a reference text and obtain bigram frequencies
18 
-  - Construct a score function for a decryption key by finding
19 
-    the frequencies of bigrams in the decrypted text
20 
-  - Use this score function with the metropolis-hastings algorithm
21 
-    to walk around the key space
22 
-+ Coded up a solution in Python in a couple of hours, also wanted
23 
-  to give it a try in Haskell, to test out iHaskell and see how good
24 
-  Haskell is for "exploratory" work
25 
-
26 
-+ TL;DR for exploratory work Haskell seems too restrictive. Mediocre
27 
-  library documentation and overly abstracted types make error messages
28 
-  impossible to debug
29 
-
30 
-
31 
-+ Keys are just maps between characters, we make RVars of them
32 
-+ Trying to make sense of the required pieces of RVars is intense
33 
-+ We need to run the whole markov chain before we can get the results; not cool!
34 
-  Somewhere in our monad stack we are inserting some strictness; we need to find
35 
-  out where!
content/posts/coding/markov-chain-decrypter/index.ipynb
History View file @ 8b9d952
36 0 
new file mode 100644
... ... 
@@ -0,0 +1,573 @@
1 
+{
2 
+ "cells": [
3 
+  {
4 
+   "cell_type": "raw",
5 
+   "metadata": {},
6 
+   "source": [
7 
+    "---\n",
8 
+    "title: Decrypting substitution ciphers using Markov chains\n",
9 
+    "date: 2019-02-25\n",
10 
+    "tags\n",
11 
+    "    - coding\n",
12 
+    "    - python\n",
13 
+    "    - probability\n",
14 
+    "draft: true\n",
15 
+    "---"
16 
+   ]
17 
+  },
18 
+  {
19 
+   "cell_type": "markdown",
20 
+   "metadata": {},
21 
+   "source": [
22 
+    "I am part of the course team for the [Casimir programming course](https://casimir.researchschool.nl/casimir-course-programming--full-sign-up-now--4414.html).\n",
23 
+    "Each year we take 50 students through a software carpentry-style intensive course in Python and scientific programming over the course of a week.\n",
24 
+    "The capstone is a project lasting a couple of days where the students put into practice all that they've learned in the course."
25 
+   ]
26 
+  },
27 
+  {
28 
+   "cell_type": "markdown",
29 
+   "metadata": {},
30 
+   "source": [
31 
+    "Coming up with cool projects is a chore, however I recently read a blog post about using Markov Chain Monte Carlo for decrypting substitution ciphers.\n",
32 
+    "This meshes well with the other themes in the course, and on the first day there is a small exercise that uses some statistical analysis for decrypting substitution ciphers, however it is not very automatic.\n",
33 
+    "The blog post references [this 2010 paper](http://probability.ca/jeff/ftpdir/decipherart.pdf) from some Masters students at the University of Toronto, which I used as inspiration."
34 
+   ]
35 
+  },
36 
+  {
37 
+   "cell_type": "markdown",
38 
+   "metadata": {},
39 
+   "source": [
40 
+    "## The General Idea\n",
41 
+    "We have some text that we know has been encrypted using a substitution cipher, however we do not know the encryption key that has been used.\n",
42 
+    "\n",
43 
+    "The space that we are searching is the space of encryption keys. \n",
44 
+    "You can think of a key as a bijective map from the alphabet to itself, e.g. `A → D, B → R, ...`.\n",
45 
+    "The associated decryption key is just the inverse of this map.\n",
46 
+    "For a given decryption key we can attempt to decrypt the ciphertext.\n",
47 
+    "We will get some cleartext that may or may not be correct.\n",
48 
+    "What is clear is that if more entries in the decryption key are correct, the closer the cleartext will be to the right answer.\n",
49 
+    "We can analyze the frequency of pairs of letters in the cleartext and compare it to the frequency in some reference text.\n",
50 
+    "A higher number of matches will make the cleartext score higher. \n",
51 
+    "If we use the ratio of scores of different pairs of letters as our transition probability (properly normalized) then we can use a Markov Chain to sample the space of keys and (if implemented well!) converge to the true key."
52 
+   ]
53 
+  },
54 
+  {
55 
+   "cell_type": "markdown",
56 
+   "metadata": {},
57 
+   "source": [
58 
+    "## Step 1: Get a reference text\n",
59 
+    "\n",
60 
+    "We'll use a large corpus of English text as our reference.\n",
61 
+    "Luckily Project Guthenberg has a good number of English texts.\n",
62 
+    "For this example we choose War and Peace."
63 
+   ]
64 
+  },
65 
+  {
66 
+   "cell_type": "code",
67 
+   "execution_count": null,
68 
+   "metadata": {},
69 
+   "outputs": [],
70 
+   "source": [
71 
+    "from urllib.parse import urlparse\n",
72 
+    "from itertools import product\n",
73 
+    "from string import ascii_lowercase, printable, punctuation\n",
74 
+    "from itertools import groupby, chain\n",
75 
+    "\n",
76 
+    "import requests\n",
77 
+    "\n",
78 
+    "def is_url(maybe_url):\n",
79 
+    "    parsed_url = urlparse(maybe_url)\n",
80 
+    "    return parsed_url.scheme and parsed_url.netloc\n",
81 
+    "\n",
82 
+    "\n",
83 
+    "WORD_MARKER = ' '\n",
84 
+    "ALPHABET = ascii_lowercase\n",
85 
+    "ALLOWED_CHARS = frozenset(ALPHABET + WORD_MARKER)\n",
86 
+    "EXCLUDED_CHARS = frozenset(printable) - ALLOWED_CHARS\n",
87 
+    "ALPHA_TO_INDEX = {a: i for i, a in enumerate(ALPHABET)}\n",
88 
+    "\n",
89 
+    "\n",
90 
+    "def normalize_text(text):\n",
91 
+    "    \"\"\"Normalize a text using certain rules\n",
92 
+    "        \n",
93 
+    "    The normalization rules are the following:\n",
94 
+    "        + all alphabetic characters are converted to lowercase\n",
95 
+    "        + all non-alphabetic characters are converted to an end-of-word marker character.\n",
96 
+    "          We will only be analyzing the text on the level of the constituent\n",
97 
+    "          words, not the grammar, so we only care about punctuation and whitespace\n",
98 
+    "          because it indicates the start/end of a word.\n",
99 
+    "    \"\"\"\n",
100 
+    "    text = text.lower()\n",
101 
+    "    # normalize punctuation to whitespace. Probably incorrect for hyphenation,\n",
102 
+    "    # but we hope that hyphenated words are rare. This also catches\n",
103 
+    "    # (and ignores) non-ascii characters\n",
104 
+    "    text = ((c if c in ALLOWED_CHARS else WORD_MARKER) for c in text)\n",
105 
+    "    # remove duplicates of WORD_MARKER\n",
106 
+    "    text = chain.from_iterable(c if c == WORD_MARKER else g for c, g in groupby(text))\n",
107 
+    "    return ''.join(text)\n",
108 
+    "    \n",
109 
+    "\n",
110 
+    "# TODO: convert this to work on streams, for truly huge reference texts,\n",
111 
+    "#       to avoid reading the whole reference text into memory at once\n",
112 
+    "def get_reference_text(name):\n",
113 
+    "    \"\"\"Returns a normalized reference text as a string.\n",
114 
+    "    \n",
115 
+    "    See the documentation for 'normalize_text' for details of the normalization.\n",
116 
+    "    \n",
117 
+    "    Parameters\n",
118 
+    "    ----------\n",
119 
+    "    name : str\n",
120 
+    "        The name of the text to fetch; either a path to a file or a URL.\n",
121 
+    "        If a URL is provided, GETting the URL must return the text.\n",
122 
+    "    \"\"\"\n",
123 
+    "    try:\n",
124 
+    "        if is_url(name):\n",
125 
+    "            text = requests.get(name).text\n",
126 
+    "        else:\n",
127 
+    "            with open(name) as file:\n",
128 
+    "                text = file.read()        \n",
129 
+    "    except Exception as error:\n",
130 
+    "        msg = f'There was a problem fetching the text from \"{name}\"'\n",
131 
+    "        raise ValueError(msg) from error\n",
132 
+    "    \n",
133 
+    "    return normalize_text(text)"
134 
+   ]
135 
+  },
136 
+  {
137 
+   "cell_type": "code",
138 
+   "execution_count": null,
139 
+   "metadata": {},
140 
+   "outputs": [],
141 
+   "source": [
142 
+    "war_and_peace = get_reference_text('http://www.gutenberg.org/files/2600/2600-0.txt')"
143 
+   ]
144 
+  },
145 
+  {
146 
+   "cell_type": "markdown",
147 
+   "metadata": {},
148 
+   "source": [
149 
+    "Next we need a few utilities for counting bigrams in a text and constructing the matrix of probabilities for finding a letter in position $X+1$ given that a given letter is in position $X$. This is exactly the normalized matrix of bigram frequencies."
150 
+   ]
151 
+  },
152 
+  {
153 
+   "cell_type": "code",
154 
+   "execution_count": null,
155 
+   "metadata": {},
156 
+   "outputs": [],
157 
+   "source": [
158 
+    "from collections import Counter\n",
159 
+    "from operator import mul\n",
160 
+    "from functools import reduce\n",
161 
+    "from itertools import islice\n",
162 
+    "\n",
163 
+    "\n",
164 
+    "def pairs(sequence):\n",
165 
+    "    return zip(sequence, islice(sequence, 1))\n",
166 
+    "\n",
167 
+    "\n",
168 
+    "def prod(iterable):\n",
169 
+    "    return reduce(mul, iterable, 1)\n",
170 
+    "\n",
171 
+    "\n",
172 
+    "def take(n, it):\n",
173 
+    "    return islice(it, n)\n",
174 
+    "\n",
175 
+    "\n",
176 
+    "def count_bigrams(text):\n",
177 
+    "    \"Return the bigrams in a text as a dict (char1, char2) → count.\"\n",
178 
+    "    return Counter(pairs(text))\n",
179 
+    "\n",
180 
+    "\n",
181 
+    "def construct_transitions(text):\n",
182 
+    "    transitions = count_bigrams(text)\n",
183 
+    "    for c in ALLOWED_CHARS:\n",
184 
+    "        total = sum(transitions[c, p] for p in ALLOWED_CHARS)\n",
185 
+    "        if total == 0:\n",
186 
+    "            continue\n",
187 
+    "        for p in ALLOWED_CHARS:\n",
188 
+    "            transitions[c, p] /= total\n",
189 
+    "    return transitions  "
190 
+   ]
191 
+  },
192 
+  {
193 
+   "cell_type": "code",
194 
+   "execution_count": null,
195 
+   "metadata": {},
196 
+   "outputs": [],
197 
+   "source": [
198 
+    "wnp_transitions = construct_transitions(war_and_peace)"
199 
+   ]
200 
+  },
201 
+  {
202 
+   "cell_type": "markdown",
203 
+   "metadata": {},
204 
+   "source": [
205 
+    "---"
206 
+   ]
207 
+  },
208 
+  {
209 
+   "cell_type": "markdown",
210 
+   "metadata": {},
211 
+   "source": [
212 
+    "Next we define some tools for working with encryption/decryption keys"
213 
+   ]
214 
+  },
215 
+  {
216 
+   "cell_type": "code",
217 
+   "execution_count": null,
218 
+   "metadata": {},
219 
+   "outputs": [],
220 
+   "source": [
221 
+    "import random\n",
222 
+    "from contextlib import contextmanager\n",
223 
+    "\n",
224 
+    "\n",
225 
+    "@contextmanager\n",
226 
+    "def set_seed(seed=None):\n",
227 
+    "    \"\"\"A context manager that sets/resets the Python RNG seed on entry and exit.\n",
228 
+    "    \n",
229 
+    "    If the provided seed is 'None', then this context manager does nothing.\n",
230 
+    "    \"\"\"\n",
231 
+    "    if seed is not None:\n",
232 
+    "        rng_state = random.getstate()\n",
233 
+    "        random.seed(seed)\n",
234 
+    "    yield\n",
235 
+    "    if seed is not None:\n",
236 
+    "        random.setstate(rng_state)"
237 
+   ]
238 
+  },
239 
+  {
240 
+   "cell_type": "code",
241 
+   "execution_count": null,
242 
+   "metadata": {},
243 
+   "outputs": [],
244 
+   "source": [
245 
+    "from string import ascii_lowercase\n",
246 
+    "from random import shuffle\n",
247 
+    "\n",
248 
+    "\n",
249 
+    "def random_key(seed=None):\n",
250 
+    "    \"\"\"Return a random map *from* ciphertext symbols *to* cleartext symbols.\n",
251 
+    "    \n",
252 
+    "    Parameters\n",
253 
+    "    ----------\n",
254 
+    "    seed : int (optional)\n",
255 
+    "        If provided, the Python random generator will be seeded with the provided\n",
256 
+    "        value before generating the key, and restored to its previous state afterwards.\n",
257 
+    "        This is useful for producing the same key twice.\n",
258 
+    "    \"\"\"\n",
259 
+    "    with set_seed(seed):\n",
260 
+    "        # 'shuffle' only operates in-place on lists\n",
261 
+    "        shuffled = list(ALPHABET)\n",
262 
+    "        shuffle(shuffled)\n",
263 
+    "\n",
264 
+    "    return dict(zip(ALPHABET, shuffled))"
265 
+   ]
266 
+  },
267 
+  {
268 
+   "cell_type": "code",
269 
+   "execution_count": null,
270 
+   "metadata": {},
271 
+   "outputs": [],
272 
+   "source": [
273 
+    "def decrypt(ciphertext, key):\n",
274 
+    "    \"\"\"Decrypt a ciphertext using a substitution cipher with the provided key.\n",
275 
+    "    \n",
276 
+    "    Parameters\n",
277 
+    "    ----------\n",
278 
+    "    ciphertext : str\n",
279 
+    "        The text to decrypt\n",
280 
+    "    key : dict : str → str\n",
281 
+    "        A map *from* ciphertext symbols *to* cleartext symbols.\n",
282 
+    "        Any characters that appear in 'ciphertext' but do not appear in 'key'\n",
283 
+    "        remain unchanged in the cleartext.\n",
284 
+    "    \"\"\"\n",
285 
+    "    # XXX: If we're going to be calling this many times, we should\n",
286 
+    "    #      consider making the output of 'maketrans' the canonical key format\n",
287 
+    "    return ciphertext.translate(str.maketrans(key))\n",
288 
+    "\n",
289 
+    "\n",
290 
+    "def encrypt(cleartext, key):\n",
291 
+    "    \"\"\"Encrypt a ciphertext using a substitution cipher with the provided key.\n",
292 
+    "    \n",
293 
+    "    Parameters\n",
294 
+    "    ----------\n",
295 
+    "    cleartext : str\n",
296 
+    "        The text to encrypt\n",
297 
+    "    key : dict : str → str\n",
298 
+    "        A map *from* ciphertext symbols *to* cleartext symbols\n",
299 
+    "        Any characters that appear in 'ciphertext' but do not appear in 'key'\n",
300 
+    "        remain unchanged in the cleartext.\n",
301 
+    "    \"\"\"\n",
302 
+    "    # Encryption is decryption with the key reversed\n",
303 
+    "    key = {v: k for k, v in key.items()}\n",
304 
+    "    return decrypt(cleartext, key)"
305 
+   ]
306 
+  },
307 
+  {
308 
+   "cell_type": "markdown",
309 
+   "metadata": {},
310 
+   "source": [
311 
+    "And some utilities for constructing the \"distance\" between 2 keys."
312 
+   ]
313 
+  },
314 
+  {
315 
+   "cell_type": "code",
316 
+   "execution_count": null,
317 
+   "metadata": {},
318 
+   "outputs": [],
319 
+   "source": [
320 
+    "def similarity(seq1, seq2):\n",
321 
+    "    l = min(len(seq1), len(seq2))\n",
322 
+    "    return sum(c1 == c2 for c1, c2 in zip(seq1, seq2)) / l\n",
323 
+    "\n",
324 
+    "\n",
325 
+    "def distance(ciphertext, key1, key2):\n",
326 
+    "    \"\"\"Return the distance between 'key1' and 'key2'\n",
327 
+    "    \n",
328 
+    "    The distance is defined as the proportion of characters that are the same between the\n",
329 
+    "    cleartexts obtained using 'key1' and 'key2'.\n",
330 
+    "    \"\"\"\n",
331 
+    "    cleartext1 = decrypt(ciphertext, key1)\n",
332 
+    "    cleartext2 = decrypt(ciphertext, key2)\n",
333 
+    "    return 1 - similarity(cleartext1, cleartext2)\n",
334 
+    "    \n",
335 
+    "\n",
336 
+    "## From https://codereview.stackexchange.com/questions/172060/finding-the-minimum-number-of-swaps-to-sort-a-list\n",
337 
+    "def cycle_decomposition(permutation):\n",
338 
+    "    \"\"\"Generate cycles in the cyclic decomposition of a permutation.\n",
339 
+    "\n",
340 
+    "        >>> list(cycle_decomposition([7, 2, 9, 5, 0, 3, 6, 8, 1, 4]))\n",
341 
+    "        [[0, 7, 8, 1, 2, 9, 4], [3, 5], [6]]\n",
342 
+    "\n",
343 
+    "    \"\"\"\n",
344 
+    "    unvisited = set(permutation)\n",
345 
+    "    while unvisited:\n",
346 
+    "        j = i = unvisited.pop()\n",
347 
+    "        cycle = [i]\n",
348 
+    "        while True:\n",
349 
+    "            j = permutation[j]\n",
350 
+    "            if j == i:\n",
351 
+    "                break\n",
352 
+    "            cycle.append(j)\n",
353 
+    "            unvisited.remove(j)\n",
354 
+    "        yield cycle\n",
355 
+    "\n",
356 
+    "        \n",
357 
+    "def minimum_swaps(seq):\n",
358 
+    "    \"\"\"Return minimum swaps needed to sort the sequence.\n",
359 
+    "\n",
360 
+    "        >>> minimum_swaps([])\n",
361 
+    "        0\n",
362 
+    "        >>> minimum_swaps([2, 1])\n",
363 
+    "        1\n",
364 
+    "        >>> minimum_swaps([4, 8, 1, 5, 9, 3, 6, 0, 7, 2])\n",
365 
+    "        7\n",
366 
+    "\n",
367 
+    "    \"\"\"\n",
368 
+    "    permutation = sorted(range(len(seq)), key=seq.__getitem__)\n",
369 
+    "    return sum(len(cycle) - 1 for cycle in cycle_decomposition(permutation))"
370 
+   ]
371 
+  },
372 
+  {
373 
+   "cell_type": "markdown",
374 
+   "metadata": {},
375 
+   "source": [
376 
+    "from random import choice\n",
377 
+    "from functools import lru_cache\n",
378 
+    "from math import log, inf, exp\n",
379 
+    "\n",
380 
+    "\n",
381 
+    "def swapped(key):\n",
382 
+    "    a, b = random.choices(ALPHABET, k=2)\n",
383 
+    "    new = key.copy()\n",
384 
+    "    new[a], new[b] = new[b], new[a]\n",
385 
+    "    return new\n",
386 
+    "\n",
387 
+    "\n",
388 
+    "def transition_probability(proposal_density, key_density):\n",
389 
+    "    if key_density == 0:\n",
390 
+    "        return 1\n",
391 
+    "    else:\n",
392 
+    "        return max(proposal_density / key_density, 1)\n",
393 
+    "\n",
394 
+    "    \n",
395 
+    "def metropolis(ciphertext, transitions, start_key=None):\n",
396 
+    "    ciphertext = normalize_text(ciphertext)\n",
397 
+    "    \n",
398 
+    "    # Equation 2.4\n",
399 
+    "    # XXX: construct this using logarithms to avoid excessive rounding error\n",
400 
+    "    def log_pl(key):\n",
401 
+    "        maybe_cleartext = decrypt(ciphertext, key)\n",
402 
+    "        return sum(log(transitions[a, b]) if transitions[a, b] != 0 else -inf\n",
403 
+    "                   for a, b in pairs(maybe_cleartext))  \n",
404 
+    "\n",
405 
+    "    key = start_key or random_key()\n",
406 
+    "    yield key\n",
407 
+    "\n",
408 
+    "    while True:\n",
409 
+    "        proposal = swapped(key)\n",
410 
+    "        log_pl_proposal = log_pl(proposal)\n",
411 
+    "        log_pl_key = log_pl(key)\n",
412 
+    "        if log_pl_proposal > log_pl_key or log_pl_key == -inf:\n",
413 
+    "            key = proposal\n",
414 
+    "            best_key = key.copy()\n",
415 
+    "        elif random.uniform(0, 1) < exp(log_pl_proposal - log_pl_key):\n",
416 
+    "            key = proposal\n",
417 
+    "        yield key"
418 
+   ]
419 
+  },
420 
+  {
421 
+   "cell_type": "markdown",
422 
+   "metadata": {},
423 
+   "source": [
424 
+    "Finally we define the Metropolis algorithm"
425 
+   ]
426 
+  },
427 
+  {
428 
+   "cell_type": "code",
429 
+   "execution_count": null,
430 
+   "metadata": {},
431 
+   "outputs": [],
432 
+   "source": [
433 
+    "from random import choice\n",
434 
+    "from functools import lru_cache\n",
435 
+    "\n",
436 
+    "\n",
437 
+    "def swapped(key):\n",
438 
+    "    a, b = random.choices(ALPHABET, k=2)\n",
439 
+    "    new = key.copy()\n",
440 
+    "    new[a], new[b] = new[b], new[a]\n",
441 
+    "    return new\n",
442 
+    "\n",
443 
+    "\n",
444 
+    "def transition_probability(proposal_density, key_density):\n",
445 
+    "    if key_density == 0:\n",
446 
+    "        return 1\n",
447 
+    "    else:\n",
448 
+    "        return max(proposal_density / key_density, 1)\n",
449 
+    "\n",
450 
+    "    \n",
451 
+    "def metropolis(ciphertext, transitions, start_key=None):\n",
452 
+    "    ciphertext = normalize_text(ciphertext)\n",
453 
+    "    \n",
454 
+    "    # Equation 2.4\n",
455 
+    "    # XXX: construct this using logarithms to avoid excessive rounding error\n",
456 
+    "    def pl(key):\n",
457 
+    "        maybe_cleartext = decrypt(ciphertext, key)\n",
458 
+    "        return prod(transitions[a, b] for a, b in pairs(maybe_cleartext))  \n",
459 
+    "\n",
460 
+    "    key = start_key or random_key()\n",
461 
+    "    yield key\n",
462 
+    "\n",
463 
+    "    while True:\n",
464 
+    "        proposal = swapped(key)\n",
465 
+    "        pl_proposal = pl(proposal)\n",
466 
+    "        pl_key = pl(key)\n",
467 
+    "        if pl_proposal > pl_key or pl_key == 0:\n",
468 
+    "            key = proposal\n",
469 
+    "            best_key = key.copy()\n",
470 
+    "        elif random.uniform(0, 1) < pl_proposal / pl_key:\n",
471 
+    "            key = proposal\n",
472 
+    "        yield key"
473 
+   ]
474 
+  },
475 
+  {
476 
+   "cell_type": "markdown",
477 
+   "metadata": {},
478 
+   "source": [
479 
+    "----\n",
480 
+    "----\n",
481 
+    "----"
482 
+   ]
483 
+  },
484 
+  {
485 
+   "cell_type": "markdown",
486 
+   "metadata": {},
487 
+   "source": [
488 
+    "And run the algorithm on some example text to see if it works!"
489 
+   ]
490 
+  },
491 
+  {
492 
+   "cell_type": "code",
493 
+   "execution_count": null,
494 
+   "metadata": {},
495 
+   "outputs": [],
496 
+   "source": [
497 
+    "cleartext = normalize_text(\"\"\"\n",
498 
+    "Enter by the narrow gate, for wide is the gate and broad the road that leads to destruction\n",
499 
+    "\"\"\")\n",
500 
+    "\n",
501 
+    "ciphertext = encrypt(cleartext, random_key())\n",
502 
+    "\n",
503 
+    "keys = metropolis(ciphertext, wnp_transitions, start_key=dict(zip(ALPHABET, ALPHABET)))\n",
504 
+    "\n",
505 
+    "for i, key in enumerate(take(50000, keys)):\n",
506 
+    "    if i % 2000 == 0:\n",
507 
+    "        print(i, ':', decrypt(ciphertext, key))"
508 
+   ]
509 
+  },
510 
+  {
511 
+   "cell_type": "code",
512 
+   "execution_count": null,
513 
+   "metadata": {},
514 
+   "outputs": [],
515 
+   "source": [
516 
+    "from itertools import tee\n",
517 
+    "\n",
518 
+    "cleartext = normalize_text(\"\"\"\n",
519 
+    "Enter by the narrow gate, for wide is the gate and broad the road that leads to destruction.\n",
520 
+    "\"\"\")\n",
521 
+    "\n",
522 
+    "solution = dict(zip(ALPHABET, ALPHABET))  #random_key()\n",
523 
+    "\n",
524 
+    "ciphertext = encrypt(cleartext, solution)\n",
525 
+    "\n",
526 
+    "keys = metropolis(ciphertext, wnp_transitions, start_key=dict(zip(ALPHABET, ALPHABET)))\n",
527 
+    "\n",
528 
+    "distances = [distance(ciphertext, k, solution) for k in take(20000, keys)]"
529 
+   ]
530 
+  },
531 
+  {
532 
+   "cell_type": "code",
533 
+   "execution_count": null,
534 
+   "metadata": {},
535 
+   "outputs": [],
536 
+   "source": [
537 
+    "import matplotlib.pyplot as plt\n",
538 
+    "\n",
539 
+    "plt.plot(distances)"
540 
+   ]
541 
+  },
542 
+  {
543 
+   "cell_type": "markdown",
544 
+   "metadata": {},
545 
+   "source": [
546 
+    "### Closing remarks\n",
547 
+    "\n",
548 
+    "The Markov chain seems to get stuck at some minimum distance from the true key. It's not 100% clear to me why this is the case; if anyone has any insights, drop me an email!"
549 
+   ]
550 
+  }
551 
+ ],
552 
+ "metadata": {
553 
+  "kernelspec": {
554 
+   "display_name": "Python 3",
555 
+   "language": "python",
556 
+   "name": "python3"
557 
+  },
558 
+  "language_info": {
559 
+   "codemirror_mode": {
560 
+    "name": "ipython",
561 
+    "version": 3
562 
+   },
563 
+   "file_extension": ".py",
564 
+   "mimetype": "text/x-python",
565 
+   "name": "python",
566 
+   "nbconvert_exporter": "python",
567 
+   "pygments_lexer": "ipython3",
568 
+   "version": "3.8.1"
569 
+  }
570 
+ },
571 
+ "nbformat": 4,
572 
+ "nbformat_minor": 2
573 
+}