{"id":13595030,"url":"https://github.com/JeffPaine/beautiful_idiomatic_python","last_synced_at":"2025-04-09T10:32:41.427Z","repository":{"id":41494843,"uuid":"112906529","full_name":"JeffPaine/beautiful_idiomatic_python","owner":"JeffPaine","description":"Notes from Raymond Hettinger's talk at PyCon US 2013.","archived":true,"fork":false,"pushed_at":"2019-08-13T20:38:57.000Z","size":13,"stargazers_count":654,"open_issues_count":0,"forks_count":132,"subscribers_count":28,"default_branch":"master","last_synced_at":"2024-11-06T17:44:57.735Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JeffPaine.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-12-03T06:44:36.000Z","updated_at":"2024-09-24T17:33:57.000Z","dependencies_parsed_at":"2022-08-10T02:35:01.000Z","dependency_job_id":null,"html_url":"https://github.com/JeffPaine/beautiful_idiomatic_python","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JeffPaine%2Fbeautiful_idiomatic_python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JeffPaine%2Fbeautiful_idiomatic_python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JeffPaine%2Fbeautiful_idiomatic_python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JeffPaine%2Fbeautiful_idiomatic_python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JeffPaine","download_url":"https://codeload.github.com/JeffPaine/beautiful_idiomatic_python/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248020593,"owners_count":21034459,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T16:01:42.703Z","updated_at":"2025-04-09T10:32:36.418Z","avatar_url":"https://github.com/JeffPaine.png","language":null,"readme":"# Transforming Code into Beautiful, Idiomatic Python\n\nNotes from Raymond Hettinger's talk at pycon US 2013 [video](http://www.youtube.com/watch?feature=player_embedded\u0026v=OSGv2VnC0go), [slides](https://speakerdeck.com/pyconslides/transforming-code-into-beautiful-idiomatic-python-by-raymond-hettinger-1).\n\nThe code examples and direct quotes are all from Raymond's talk. I've reproduced them here for my own edification and the hopes that others will find them as handy as I have!\n\n## Looping over a range of numbers\n\n```python\nfor i in [0, 1, 2, 3, 4, 5]:\n    print i**2\n\nfor i in range(6):\n    print i**2\n```\n\n### Better\n\n```python\nfor i in xrange(6):\n    print i**2\n```\n`xrange` creates an iterator over the range producing the values one at a time. This approach is much more memory efficient than `range`. `xrange` was renamed to `range` in python 3.\n\n## Looping over a collection\n\n```python\ncolors = ['red', 'green', 'blue', 'yellow']\n\nfor i in range(len(colors)):\n    print colors[i]\n```\n\n### Better\n\n```python\nfor color in colors:\n    print color\n```\n\n## Looping backwards\n\n```python\ncolors = ['red', 'green', 'blue', 'yellow']\n\nfor i in range(len(colors)-1, -1, -1):\n    print colors[i]\n```\n\n### Better\n\n```python\nfor color in reversed(colors):\n    print color\n```\n\n## Looping over a collection and indices\n\n```python\ncolors = ['red', 'green', 'blue', 'yellow']\n\nfor i in range(len(colors)):\n    print i, '---\u003e', colors[i]\n```\n\n### Better\n\n```python\nfor i, color in enumerate(colors):\n    print i, '---\u003e', color\n```\n\u003e It's fast and beautiful and saves you from tracking the individual indices and incrementing them.\n\n\u003e Whenever you find yourself manipulating indices [in a collection], you're probably doing it wrong.\n\n## Looping over two collections\n\n```python\nnames = ['raymond', 'rachel', 'matthew']\ncolors = ['red', 'green', 'blue', 'yellow']\n\nn = min(len(names), len(colors))\nfor i in range(n):\n    print names[i], '---\u003e', colors[i]\n\nfor name, color in zip(names, colors):\n    print name, '---\u003e', color\n```\n\n### Better\n\n```python\nfor name, color in izip(names, colors):\n    print name, '---\u003e', color\n```\n\n`zip` creates a new list in memory and takes more memory. `izip` is more efficient than `zip`.\nNote: in python 3 `izip` was renamed to `zip` and promoted to a builtin replacing the old `zip`.\n\n## Looping in sorted order\n\n```python\ncolors = ['red', 'green', 'blue', 'yellow']\n\n# Forward sorted order\nfor color in sorted(colors):\n    print color\n\n# Backwards sorted order\nfor color in sorted(colors, reverse=True):\n    print color\n```\n\n## Custom Sort Order\n\n```python\ncolors = ['red', 'green', 'blue', 'yellow']\n\ndef compare_length(c1, c2):\n    if len(c1) \u003c len(c2): return -1\n    if len(c1) \u003e len(c2): return 1\n    return 0\n\nprint sorted(colors, cmp=compare_length)\n```\n\n### Better\n\n```python\nprint sorted(colors, key=len)\n```\n\nThe original is slow and unpleasant to write. Also, comparison functions are no longer available in python 3.\n\n## Call a function until a sentinel value\n\n```python\nblocks = []\nwhile True:\n    block = f.read(32)\n    if block == '':\n        break\n    blocks.append(block)\n```\n\n### Better\n\n```python\nblocks = []\nfor block in iter(partial(f.read, 32), ''):\n    blocks.append(block)\n```\n\n`iter` takes two arguments. The first you call over and over again and the second is a sentinel value.\n\n## Distinguishing multiple exit points in loops\n\n```python\ndef find(seq, target):\n    found = False\n    for i, value in enumerate(seq):\n        if value == target:\n            found = True\n            break\n    if not found:\n        return -1\n    return i\n```\n\n### Better\n\n```python\ndef find(seq, target):\n    for i, value in enumerate(seq):\n        if value == target:\n            break\n    else:\n        return -1\n    return i\n```\n\nInside of every `for` loop is an `else`.\n\n## Looping over dictionary keys\n\n```python\nd = {'matthew': 'blue', 'rachel': 'green', 'raymond': 'red'}\n\nfor k in d:\n    print k\n\nfor k in d.keys():\n    if k.startswith('r'):\n        del d[k]\n```\n\nWhen should you use the second and not the first? When you're mutating the dictionary.\n\n\u003e If you mutate something while you're iterating over it, you're living in a state of sin and deserve what ever happens to you.\n\n`d.keys()` makes a copy of all the keys and stores them in a list. Then you can modify the dictionary.\nNote: in python 3 to iterate through a dictionary you have to explicitly write: `list(d.keys())` because `d.keys()` returns a \"dictionary view\" (an iterable that provide a dynamic view on the dictionary’s keys). See [documentation](https://docs.python.org/3/library/stdtypes.html#dict-views).\n\n## Looping over dictionary keys and values\n\n```python\n# Not very fast, has to re-hash every key and do a lookup\nfor k in d:\n    print k, '---\u003e', d[k]\n\n# Makes a big huge list\nfor k, v in d.items():\n    print k, '---\u003e', v\n```\n\n### Better\n\n```python\nfor k, v in d.iteritems():\n    print k, '---\u003e', v\n```\n\n`iteritems()` is better as it returns an iterator.\nNote: in python 3 there is no `iteritems()` and `items()` behaviour is close to what `iteritems()` had. See [documentation](https://docs.python.org/3/library/stdtypes.html#dict-views).\n \n## Construct a dictionary from pairs\n\n```python\nnames = ['raymond', 'rachel', 'matthew']\ncolors = ['red', 'green', 'blue']\n\nd = dict(izip(names, colors))\n# {'matthew': 'blue', 'rachel': 'green', 'raymond': 'red'}\n```\nFor python 3: `d = dict(zip(names, colors))`\n\n## Counting with dictionaries\n\n```python\ncolors = ['red', 'green', 'red', 'blue', 'green', 'red']\n\n# Simple, basic way to count. A good start for beginners.\nd = {}\nfor color in colors:\n    if color not in d:\n        d[color] = 0\n    d[color] += 1\n\n# {'blue': 1, 'green': 2, 'red': 3}\n```\n\n### Better\n\n```python\nd = {}\nfor color in colors:\n    d[color] = d.get(color, 0) + 1\n\n# Slightly more modern but has several caveats, better for advanced users\n# who understand the intricacies\nd = collections.defaultdict(int)\nfor color in colors:\n    d[color] += 1\n```\n\n## Grouping with dictionaries -- Part I and II\n\n```python\nnames = ['raymond', 'rachel', 'matthew', 'roger',\n         'betty', 'melissa', 'judith', 'charlie']\n\n# In this example, we're grouping by name length\nd = {}\nfor name in names:\n    key = len(name)\n    if key not in d:\n        d[key] = []\n    d[key].append(name)\n\n# {5: ['roger', 'betty'], 6: ['rachel', 'judith'], 7: ['raymond', 'matthew', 'melissa', 'charlie']}\n\nd = {}\nfor name in names:\n    key = len(name)\n    d.setdefault(key, []).append(name)\n```\n\n### Better\n\n```python\nd = collections.defaultdict(list)\nfor name in names:\n    key = len(name)\n    d[key].append(name)\n```\n\n## Is a dictionary popitem() atomic?\n\n```python\nd = {'matthew': 'blue', 'rachel': 'green', 'raymond': 'red'}\n\nwhile d:\n    key, value = d.popitem()\n    print key, '--\u003e', value\n```\n\n`popitem` is atomic so you don't have to put locks around it to use it in threads.\n\n## Linking dictionaries\n\n```python\ndefaults = {'color': 'red', 'user': 'guest'}\nparser = argparse.ArgumentParser()\nparser.add_argument('-u', '--user')\nparser.add_argument('-c', '--color')\nnamespace = parser.parse_args([])\ncommand_line_args = {k:v for k, v in vars(namespace).items() if v}\n\n# The common approach below allows you to use defaults at first, then override them\n# with environment variables and then finally override them with command line arguments.\n# It copies data like crazy, unfortunately.\nd = defaults.copy()\nd.update(os.environ)\nd.update(command_line_args)\n```\n\n### Better\n\n```python\nd = ChainMap(command_line_args, os.environ, defaults)\n```\n\n`ChainMap` has been introduced into python 3. Fast and beautiful.\n\n## Improving Clarity\n * Positional arguments and indicies are nice\n * Keywords and names are better\n * The first way is convenient for the computer\n * The second corresponds to how human’s think\n\n## Clarify function calls with keyword arguments\n\n```python\ntwitter_search('@obama', False, 20, True)\n```\n\n### Better\n\n```python\ntwitter_search('@obama', retweets=False, numtweets=20, popular=True)\n```\n\nIs slightly (microseconds) slower but is worth it for the code clarity and developer time savings.\n\n## Clarify multiple return values with named tuples\n\n```python\n# Old testmod return value\ndoctest.testmod()\n# (0, 4)\n# Is this good or bad? You don't know because it's not clear.\n```\n\n### Better\n\n```python\n# New testmod return value, a named tuple\ndoctest.testmod()\n# TestResults(failed=0, attempted=4)\n```\n\nA named tuple is a subclass of tuple so they still work like a regular tuple, but are more friendly.\n\nTo make a named tuple, call namedtuple factory function in collections module:\n\n```python\nfrom collections import namedtuple\nTestResults = namedtuple('TestResults', ['failed', 'attempted'])\n```\n\n## Unpacking sequences\n\n```python\np = 'Raymond', 'Hettinger', 0x30, 'python@example.com'\n\n# A common approach / habit from other languages\nfname = p[0]\nlname = p[1]\nage = p[2]\nemail = p[3]\n```\n\n### Better\n\n```python\nfname, lname, age, email = p\n```\n\nThe second approach uses tuple unpacking and is faster and more readable.\n\n## Updating multiple state variables\n\n```python\ndef fibonacci(n):\n    x = 0\n    y = 1\n    for i in range(n):\n        print x\n        t = y\n        y = x + y\n        x = t\n```\n\n### Better\n\n```python\ndef fibonacci(n):\n    x, y = 0, 1\n    for i in range(n):\n        print x\n        x, y = y, x + y\n```\n\nProblems with first approach\n\n * x and y are state, and state should be updated all at once or in between lines that state is mis-matched and a common source of issues\n * ordering matters\n * it's too low level\n\n\nThe second approach is more high-level, doesn't risk getting the order wrong and is fast.\n\n## Simultaneous state updates\n\n```python\ntmp_x = x + dx * t\ntmp_y = y + dy * t\n# NOTE: The \"influence\" function here is just an example function, what it does \n# is not important. The important part is how to manage updating multiple \n# variables at once.\ntmp_dx = influence(m, x, y, dx, dy, partial='x')\ntmp_dy = influence(m, x, y, dx, dy, partial='y')\nx = tmp_x\ny = tmp_y\ndx = tmp_dx\ndy = tmp_dy\n```\n\n### Better\n\n```python\n# NOTE: The \"influence\" function here is just an example function, what it does \n# is not important. The important part is how to manage updating multiple \n# variables at once.\nx, y, dx, dy = (x + dx * t,\n                y + dy * t,\n                influence(m, x, y, dx, dy, partial='x'),\n                influence(m, x, y, dx, dy, partial='y'))\n```\n\n## Efficiency\n * An optimization fundamental rule\n * Don’t cause data to move around unnecessarily\n * It takes only a little care to avoid O(n**2) behavior instead of linear behavior\n\n\u003e Basically, just don't move data around unecessarily.\n\n## Concatenating strings\n\n```python\nnames = ['raymond', 'rachel', 'matthew', 'roger',\n         'betty', 'melissa', 'judith', 'charlie']\n\ns = names[0]\nfor name in names[1:]:\n    s += ', ' + name\nprint s\n```\n\n### Better\n\n```python\nprint ', '.join(names)\n```\n\n## Updating sequences\n\n```python\nnames = ['raymond', 'rachel', 'matthew', 'roger',\n         'betty', 'melissa', 'judith', 'charlie']\n\ndel names[0]\n# The below are signs you're using the wrong data structure\nnames.pop(0)\nnames.insert(0, 'mark')\n```\n\n### Better\n\n```python\nnames = collections.deque(['raymond', 'rachel', 'matthew', 'roger',\n               'betty', 'melissa', 'judith', 'charlie'])\n\n# More efficient with collections.deque\ndel names[0]\nnames.popleft()\nnames.appendleft('mark')\n```\n## Decorators and Context Managers\n * Helps separate business logic from administrative logic\n * Clean, beautiful tools for factoring code and improving code reuse\n * Good naming is essential.\n * Remember the Spiderman rule: With great power, comes great responsibility!\n\n## Using decorators to factor-out administrative logic\n\n```python\n# Mixes business / administrative logic and is not reusable\ndef web_lookup(url, saved={}):\n    if url in saved:\n        return saved[url]\n    page = urllib.urlopen(url).read()\n    saved[url] = page\n    return page\n```\n\n### Better\n\n```python\n@cache\ndef web_lookup(url):\n    return urllib.urlopen(url).read()\n```\n\nNote: since python 3.2 there is a decorator for this in the [standard library](https://docs.python.org/3/library/functools.html): [`functools.lru_cache`](https://pypi.python.org/pypi/backports.functools_lru_cache/1.2.1).\n\n## Factor-out temporary contexts\n\n```python\n# Saving the old, restoring the new\nold_context = getcontext().copy()\ngetcontext().prec = 50\nprint Decimal(355) / Decimal(113)\nsetcontext(old_context)\n```\n\n### Better\n\n```python\nwith localcontext(Context(prec=50)):\n    print Decimal(355) / Decimal(113)\n```\n\n## How to open and close files\n\n```python\nf = open('data.txt')\ntry:\n    data = f.read()\nfinally:\n    f.close()\n```\n\n### Better\n\n```python\nwith open('data.txt') as f:\n    data = f.read()\n```\n\n## How to use locks\n\n```python\n# Make a lock\nlock = threading.Lock()\n\n# Old-way to use a lock\nlock.acquire()\ntry:\n    print 'Critical section 1'\n    print 'Critical section 2'\nfinally:\n    lock.release()\n```\n\n### Better\n\n```python\n# New-way to use a lock\nwith lock:\n    print 'Critical section 1'\n    print 'Critical section 2'\n```\n\n## Factor-out temporary contexts\n\n```python\ntry:\n    os.remove('somefile.tmp')\nexcept OSError:\n    pass\n```\n\n### Better\n\n```python\nwith ignored(OSError):\n    os.remove('somefile.tmp')\n```\n\n`ignored` is is new in python 3.4, [documentation](http://docs.python.org/dev/library/contextlib.html#contextlib.ignored).\nNote: `ignored` is actually called `suppress` in the standard library.\n\nTo make your own `ignored` context manager in the meantime:\n\n```python\n@contextmanager\ndef ignored(*exceptions):\n    try:\n        yield\n    except exceptions:\n        pass\n```\n\n\u003e Stick that in your utils directory and you too can ignore exceptions\n\n## Factor-out temporary contexts\n\n```python\n# Temporarily redirect standard out to a file and then return it to normal\nwith open('help.txt', 'w') as f:\n    oldstdout = sys.stdout\n    sys.stdout = f\n    try:\n        help(pow)\n    finally:\n        sys.stdout = oldstdout\n```\n\n### Better\n\n```python\nwith open('help.txt', 'w') as f:\n    with redirect_stdout(f):\n        help(pow)\n```\n\n`redirect_stdout` is proposed for python 3.4, [bug report](http://bugs.python.org/issue15805).\n\nTo roll your own `redirect_stdout` context manager\n\n```python\n@contextmanager\ndef redirect_stdout(fileobj):\n    oldstdout = sys.stdout\n    sys.stdout = fileobj\n    try:\n        yield fileobj\n    finally:\n        sys.stdout = oldstdout\n```\n\n## Concise Expressive One-Liners\nTwo conflicting rules:\n\n * Don’t put too much on one line\n * Don’t break atoms of thought into subatomic particles\n\nRaymond’s rule:\n\n * One logical line of code equals one sentence in English\n\n## List Comprehensions and Generator Expressions\n\n```python\nresult = []\nfor i in range(10):\n    s = i ** 2\n    result.append(s)\nprint sum(result)\n```\n\n### Better\n\n```python\nprint sum(i**2 for i in xrange(10))\n```\n\nFirst way tells you what to do, second way tells you what you want.\n","funding_links":[],"categories":["Others"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJeffPaine%2Fbeautiful_idiomatic_python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJeffPaine%2Fbeautiful_idiomatic_python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJeffPaine%2Fbeautiful_idiomatic_python/lists"}