{"id":18267712,"url":"https://github.com/maebert/python-fails","last_synced_at":"2025-04-09T02:27:34.817Z","repository":{"id":30569435,"uuid":"34124368","full_name":"maebert/python-fails","owner":"maebert","description":"How to Shoot Yourself in the Foot with Python","archived":false,"fork":false,"pushed_at":"2017-12-16T00:24:54.000Z","size":7,"stargazers_count":59,"open_issues_count":1,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-01T19:45:17.214Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maebert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-04-17T15:30:28.000Z","updated_at":"2024-09-23T10:49:06.000Z","dependencies_parsed_at":"2022-08-03T15:14:28.275Z","dependency_job_id":null,"html_url":"https://github.com/maebert/python-fails","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maebert%2Fpython-fails","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maebert%2Fpython-fails/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maebert%2Fpython-fails/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maebert%2Fpython-fails/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maebert","download_url":"https://codeload.github.com/maebert/python-fails/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247964517,"owners_count":21025203,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-05T11:28:36.275Z","updated_at":"2025-04-09T02:27:34.796Z","avatar_url":"https://github.com/maebert.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# How to Shoot Yourself in the Foot with Python\n\n## Common pitfalls and misunderstandings\n\n![Snake Fail](http://cl.jroo.me/z3/e/D/s/e/a.baa-snake-will-eat-itself.jpg)\n\nThis document doesn't list _bugs_ in Python, but rather unexpected behaviours. Of course, \"unexpected behaviour\" depends a lot on what you expect Python to do.\n\nPlease feel free to add corrections, clarifications and more common pitfalls by sending a pull request or opening an issue!\n\n### Contents\n\n- [Arithmetic Fail](#arithmetic-fail)\n- [Class Property Fail](#class-property-fail)\n- [Scope Fail](#scope-fail)\n- [Oscar Speech Fail / Immutables Part I](#oscar-speech-fail--immutables-part-i)\n- [Immutable Fail, part II](#immutable-fail-part-ii)\n- [Cooking the Books Fail](#cooking-the-books-fail)\n- [Integer Division Fail](#integer-division-fail)\n- [Closure Fail](#closure-fail)\n\n\n### Arithmetic Fail\n\nLet's do some first grade arithmetic:\n\n```python\n\u003e\u003e\u003e a = 2\n\u003e\u003e\u003e a * a is 4\nTrue\n```\n\nWorks as advertised. Let's see if Python can handle slightly larger numbers, too:\n\n```python\n\u003e\u003e\u003e a = 20\n\u003e\u003e\u003e a * a is 400\nFalse\n```\n\n__What's happening here?__ Remember that everything in Python is an object, even numbers. Also remember that `is` checks for _identity_, not _equality_. So `2 * 2 is 4` is the same as `id(2 * 2) == id(4)`. The reason this works for small numbers is that Python creates singletons for integers from -9 to 255 on start-up because they're frequently used -- it's an implementation detail of CPython, not a language feature. However, when we compute `20 * 20`, a new object with value 400 is created, which is a different object than `400`.\n\n\u003e __How do avoid this issue:__ only use `is` to check if things are `True`, `False` or `None`. These are singletons (i.e. every `False` in your code is the same object.)\n\n### Class Property Fail\n\n\"In the wild, life is a constant battle to find enough to eat...\"\n\n```python\nclass Mammal(object):\n    awkwardness = 0\n\nclass Platypus(Mammal):\n    pass\n\nclass Dolphin(Mammal):\n    pass\n```\n\nWe create a mammal class and two sub-classes.\n\n```python\n\u003e\u003e\u003e print(Mammal.awkwardness, Platypus.awkwardness, Dolphin.awkwardness)\n0 0 0\n```\n\nNothing too unexpected. Let's set the awkwardness of the platypus to a well-deserved 10:\n\n```python\n\u003e\u003e\u003e Platypus.awkwardness = 10\n\u003e\u003e\u003e print(Mammal.awkwardness, Platypus.awkwardness, Dolphin.awkwardness)\n0 10 0\n```\n\nAll as expected. No remember that all mammals are basically __tubes__, and feel very self-conscious about being a mammal, too. Let's bump the awkwardness of mammals to 3:\n\n```python\n\u003e\u003e\u003e Mammal.awkwardness = 3\n\u003e\u003e\u003e print(Mammal.awkwardness, Platypus.awkwardness, Dolphin.awkwardness)\n3 10 3\n```\n\n__Why did the awkwardness of dolphins change? Dolphins are *cute*!__ We're dealing with class properties here. If untouched, they are simply references to the parent's class properties. When we set `Platypus.awkwardness = 10` we create a __new__ class property on the platypus class.\n\n### Scope Fail\n\nHere's one of my favourite Python party tricks (I'm an unpopular party guest). The setup:\n\n```python\nanswer = 42\n\ndef ultimate_question_of_life():\n    print(answer)\n```\n\nNow for the easy part:\n\n```python\n\u003e\u003e\u003e ultimate_question_of_life()\n42\n```\n\nRight on. But what if we try to one-up Douglas Adams?\n\n```python\nanswer = 42\n\ndef ultimate_question_of_life():\n    print(answer)\n    answer += 1\n\nultimate_question_of_life()\n```\n\nOuch:\n\n```python\n---------------------------------------------------------------------------\nUnboundLocalError                         Traceback (most recent call last)\nin \u003cmodule\u003e()\n----\u003e 7 ultimate_question_of_life()\n\nin ultimate_question_of_life()\n      3 def ultimate_question_of_life():\n----\u003e 4     print(answer)\n      5     answer += 1\n\nUnboundLocalError: local variable 'answer' referenced before assignment\n```\n\n__Alright, this fails.__ But wait a second, where does it fail? At the print statement that used to succeed in the example above! By adding a line _after_ a perfectly innocuous statement we make this statement suddenly break things! Madness!!\n\nThe problem here is that Python is, contrary to common misconception, not interpreted line-by-line. Instead, when we execute code (ie. import a module), Python computes scopes for all blocks, which variables are available inside the scope and where they point to. Since we assign `answer` inside the scope of `ultimate_question_of_life` (note that `+=` doesn't change the value of `answer`, but creates a new object!), we won't be able to refer to the `answer` that's declared outside that scope anymore.\n\n### Oscar Speech Fail / Immutables Part I\n\nAs any academy award winning director knows, the most unforgivable of all faux pas is to forget to thank your spouse. Let's write a Python script that takes care of our Oscar® speech:\n\n```python\ndef oscar_speech(people_to_thank=[]):\n    people_to_thank.append(\"my wife\")\n    for person in people_to_thank:\n        print(\"I want to thank {}\".format(person))\n```\n\nAlright, ready for the spotlight?\n\n```python\n\u003e\u003e\u003e oscar_speech()\nI want to thank my wife\n\u003e\u003e\u003e oscar_speech([\"The Academy\", \"Lars von Trier\"])\nI want to thank The Academy\nI want to thank Lars von Trier\nI want to thank my wife\n```\n\nGreat. Let's practice some more:\n\n```python\n\u003e\u003e\u003e oscar_speech()\nI want to thank my wife\nI want to thank my wife\n\u003e\u003e\u003e oscar_speech()\nI want to thank my wife\nI want to thank my wife\nI want to thank my wife\n```\n\n__Huh?__ The problem is that the list we pass on as the default argument only gets created once, at import time - no every time we call the function. So we end up appending our wife to the same list over and over again. This piece of code is identical to the one above and clarifies the issue:\n\n```python\ndefault_list = []\ndef oscar_speech(people_to_thank=default_list):\n    people_to_thank.append(\"my wife\")\n```\n\n### Immutable Fail, part II\n\nThe pledge:\n\n```python\nflying_circus = [\"Eric Idle\", \"Terry Gilliam\"]\n\ndef casting_a():\n    flying_circus.append(\"John Cleese\")\n    return flying_circus\n\ndef casting_b():\n    flying_circus += [\"Terry Jones\"]\n    return flying_circus\n```\n\nThe turn:\n\n```python\n\u003e\u003e\u003e casting_a()\n['Eric Idle', 'Terry Gilliam', 'John Cleese']\n```\n\n\nThe prestige:\n\n```python\n\u003e\u003e\u003e casting_b()\n---------------------------------------------------------------------------\nUnboundLocalError                         Traceback (most recent call last)\nin \u003cmodule\u003e()\n----\u003e 1 casting_b()\n\nin casting_b()\n      7 def casting_b():\n----\u003e 8     flying_circus += [\"Terry Jones\"]\n      9     return flying_circus\n\nUnboundLocalError: local variable 'flying_circus' referenced before assignment\n```\n\nWhy does `list.append` succeed, but `list += [...]` fail? Because `list.append` alters the object, whereas `+=` tries to create a new object. Remember our scope fail above. `flying_circus += [\"Terry Jones\"]` is the same as `flying_circus = flying_circus + [\"Terry Jones\"]`. Because we will assign the variable `flying_circus` it won't be available in our scope until after the assignment. However before we try to assign it, we try to compute `flying_circus + [\"Terry Jones\"]`. For comparison,\n\n```python\ndef casting_c():\n    flying_circus_new = flying_circus + [\"Terry Jones\"]\n    return flying_circus_new\n```\n\nwill work perfectly fine.\n\n\n### Cooking the Books Fail\n\nLet's turn our attention to the use of Python in the scientific community. A frequent problem many scientists encounter is that their data doesn't _quite_ match the hypothesis. Instead of going through the arduous step of refining our hypothesis, we can just, you know, _tweak_ the data a little bit until it looks like what it was _supposed_ to look like to start with.\n\n```python\ndata = {\n    'x': [0,1,2,3],\n    'y': [1,3,9,16]\n}\n```\n\nSo, obviously the effect here is quadratic, right? And the `3` on the y-axis is just a tiny perturbance in our measurements. Let's fix that! But just to be safe, let's work on a copy of our data and not touch the original:\n\n```python\n\u003e\u003e\u003e baked_data = data.copy()\n\u003e\u003e\u003e baked_data['y'][1] = 4\n\u003e\u003e\u003e print(baked_data)\n{'y': [1, 4, 9, 16], 'x': [0, 1, 2, 3]}\n```\n\nMuch better! Let's just make sure our original data is still the same.\n\n```python\n\u003e\u003e\u003e print(data)\n{'y': [1, 4, 9, 16], 'x': [0, 1, 2, 3]}\n```\n\n__Damn.__ When we created a copy of our data, we actually created a so-called __shallow copy__. This means that we create a new `dict` object, but we only copy the references of the keys and values. So the list we're altering in `baked_data` is actually the same list as the one in the original `data`.\n\nSimilarly, copying a list with `[:]`, as in `my_list = [[1, 2], 3, 4, 5], ; new_list = my_list[:]` only creates a shallow copy and can lead to similar unexpected effects.\n\n\u003e __How to avoid this issue:__ Use the `deepcopy` module.\n\n### Integer Division Fail\n\nHere's something that works, but is inadvisable.\n\n```python\nducks = [\"Donald\", \"Huey\", \"Dewey\", \"Louie\"]\nmiddle = len(ducks) / 2\nprint(ducks[middle])\n```\n\nAs any adventurous and brave pythonista does these days, you upgrade your code to Python3, and suddenly:\n\n\n```python\n# In Python3\nducks = [\"Donald\", \"Huey\", \"Dewey\", \"Louie\"]\nmiddle = len(ducks) / 2\nprint(ducks[middle])\n---------------------------------------------------------------------------\nUnboundLocalError                         Traceback (most recent call last)\nin \u003cmodule\u003e\n----\u003e 1 print(ducks[middle])\n\nTypeError: list indices must be integers, not float\n```\n\n___Why?___ Because in Python2, `/` has different meanings depending on wheather you feed in floats or integers. If both left and right side are integers, the result will also be an integer. In Python3, `/` will __always__ produce a float, and of course you can't index a list with floats.\n\n\u003e __How to avoid this issue:__ Use `//` for integer devision.\n\n### Closure Fail\n\nThis is a real-life example from production code I once wrote and now feel very ashamed for.\n\n```python\ndef valid_password(pwd):\n    return False  # In production, we'd do actual password validation here\n\ndef wrong_password_prompts():\n    return [lambda pwd: \"Password {} incorrect - {} attempts left\".format(pwd, 3-i) for i in range(3)]\n\ndef get_password():\n    for bad_attempt in wrong_password_prompts():\n        pwd = input()\n        if not valid_password(pwd):\n            print(bad_attempt(pwd))\n        else:\n            return True\n    return False\n```\n\nLet this sink in for a second. The crucial and most shameful part is `wrong_password_prompts`, where we return a list of three anonymous functions. The first function should return `\"Password xyz incorrect - 3 attempts left\"` when called with password `\"xyz\"`. The second function should return `\"Password xyz incorrect - 2 attempts left\"` and so on. Let's see what happens:\n\n```python\n\u003e\u003e\u003e get_password()\nxyz\nPassword xyz incorrect - 1 attempts left\nswordfish\nPassword swordfish incorrect - 1 attempts left\nshibboleth\nPassword shibboleth incorrect - 1 attempts left\n```\n\n__Why is there always only one attempt left?__ because the string we return only gets formatted when we call the anonymous functions. And the `i` we use to format it is actually just the `i` that gets  \"left over\" after the loop over `range(3)` is done - which has value `2`. Specifically, it leaked outside the scope.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaebert%2Fpython-fails","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaebert%2Fpython-fails","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaebert%2Fpython-fails/lists"}