{"id":19560569,"url":"https://github.com/buzzfeed/caliendo","last_synced_at":"2025-04-26T23:33:15.264Z","repository":{"id":4846842,"uuid":"6001173","full_name":"buzzfeed/caliendo","owner":"buzzfeed","description":"caliendo","archived":false,"fork":false,"pushed_at":"2015-06-16T19:30:26.000Z","size":1020,"stargazers_count":15,"open_issues_count":15,"forks_count":5,"subscribers_count":228,"default_branch":"master","last_synced_at":"2025-04-04T18:38:46.181Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"scribu/wp-posts-to-posts","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/buzzfeed.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-09-28T18:13:46.000Z","updated_at":"2024-11-28T16:30:41.000Z","dependencies_parsed_at":"2022-09-05T19:21:02.824Z","dependency_job_id":null,"html_url":"https://github.com/buzzfeed/caliendo","commit_stats":null,"previous_names":[],"tags_count":40,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buzzfeed%2Fcaliendo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buzzfeed%2Fcaliendo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buzzfeed%2Fcaliendo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buzzfeed%2Fcaliendo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/buzzfeed","download_url":"https://codeload.github.com/buzzfeed/caliendo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251063896,"owners_count":21530858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T05:08:04.200Z","updated_at":"2025-04-26T23:33:11.406Z","avatar_url":"https://github.com/buzzfeed.png","language":"Python","readme":"# Caliendo\n\n## About\n\nCaliendo is a very simple interface for mocking APIs. It allows you to skip\n(potentially heavy) calls to your database or remote resources by storing sets\nof calls and caching responses based on the sequence of execution as well as\nfunction arguments. In some cases this improves unit test performance by\nseveral orders of magnitude.\n\nIf you have any questions or comments about caliendo or if you're interested in contributing feel free to contact Andrew at andrew.kelleher@buzzfeed.com with 'caliendo' in the subject line.\n\n## Installation\n\nCaliendo is set up to install with `pip`. You can install it directly from\nGitHub by running:\n\n```console\npip install git+git://github.com/buzzfeed/caliendo.git#egg=caliendo\n```\n\nOr from pypi using the standard (to get the latest release version).\n\n```console\npip install caliendo\n```\n\nAlternatively if you have a copy of the source on your machine; cd to the\nparent directory and run:\n\n```console\npip install ./caliendo\n```\n\nTo run tests you can use the standard unittest module. You'll have various\nprompts during the process. You can just hit ctrl+d to continue. To run all\ntests you should use nose with --nocapture. nose capturing interferes with\nthe interactive prompts.\n\nYour tests will need to be written is TestCases of some sort (classes).\nCaliendo uses the TestCase instance to figure out what module the test came\nfrom at runtime by referring to self, which is the first argument to the\ntest methods.\n\n```console\n\npython setup.py test\n```\n\n```console\n\nnosetests --all-modules --nocapture test/\n```\n\n## Configuration\n\nCaliendo requires file read/write permissions for caching objects. The first time\nyou invoke tests calling caliendo:\n\n1. Caliendo writes to the specified cache files. The default location is in the\n   caliendo build, caliendo/cache, caliendo/evs, and caliendo/seeds, and\n   caliendo/used. You can change where caliendo creates these directories and\n   file by setting the environment variable:\n\n```console\nexport CALIENDO_CACHE_PREFIX=/some/absolute/path\n\n```\n\n2. If you would like to be prompted to overwrite or modify existing cached values\n   you can write the environment variable CALIENDO_PROMPT.\n\n```console\nexport CALIENDO_PROMPT=True\n\n```\n\n### Configuration Best Practices\n\nThere are a lot of ways to set environment variables in your application. On our team we've come up with a few 'best practices' that work really well for us.\n\n1. When using `nosetests` set the variables in the `__init__.py` file for your integration testing suite.\n\n    ```python\n    import os\n    import sys\n    os.environ['CALIENDO_CACHE_PREFIX'] = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'caliendo')\n    os.environ['PURGE_CALIENDO'] = 'True'\n    os.environ['USE_CALIENDO'] = 'True' # Important!\n\n    ```\n\n2. When using `unittest` with setup.py (invoked like `python setup.py test`) set the variables as above in either setup.py or the `__init__.py` file for your integration test folder.\n\n3. When using `fabric` to run tests using either of these methods include the above in your `fabfile/__init__.py` file.\n\n## Examples\n\nHere are a few basic examples of use.\n\n### The Cache\n\nCaliendo offers a cache which decorates callables. If you pass the cache the handle for the callable, and the args/kwargs; it will be 'cached'. The behavior is a little complex. Explained below:\n\n  * When the method is called the first time a counter is issued that is keyed on a hash of the stack trace and a serialization of the function parameters.\n  * If/When a matching hash is generated (e.g. a method is called with the same parameters by the same calling method the counter is incremented.\n  * With each unique counter the result of the function call is pickled and stored matching a CallDescriptor. If a return value can't be pickled caliendo will attempt to munge it. If caliendo fails to munge it an error will be thrown.\n  * When a method is called that matches an existing counter; the stored CallDescriptor rebuilds the original call and the original return value is returned by the cache.\n\n```python\nfrom caliendo.facade import cache\n\nglobal side_effect\nside_effect = 0\ndef foo():\n  global side_effect\n  side_effect += 1\n  return side_effect\n\nfor i in range(3):\n  assert cache(handle=foo) == i + 1\n\nprint side_effect\n```\n\nWhen the above example is run the first time; it will print 2. For every subsequent time it is run it will print 0 unless caliendo's cache is cleared.\n\n### Service Patching\n\nAn interface inspired greatly by python Mock is `patch()`.\n\n`patch()` is intended to be used as a decorator for integration/unit tests that need to be decoupled from external services.\n\nWhen `patch` is called it returns the test it decorates in the context of the specified method replaced by it's `caliendo.facade.cache` decorated version.\n\nWhen the decorated test is invoked it is patched at runtime. After the test returns it is automatically unpatched.\n\n`patch`, by default, uses `caliendo.facade.cache`. If you pass an `rvalue` as the second parameter; your patched method will return that value.\n\n```python\n\n# Pretend these methods are all defined in various modules in the codebase.\n# Let foo() be defined in api.services.foos\ndef foo():\n  return 'foo'\n# Let bar() be defined in api.services.bars\ndef bar():\n  return foo()\n# Let baz be defined in api.bazs\ndef baz():\n  return bar()\n\n# Now for our test suite.\nimport unittest\nfrom caliendo.patch import patch\nfrom api.bazs import baz\n\nclass ApiTest(unittest.TestCase):\n\n  @patch('api.services.bars.bar', rvalue='biz')\n  def test_baz(self):\n    assert baz() == 'biz'\n```\n\nIn the above example `bar` is nested in the service layer of the architecture. We can import it once at the head of the test suite and effectively patch it at the test's invocation.\n\nWe set the rvalue to 'biz', but if we left it alone the value 'foo' would have been cached on the initial run. Every subsequent run would not have called the `foo` or `bar` method, and would have simply returned the cached value from the initial invokation of the test.\n\n### Expected Values\n\nThere are a bunch of idiomatic methods for testing that expected values match observed values. At the root of this functionality is the `cache`.\n\n#### The basic behavior of these methods is all the same:\n\n  1. The observed value is passed for the first time.\n  2. Caliendo will give the user an interactive shell to check the expected value (stored in the variable `ev`)\n  3. The user can modify the expected value by modifying `ev` in the shell.\n  4. When the user quits with `ctrl+d` the expected value, `ev`, will be cached by `cache`.\n  5. On this run the check is trivial. If the expected value stored is valid for `cache`ing (e.g. vaguely `pickle`able) the test will pass.\n  6. When the `expected_value` method is invoked again in the same test/call the `cache`d value will be used for comparison to the new observed value.\n\n#### The available methods are:\n\n##### `expected_value.is_true_under(validator, observed_value)`\n\nThis is the most complicated method. As the first argument it takes a validator used to validate the observed value against the expected value.\n\nThe validator should take the expected value as the first argument and the observed value as the second. The return value doesn't really matter. The output of `expected_value.is_true_under()` is whatever your return value is.\n\n```python\ndef validator(expected_value, observed_value):\n    # Do a bunch of assertions here\n    return anything\n\n```\n\n##### `expected_value.is_equal_to(observed_value)`\n\nJust compares the observed value to the cached value.\n\n##### `expected_value.is_greater_than(observed_value)`\n\nTests that the expected value is greater than the observed value.\n\n##### `expected_value.is_less_than(observed_value)`\n\nTests that the expected value is less than the observed value.\n\n##### `expected_value.contains(observed_value, el)`\n\nSorry, this one isn't so idiomatic. Tests that the observed value contains `el`\n\n##### `expected_value.does_not_contain(observed_value, el)`\n\nSorry, this one isn't so idiomatic either. Tests that the observed value does not contain `el`\n\n### Side Effects\n\nSide effects can be run by patched methods.\n\nIf you pass an Exception that inherits from `BaseException`, `Exception`, or `StandardError` your exception will be raised.\n\n```python\n\nimport unittest\nfrom caliendo.patch import patch\nfrom api.bazs import baz\nimport sys\n\nclass ApiTest(unittest.TestCase):\n\n  @patch('api.services.bars.bar', side_effect=Exception(\"Things went foobar!\"))\n  def test_baz(self):\n    with self.assertRaisesRegexp(Exception, r\"Things went foobar!\"):\n        baz()\n\n```\n\nIf you pass a callable as the side effect it will be called and the result returned. The arguments and keyword arguments to the method being patched will be passed.\n\n```python\n\nimport unittest\nfrom caliendo.patch import patch\nfrom api.bazs import baz\nimport sys\n\ncounter = 0\ndef example_side_effect(*args, **kwargs):\n    global counter\n    counter += 1\n    return 'foo'\n\nclass ApiTest(unittest.TestCase):\n\n  @patch('api.services.bars.bar', side_effect=example_side_effect)\n  def test_baz(self):\n    assert baz() == 'foo'\n    assert counter == 1\n\n```\n\n### Record and Replay calls to Callback Functions \n\nWhen you patch a method with a callback function there's a small problem. When the cache hit occurs; the callback function never executes. \n\nThere's a decorator to allow you to execute the callback functions almost normally. If you have a patched method which calls a few callbacks before completing execution you can add a `replay` decorator to indicate calls to the callbacks should be replayed.\n\nFor example:\n\n```python\n\n\"\"\"\nIn test/api/foobar.py\n\"\"\"\ndef callback_for_method(a, b, c):\n    assert a == 1\n    assert b == 2\n    assert c == 3\n    path = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'callback_notes')\n    if os.path.exists(path):\n        with open(path, 'a') as f:\n            f.write('.')\n    else:\n        with open(path, 'w+') as f:\n            f.write('.')\n    return path\n\ndef method_with_callback(callback):\n    return callback(1, 2, 3)\n\n\"\"\"\nIn my test module\n\"\"\"\n@replay('test.api.foobar.callback_for_method')\n@patch('test.api.foobar.method_with_callback')\ndef test(i):\n    filename = method_with_callback(callback_for_method)\n    with open(filename, 'rb') as f:\n        assert f.read() == ('.' * (i+1))\n\n```\n\nIf test(i) is run in many sessions, were i is in index of the session this test will always pass.\n\nEven though `method_with_callback` is only called the very first time the test is run, a hook is created for the callback such that each time there is a cache-hit associated with method_with_callback, the callback is executed with the expected arguments.\n\nThere are some downsides. Arguments must be pickleable. Furthermore; runtime resources such as database connections passed via closures will be lost. One workaround is to establish those in the callback at runtime.\n\n### Ignore and subsequent_rvalue: Patching runtime resources and dynamic parameters. \n\nThere are two situations it's particularly useful to ignore certain input parameters and return values.\n * The input parameters or return value changes from one call to the next (even though the call is in the same order) or\n * The input parameters or return value is a runtime resource (like a database cursor) and, hence, is not available when calls to the cache are made rather than to the services. \n\nIn the event you must pass a dynamic argument to a patched method you can use the `Ignore` class to specify which paramters should be ignored.\n\n```python\n\nfrom caliendo import Ignore\nfrom random import random\nfrom datetime import datetime\n\ndynamic_arguments = Ignore(args=[0,1], kwargs=['current_time'])\n\nclass ApiTest(unittest.TestCase):\n\n    @patch('api.services.bars.bar', ignore=dynamic_arguments)\n    def test_baz(self):\n        assert baz(random(), random(), 'a', mykwarg='b', current_time=datetime.now()) == 'foo'\n```\n\nThe above test will always pass, even though positional args 0 and 1 change, as well as the current_time keyword argument.\n\nSince the value is `Ignore`'d it won't be pickled and hashed to for the CallDescriptor key. This means you can use `Ignore` to skip pickling input parameters when the additional information is not needed to make the CallDescriptor hash unique. \n\nWhen you need to avoid referencing a runtime resource when the cache is called (so the resource doesn't exist) you can use `subsequent_rvalue`. This is a parameter to the `caliendo.patch.patch` call.\n\n```python\nfrom caliendo import Ignore\nfrom my_database_client import find_with_cursor\nfrom my_application import from_cursor_to_list_of_models\n\nclass ApiTest(unittest.TestCase):\n\n    @patch('my_database_client.find_with_cursor', subsequent_rvalue=None)\n    @patch('my_application.from_cursor_to_list_of_models', ignore=Ignore(args=[0]))\n    def test_models(self):\n        cursor = find_with_cursor('my query goes here')\n        models = from_cursor_to_list_of_models(cursor)\n        # Assert stuff about the models here.\n```\n\nIn the above example we use `Ignore` along with subsequent_rvalue to allow us to call our services to return models in such a way that we avoid using runtime resources in our tests entirely (after the first run).\n\nHere; `find_with_cursor` will return a cursor the first time it's called. Each subsequent time it will return `None`.\n\n### Purge\n\nYou can purge unused cache file from the cache by using the purge functionality at `caliendo.db.flatfiles.purge`.\n\nBy including a call to purge at the end of a full run of the tests; any unused portion of any part of the cache will be erased.\n\nThis is a good way to commit minimal files to your code base.\n\n```python\n\nfrom caliendo.db.flatfiles import purge\n\n# Run all your tests:\nunittest.main()\n\n# Then purge unused files.\npurge()\n\n```\n\n### The Facade\n\nIf you have an api you want to run under Caliendo you can invoke it like so:\n\n```python\nsome_api     = SomeAPI()\ncaliendo_api = Facade( some_api ) # Note: caliendo is invoked with the INSTANCE, not the CLASS\n```\n\n### Chaining\n\nAs of revision v0.0.19 caliendo supports chaining so you can invoke it like:\n\n```python\ncaliendo_api = Facade(some_api)\nbaz = caliendo_api.get_foo().get_bar().get_baz()\n```\n\nIf type(baz) is not in ( float, long, str, int, dict, list, unicode ) it will be automatically wrapped by caliendo.\n\n### Type Checking\n\nSome APIs check the types or __class__ of the variables being passed in. A caliendo facade will have a class, `caliendo.facade.Wrapper`.\n\nIn order to unwrap an object to be type-checked by the target API you have to invoke the `wrapper__unwrap()` method on the Facade'd API releasing the object to the target API.\n\nA second method allows the implementer to specify a list of object to avoid Facading entirely. (Useful for exporting models).\n\n```python\nfacaded_api = Facade(SOMEAPI())\nfacaded_api.wrapper__ignore( somemodule.SomeClassDefinition )\n```\n\nThe above example will ensure objects with `__class__` `somemodule.SomeClassDefinition` will never be wrapped.\n\nTo stop ignoring a particular class you can do:\n\n```python\nfacaded_api = Facade(SOMEAPI())\nfacaded_api.wrapper__ignore( somemodule.SomeClassDefinition )\nfacaded_api.wrapper__unignore( somemodule.SomeClassDefinition )\n```\n\n## Execution\n\nOnce you have an instance of an API running under a `Facade` you should be able\nto call all the methods normally.\n\n### Hooks and Stacks (advanced)\n\nWhen the cache is called the first time you can specify a hook that gets called each subsequent time a call matching that CallDescriptor is made. After a new `CallDescriptor` is added to the `CallStack` for the current `patch` the method  you specified will be called, and passed the most recent CallDescriptor as it's argument.\n\nThe callback has to be pickleable. For example:\n\n```python\nimport unittest\nfrom caliendo.patch import patch\n\ndef callback(most_recent_call_descriptor):\n    print \"The last return value was: '%s'\\n\" % most_recent_call_descriptor.returnval\n\nclass ApiTest(unittest.TestCase):\n    @patch('api.services.bars.bar', callback=callback)\n    def test_baz(self):\n        assert baz() == 'baz'\n\n```\n\nThe above example will print \"The last return value was: 'baz'\" followed by a newline to stdout.\n\n## Troubleshooting\n\n1. If you start getting unexpected API results you should clear the cache by\n   simply deleting all the rows in the `test_io` table.\n\n2. Caliendo doesn't support a large level of nested objects in arguments or\n   return values. If you start getting unexpected results this may be the\n   problem. Better nesting is to come.\n\n3. If you alternate between calls to the `Facade` instance of an API and the\n   API itself you will probably see unexpected results. Caliendo maintains\n   the state of the API by maintaining a reference to the API internally.\n\n4. If you have a class inheriting from `dict` you'll need to define a\n   `__getstate__` and a `__setstate__` method. Described:\n   [http://stackoverflow.com/questions/5247250/why-does-pickle-getstate-accept-as-a-return-value-the-very-instance-it-requi]\n\n5. If you're trying to mock a module that contains class definitions; you can\n   use the classes normally except that the type will be that of a lambda\n   function instead of a class.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuzzfeed%2Fcaliendo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbuzzfeed%2Fcaliendo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuzzfeed%2Fcaliendo/lists"}