{"id":22379115,"url":"https://github.com/sky-uk/mite","last_synced_at":"2025-04-06T18:17:43.221Z","repository":{"id":36132930,"uuid":"159649341","full_name":"sky-uk/mite","owner":"sky-uk","description":"Mite - A Python Performance Testing Framework","archived":false,"fork":false,"pushed_at":"2024-11-26T13:54:13.000Z","size":3339,"stargazers_count":25,"open_issues_count":29,"forks_count":7,"subscribers_count":32,"default_branch":"master","last_synced_at":"2024-11-27T15:41:24.041Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://sky-uk.github.io/mite/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sky-uk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-11-29T10:35:06.000Z","updated_at":"2024-11-26T13:52:53.000Z","dependencies_parsed_at":"2024-03-26T12:27:35.092Z","dependency_job_id":"e7490fc0-3482-4996-9a08-81eae22377d1","html_url":"https://github.com/sky-uk/mite","commit_stats":{"total_commits":230,"total_committers":21,"mean_commits":"10.952380952380953","dds":0.5913043478260869,"last_synced_commit":"cebdfe22860004a08cd906b19220948dcf5f1551"},"previous_names":[],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-uk%2Fmite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-uk%2Fmite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-uk%2Fmite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sky-uk%2Fmite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sky-uk","download_url":"https://codeload.github.com/sky-uk/mite/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247526768,"owners_count":20953143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-04T23:09:00.665Z","updated_at":"2025-04-06T18:17:43.187Z","avatar_url":"https://github.com/sky-uk.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mite\n\n[![Python 3.10](https://img.shields.io/badge/python-3.10-blue.svg)](https://www.python.org/downloads/release/python-3100/) [![Python 3.11](https://img.shields.io/badge/python-3.11-blue.svg)](https://www.python.org/downloads/release/python-3110/) [![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/sky-uk/mite/blob/master/LICENSE) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black) [![Docs](https://img.shields.io/badge/docs-read-blue)](https://sky-uk.github.io/mite/)\n\n## Load Testing Framework\n\nMite is a load testing framework with distributed components written in Python.\nRequests are executed asynchronously, allowing large throughput from relatively small infrastructure.\n\n## Installation\n\n```bash\npip install mite\n```\n\n\nThis requires that you have libcurl installed on your system (including C header files for development, which are often distributed separately from the shared libraries).  On Ubuntu, this can be accomplished with the command:\n\n```\nsudo apt install libcurl4 libcurl4-openssl-dev\n```\n(NB we recommend using a version of libcurl linked against openssl\nrather than gnutls, since the latter has memory leak problems)\n\n\nYou can also use the dockerfile included in this repository to run\nmite.  In order to get a shell in a container with mite installed, run\nthese commands (assuming you have docker installed on your machine):\n```\ndocker build -t mite .\ndocker run --rm -it mite sh\n```\n\nRun `mite --help` for a full list of commands\n\n## Your first scenario\n\nScenarios are a combination of 3 things, a set of journeys to run, a\ndatapool to provide test data for the journey (if applicable), and a\nvolume model.\n\n### Journeys\n\nJourneys are async python functions, and are where you put the logic of\nwhat you're trying to achieve.  Below is a simple example:\n\n```python\nimport asyncio\n\nasync def journey(ctx, arg1, arg2):\n    async with ctx.transaction('test1'):\n        ctx.send('test_message', content=ctx.config.get('test_msg', 'Not set'), sum=arg1 + arg2)\n        await asyncio.sleep(0.5)\n```\n\nThis journey just sends a log message to be collected by the framework\nand waits for half a second.  This journey takes 3 arguments, a context\nand two numbers (which will be supplied by the datapool, see below).\n\n#### Context\n\nThe context is important for every journey.  It provides a number of\nuseful methods for timing/naming of transactions, sending messages and\ndefaults to including http functionality from acurl.  Functionality can\nbe included in the journey by attaching it to the context.\n\nIn the above example we see an example of naming a transaction `async\nwith ctx.transaction('test1'):`.  This will capture timing for the\nindented block.\n\nWe also see that the config is attached to the context with\n`ctx.config.get('test_msg', 'Not set')` and in this case, the value of\nthat config value is sent to the log collector with\n`ctx.send('test_message', content=ctx.config.get('test_msg', 'Not set'))`\n\n### Datapools\n\nTo feed data into your test journey, you'll need a datapool. Several of\nthese are already provided in mite and usually read data in from an\niterable. To specify a custom datapool implementation, you simply need a\nclass that implements a `checkin` method which adds data to the pool and\na `checkout` method which removes data from the pool to be used by\njourneys.\n\nFor the above journey, which expects two arguments, we will show an\nexample of the RecyclableIterableDataPool.\n\n```python\nfrom mite.datapools import RecyclableIterableDataPool\n\ndatapool = RecyclableIterableDataPool([(i, i+2) for i in range(5000)])\n```\n\nThis pool will share data out to runners and check it back in when the\nrunners are done with that block of data.  In this case it cycles\nthrough a list of tuples which each contain two integers.\n\n### Volume Model\n\nA volume model defines how many instances of the journey should be ran\nwithin a window of time.  The window is definied as a start and an end\ntime (measured in seconds since the beginning of the test), which will be\nfed to the model by the framework.  This allows complex calculations to\nspecify the current volume based on the current time.  The volume model\ncan also raise `StopVolumeModel` to inform mite that the load injection\nshould stop for a given journey.  For this example we'll use a simple\nvolume model which merely runs ten journeys simultaneously, forever.\n\n```python\nvolumemodel = lambda start, end: 10\n```\n\n### Scenario\n\nWe now need to package the journey, datapool and volume model into a\nscenario.  This is a simple as defining a function which returns\na list of triplets of (journey name, datapool, volumemodel).\n\n```python\ndef scenario():\n    return [\n        ['mite.example:journey', datapool, volumemodel],\n    ]\n```\n\nThe journey name should be a string with two parts separated by a\ncolon.  The first part is the name of a python module that is\nimportable; the second is the name of a journey (an async function)\nwithin that module.  It is necessary to specify this as a string, rather\nthan as a python object, because the journey will be executed in a\ndifferent python process than the scenario function is.  Thus, we need a\nname for the journey that allows any python process to find it.\n\nThe volume model and data pool, on the other hand, are only used in the\npython process where the scenario function runs.  They are both python\nobjects.\n\n### Testing the journey\n\nBefore running the scenario, we should test the journey in isolation as\na scenario can be made up of multiple journeys.  This can be done with\nthe `mite journey test` command.  We just need to pass the name of the\njourney and the datapool it requires:\n\n```sh\nMITE_CONF_test_msg=\"Hello from mite\" mite journey test mite.example:journey mite.example:datapool\n```\n\nIf something goes wrong, adding the `--debugging` flag to this command\nwill drop excution into a debug session. The choice of debugger used can be\nmanaged by setting the [`PYTHONBREAKPOINT` environment variable](https://www.python.org/dev/peps/pep-0553/#environment-variable)\nbefore running mite. Python's built-in [pdb](https://docs.python.org/3/library/pdb.html))\ndebugger is invoked by default, but this can be changed to use, say, the\n[ipdb debugger](https://github.com/gotcha/ipdb):\n\n```\npip install ipdb\nexport PYTHONBREAKPOINT=ipdb.set_trace\nexport PYTHONPOSTMORTEM=ipdb.post_mortem\n```\n\n`PYTHONPOSTMORTEM` is a mite-specific extension to [PEP\n553](https://www.python.org/dev/peps/pep-0553/) which defines the\n`PYTHONBREAKPOINT` functionality.\n\n### Run the scenario\n\nIn order to run the finished scenario locally, which will include all\nthe necessary fixtures, run the following command:\n\n```sh\nMITE_CONF_test_msg=\"Hello from mite\" mite scenario test mite.example:scenario\n```\n\n## Distributed deployments\n\nIn order to scale up miteʼs capability to inject load, you will need to\nrun it as a distributed suite of components.  You can learn more about\nhow to accomplish this in the [documentation](https://sky-uk.github.io/mite/design-deployment.html).\n\n\n### Deploy distributed mite with docker compose\n\nBuild mite image: \n```\ndocker build -t mite .\n````\n\nRun mite deployments:\n\nUse `make` from `mite/local` dir:\n```\nmake up # start mite containers\nmake status # check status of mite containers\nmake clean # remove all mite containers\n```\nor\n```\ndocker-compose -f docker_compose.yml up\n```\n\nFor more information on distributed mite usage, [info](/local/README.md)\n\n## Maintainers\n\nIf you run into any trouble or need support getting to grips with Mite,\nreach out on [Slack](https://sky.slack.com/messages/mite) if you work at Sky,\n or contact one of the maintainers if you're an external contributer:\n\n| [\u003cimg src=\"https://avatars.githubusercontent.com/jb098\" width=100 height=100 alt=\"Jordan Brennan\" /\u003e\u003cbr /\u003eJordan Brennan](https://github.com/jb098)\u003cbr /\u003e\u003csub\u003e💻\u003c/sub\u003e | [\u003cimg src=\"https://avatars.githubusercontent.com/aecay\" width=100 height=100 alt=\"Aaron Ecay\" /\u003e \u003cbr /\u003eAaron Ecay](https://github.com/aecay)\u003cbr /\u003e\u003csub\u003e💻\u003c/sub\u003e | [\u003cimg src=\"https://avatars.githubusercontent.com/DavAnnunz\" width=100 height=100 alt=\"Davide Annunziata\" /\u003e\u003cbr /\u003eDavide Annunziata](https://github.com/DavAnnunz)\u003cbr /\u003e\u003csub\u003e💻\u003c/sub\u003e | [\u003cimg src=\"https://avatars.githubusercontent.com/ryanlinnit-sky\" width=100 height=100 alt=\"Ryan Linnit\" /\u003e\u003cbr /\u003eRyan Linnit](https://github.com/ryanlinnit-sky)\u003cbr /\u003e\u003csub\u003e💻\u003c/sub\u003e | [\u003cimg src=\"https://avatars.githubusercontent.com/cosmaprc\" width=100 height=100 alt=\"Cosmin Purcherea\" /\u003e\u003cbr /\u003eCosmin Purcherea](https://github.com/cosmaprc)\u003cbr /\u003e\u003csub\u003e💻\u003c/sub\u003e |\n| :---: | :---: | :---: | :---: | :---: |\n\n**Special thanks to the following contributors:**\n\n* [Tony Simpson](https://github.com/tonysimpson)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsky-uk%2Fmite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsky-uk%2Fmite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsky-uk%2Fmite/lists"}