{"id":48273332,"url":"https://github.com/villagecomputing/superopenai","last_synced_at":"2026-04-04T22:18:57.343Z","repository":{"id":224799440,"uuid":"764284933","full_name":"villagecomputing/superopenai","owner":"villagecomputing","description":"Logging and caching superpowers for the openai sdk","archived":false,"fork":false,"pushed_at":"2024-03-14T16:32:31.000Z","size":966,"stargazers_count":105,"open_issues_count":5,"forks_count":4,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-22T18:13:25.073Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/villagecomputing.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-27T20:02:52.000Z","updated_at":"2025-11-04T23:16:27.000Z","dependencies_parsed_at":"2024-03-14T17:56:07.658Z","dependency_job_id":null,"html_url":"https://github.com/villagecomputing/superopenai","commit_stats":null,"previous_names":["villagecomputing/super-openai"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/villagecomputing/superopenai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/villagecomputing%2Fsuperopenai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/villagecomputing%2Fsuperopenai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/villagecomputing%2Fsuperopenai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/villagecomputing%2Fsuperopenai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/villagecomputing","download_url":"https://codeload.github.com/villagecomputing/superopenai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/villagecomputing%2Fsuperopenai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31416763,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T20:09:54.854Z","status":"ssl_error","status_checked_at":"2026-04-04T20:09:44.350Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-04T22:18:57.236Z","updated_at":"2026-04-04T22:18:57.330Z","avatar_url":"https://github.com/villagecomputing.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# superopenai: logging and caching superpowers for the openai sdk\n\n_superopenai is a minimal convenience library for logging and caching LLM requests and responses for visibility and rapid iteration during development._\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/villagecomputing/superopenai\"\u003eStar us on Github\u003c/a\u003e \u0026nbsp; \u003ca href=\"https://amandhe.github.io/superopenai/llm/python/openai/2024/03/05/introducing-super-openai.html\"\u003eRead the blogpost\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://twitter.com/amansplaining\"\u003e\u003cimg src=\"https://img.shields.io/twitter/follow/amansplaining?style=social\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.python.org/pypi/superopenai\"\u003e\u003cimg src=\"https://img.shields.io/pypi/dm/superopenai.svg\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"./readme_img.png\" style=\"width: 400px;\" /\u003e\u003c/p\u003e\n\n## Introduction\n\nsuperopenai was built to solve the following problems:\n\n**Prompt and request visibility** \u0026mdash; LLM libraries like `langchain`, `guardrails`, `instructor` modify your prompts or even make additional requests under the hood. Sometimes this is useful, sometimes it's counter-productive. We think it's good to adopt a \"[show me the prompt](https://hamel.dev/blog/posts/prompt/)\" attitude.\n\n**Debugging** \u0026mdash; Complex chains or agents go wrong because of cascading failure. To debug the failure we need to inspect intermediate results and identify the source of error, then improve the prompt, try a different model, etc. It starts with logging and eyeballing the sequence of requests.\n\n**Quality-cost-speed tradeoff** \u0026mdash; Better models, multi-step prompting, longer prompts can improve quality of results but at the expense of cost and speed. View cost, token usage and latency in your logs as you’re developing so you can make this tradeoff ex ante not ex post.\n\n**Repeated identical requests** \u0026mdash; LLM development is slow because often you change a single prompt or parameter and have to wait for every LLM request to re-run. Can’t iterate quickly if you need to wait that long between experiments. Superopenai caches the last N LLM requests and responses in memory (when `temperature=0`)\n\n**Privacy, security and speed**. Superopenai focuses on local development when you’re just starting a new project, iterating on prompts or playing around. No need to sign up, get api keys, or learn a new interface. Your data never leaves your environment.\n\n### Installation \u0026 basic usage\n\nRun `pip install superopenai` or `poetry add superopenai`\n\nTo initialize superopenai, before initializing your openai client, run\n\n```python\nfrom superopenai import init_superopenai\n\ninit_superopenai()\n```\n\nThis will monkey-patch the relevant functions in the `OpenAI` class. Then you can use `openai` library as usual with all the superpowers of superopenai\n\n**Basic logging example**\n\n```python\nfrom openai import OpenAI\nfrom superopenai import init_logger, init_superopenai\n\ninit_superopenai()\nclient = OpenAI()\n\nwith init_logger() as logger:\n  client.chat.completions.create(\n    model=\"gpt-4-1106-preview\",\n    messages=[\n      {\"role\": \"user\", \"content\": \"What's the capital of France?\"}\n    ])\n  for log in logger.logs:\n    print(log)\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e Expand to see output\u003c/summary\u003e\n\n```\n+-----------+----------------------------------------------+\n| Messages  | - user: What's the capital of France?        |\n+-----------+----------------------------------------------+\n| Arguments | - model: gpt-4-1106-preview                  |\n+-----------+----------------------------------------------+\n| Output    | - assistant: The capital of France is Paris. |\n+-----------+----------------------------------------------+\n| Metadata  | - Cost: $0.00035                             |\n|           | - Prompt tokens: 14                          |\n|           | - Completion tokens: 7                       |\n|           | - Total tokens: 21                           |\n|           | - Start time: 1709914488.7480488             |\n|           | - Latency: 0.7773971557617188                |\n+-----------+----------------------------------------------+\n| Cached    | False                                        |\n+-----------+----------------------------------------------+\n```\n\n\u003c/details\u003e\n\n\u003cbr\u003e\nYou can also avoid the context manager and directly manage starting and stopping loggers.\n\n```python\nlogger = init_logger()\nclient.chat.completions.create(\n  model=\"gpt-4-1106-preview\",\n  messages=[\n  {\"role\": \"user\", \"content\": \"What's the capital of France?\"}\n  ])\nfor log in logger.logs:\n  print(log)\nlogger.end()\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e Expand to see output\u003c/summary\u003e\n\n```\n+-----------+----------------------------------------------+\n| Messages  | - user: What's the capital of France?        |\n+-----------+----------------------------------------------+\n| Arguments | - model: gpt-4-1106-preview                  |\n+-----------+----------------------------------------------+\n| Output    | - assistant: The capital of France is Paris. |\n+-----------+----------------------------------------------+\n| Metadata  | - Cost: $0.00035                             |\n|           | - Prompt tokens: 14                          |\n|           | - Completion tokens: 7                       |\n|           | - Total tokens: 21                           |\n|           | - Start time: 1709914489.536513              |\n|           | - Latency: 3.981590270996094e-05             |\n+-----------+----------------------------------------------+\n| Cached    | True                                         |\n+-----------+----------------------------------------------+\n```\n\n\u003c/details\u003e\n\n\u003cbr\u003e\n\nNotice the second request's latency is almost 0 and `Cached` is `True`\n\n## Logging\n\nsuperopenai wraps the `OpenAI.chat.completions.create` and `AsyncOpenAI.chat.completions.create` functions and stores logs into a `superopenai.Logger` object. The following fields are captured and logged:\n\n**Basic logging**\n\nTo start logging, call `init_logger()` either as a context manager `with init_logger() as logger` or as a simple function call. If not using a context manager, make sure to called `logger.end()`.\n\nEvery openai chat completion request will not be logged and logs will be stored in `logger.logs`. Each log is a `ChatCompletionLog` object containing the following fields:\n\n- `input_messages`: a list of input prompts\n- `input_args`: an object containing request arguments (model, streaming, temperature, etc.)\n- `output`: a list of outputs (completion responses) produced by the LLM request\n- `metadata`: metadata about the request\n- `cached`: whether the response was returned from cache\n\nBy default all logs are stored in the `logs` folder in your project root. A new logfile is created for every day, so today's logs will be stored in `./logs/2024-03-08.log`. You can change the log directory when calling `init_logger`:\n\n```python\nwith init_logger(\"/path/to/log/dir\") as logger:\n  # your code\n```\n\n**Token usage, cost and latency**\n\nInside the `metadata` field of each log you will find information about how many prompt and completions tokens were used, what the total cost was and the latency, ie. time between request being sent and response being received.\n\nCost is calculated based on prompt and completion token prices tokens defined in `estimator.py`. Only OpenAI models have pre-defined prices. If you're using non-OpenAI models, you can optionally specify a price dictionary when initializing `superopenai`. Prices are specified per 1M tokens in a tuple representing prompt and completion tokens respectively.\n\n```python\ninit_superopenai(cost_dict={\n  'mistralai/Mixtral-8x7B-Instruct-v0.1': [0.5, 1.0]\n})\n\n```\n\n**Streaming and async**\n\nLogging works in streaming mode (setting `stream=True` in the chat completion request) as well as when using the async chat completion api.\n\nIn streaming mode, the output is a list of streamed chunks rather than a list of completion responses. All other fields are the same. The log object is a `StreamingChatCompletionLog` object.\n\n**Function Calling and Tools**\n\n`superopenai` works out of the box when using function calling or tools. The functions called and their arguments will be captured and printed in the `output` field. This works in streaming mode too.\n\n**Statistics**\n\nWhen you run a chain or agent with multiple LLM calls, it's useful to look at summary statistics over all the calls rather than individual ones.\n\nTo look at summary statistics, call `logger.summary_statistics()`\n\n```python\nwith init_logger() as logger:\n  client.chat.completions.create(\n    model=\"gpt-4-1106-preview\",\n    messages=[\n      {\"role\": \"user\", \"content\": \"What's the capital of France?\"}\n    ]\n  )\n  print(logger.summary_statistics())\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e Expand to see output\u003c/summary\u003e\n\n```\n+----------------------------+----------------------------+\n|      Number of Calls       |             1              |\n|       Number Cached        |             1              |\n+----------------------------+----------------------------+\n|            Cost            |          $0.00035          |\n+----------------------------+----------------------------+\n|       Prompt Tokens        |             14             |\n|     Completion Tokens      |             7              |\n|        Total Tokens        |             21             |\n+----------------------------+----------------------------+\n|   Prompt Tokens by Model   | {'gpt-4-1106-preview': 14} |\n| Completion Tokens by Model | {'gpt-4-1106-preview': 7}  |\n|   Total Tokens by Model    | {'gpt-4-1106-preview': 21} |\n+----------------------------+----------------------------+\n|       Total Latency        |   3.981590270996094e-05    |\n|      Average Latency       |   3.981590270996094e-05    |\n|  Average Latency (Cached)  |   3.981590270996094e-05    |\n| Average Latency (Uncached) |             0              |\n+----------------------------+----------------------------+\n```\n\n\u003c/details\u003e\n\n## Caching\n\n`superopenai` caches all requests in-memory using `cachetools` and returns the cached response next time if all request parameters are exactly the same and the same `OpenAI` client is used.\n\nCaching is automatically enabled when you called `init_superopenai` and applies both to regular `chat.completion.create` and async `chat.completion.create` requests. It works in both streaming and regular mode.\n\nYou can disable caching or change the cache size (default 1000) when initializating superopenai:\n\n```python\ninit_superopenai(enable_caching=True, cache_size=100)\n```\n\n## Using with langchain, etc.\n\nsuperopenai is fully compatible with `langchain`, `llama-index`, `instructor`, `guidance`, `DSpy` and most other third party libraries.\n\nThis is particularly useful when you're doing local development with `langchain` and want to quickly inspect your chain runs, or understand what requests were made under the hood. For example:\n\n```python\nfrom langchain.prompts import PromptTemplate\nfrom langchain_experimental.smart_llm import SmartLLMChain\nfrom langchain_openai import ChatOpenAI\n\nhard_question = \"I have a 12 liter jug and a 6 liter jug.\\\nI want to measure 6 liters. How do I do it?\"\nprompt = PromptTemplate.from_template(hard_question)\nllm = ChatOpenAI(temperature=0.8, model_name=\"gpt-3.5-turbo\")\n\nwith init_logger() as logger:\n  chain = SmartLLMChain(llm=llm, prompt=prompt,\n                      n_ideas=2,\n                      verbose=True)\n  result = chain.run({})\n\nprint(logger.summary_statistics())\n```\n\nOutput:\n\n```\n+----------------------------+-------------------------+\n|      Number of Calls       |            4            |\n|       Number Cached        |            0            |\n+----------------------------+-------------------------+\n|            Cost            |        $0.001318        |\n+----------------------------+-------------------------+\n|       Prompt Tokens        |           1094          |\n|     Completion Tokens      |           514           |\n|        Total Tokens        |           1608          |\n+----------------------------+-------------------------+\n|   Prompt Tokens by Model   | {'gpt-3.5-turbo': 1094} |\n| Completion Tokens by Model |  {'gpt-3.5-turbo': 514} |\n|   Total Tokens by Model    | {'gpt-3.5-turbo': 1608} |\n+----------------------------+-------------------------+\n|       Total Latency        |    10.062347888946533   |\n|      Average Latency       |    2.5155869722366333   |\n|  Average Latency (Cached)  |            0            |\n| Average Latency (Uncached) |    2.5155869722366333   |\n+----------------------------+-------------------------+\n```\n\n## Future work\n\n- Port to TypeScript\n- Simplifying retries\n- Tracing\n- Disk and remote caching\n- Thread-safe caching\n- Integrate with 3rd party hosted logging services\n\n## Contributing\n\n`superopenai` is free, open-source, and licensed under the MIT license. We welcome contributions from the community. You can always contribute by [giving us a star](https://github.com/villagecomputing/superopenai) :)\n\n## License\n\n`superopenai` is released under the MIT License. See the `LICENSE` file for more details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvillagecomputing%2Fsuperopenai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvillagecomputing%2Fsuperopenai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvillagecomputing%2Fsuperopenai/lists"}