{"id":29194757,"url":"https://github.com/paradite/model-quirks","last_synced_at":"2026-02-23T09:06:34.193Z","repository":{"id":293203685,"uuid":"982006004","full_name":"paradite/model-quirks","owner":"paradite","description":"Quirks and interesting bits of various models","archived":false,"fork":false,"pushed_at":"2025-06-03T07:26:16.000Z","size":9,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-14T09:27:57.935Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paradite.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-12T08:31:33.000Z","updated_at":"2025-06-03T07:26:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"64bf550b-2827-4afd-b55d-61b789d26cb3","html_url":"https://github.com/paradite/model-quirks","commit_stats":null,"previous_names":["paradite/model-quirks"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/paradite/model-quirks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fmodel-quirks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fmodel-quirks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fmodel-quirks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fmodel-quirks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paradite","download_url":"https://codeload.github.com/paradite/model-quirks/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paradite%2Fmodel-quirks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275448208,"owners_count":25466566,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-16T02:00:10.229Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-02T04:38:03.623Z","updated_at":"2025-09-16T16:27:11.022Z","avatar_url":"https://github.com/paradite.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# model-quirks\n\nQuirks, edge cases, and interesting bits of various models.\n\nRelated projects:\n\n- [llm-info](https://github.com/paradite/llm-info): Information on LLM models, context window token limit, output token limit, pricing and more.\n- [send-prompt](https://github.com/paradite/send-prompt): A TypeScript library for interacting with models across providers with standardized interfaces and tool calling.\n- [ai-file-edit](https://github.com/paradite/ai-file-edit): A library for editing files using AI models.\n- [16x Eval](https://eval.16x.engineer): Your personal workspace for prompt engineering and model evaluation.\n\n## Claude 3.7 Sonnet\n\nMax tokens for output\n\n- Claude 3.7 Sonnet requires the `max_tokens` parameter to be set in the API request. [source](https://docs.anthropic.com/en/api/messages#body-max-tokens)\n- Models may have different max output token limits, and an `invalid_request_error` error will be returned if the `max_tokens` parameter is higher than the model's max output token limit. So `max_tokens` needs to be set dynamically for each model to get the maximum output token limit for the model.\n\nTool use (function calling)\n\n- Claude 3.7 Sonnet (and Claude 3.5 Sonnet) do not support calling the same tools multiple times in one response (for example, calling a file-edit tool to edit multiple times to edit multiple files). This is case even with `token-efficient-tools-2025-02-19` header.\n- Anthropic has a [sample notebook implementation](https://github.com/anthropics/anthropic-cookbook/blob/main/tool_use/parallel_tools_claude_3_7_sonnet.ipynb) of \"batch tool\" that can act as a meta-tool to support parallel tool calls, but I have not verified if it works for calling the same tools multiple times.\n\n## Gemini 2.5 Pro\n\nThinking budget\n\n- While Gemini 2.5 Flash supports thinking budget, it is not currently available for Gemini 2.5 Pro as of May 12, 2025. [source](https://ai.google.dev/gemini-api/docs/thinking)\n\nMulti-round tool use\n\n- For multi-round tool use, Gemini 2.5 Pro requires the response from the previous tool call to be included in the messages. [source](https://ai.google.dev/gemini-api/docs/function-calling?example=meeting#step_4_create_user_friendly_response_with_function_result_and_call_the_model_again)\n- Empirically, if the response from the previous tool call is not included, it will be stuck at the first tool call.\n\n## Message Roles\n\n- Google models accept `user` or `model` as roles. [source](https://ai.google.dev/gemini-api/docs/text-generation)\n- OpenAI models accept `user`, `assistant`, `system` and `developer` roles. [source](https://platform.openai.com/docs/guides/text?api-mode=chat)\n  - OpenAI recommends using `developer` role in favor of `system` role for o1 models and newer. [source](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages)\n- Anthropic models accept `user` and `assistant` roles. [source](https://docs.anthropic.com/en/api/messages)\n\n## Reasoning content and tags\n\nDeepSeek-R1 and Qwen3 both use `\u003cthink\u003e` and `\u003c/think\u003e` tags to denote reasoning content. [DeepSeek-R1 source](https://huggingface.co/deepseek-ai/DeepSeek-R1) [Qwen3 source](https://qwenlm.github.io/blog/qwen3/)\n\nDifferent providers have different ways of providing reasoning content.\n\n- DeepSeek providers reasoning content is in the `reasoning_content` field. [source](https://api-docs.deepseek.com/guides/reasoning_model)\n- OpenRouter providers reasoning content is in the `reasoning` field. [source](https://openrouter.ai/docs/use-cases/reasoning-tokens)\n- Fireworks do not expose separate reasoning field, the reasoning content is within the message content. [source](https://docs.fireworks.ai/api-reference/post-chatcompletions)\n\n## Reasoning token count from OpenRouter\n\nTo get reasoning token count from OpenRouter, you need to pass `usage: { include: true }` to the API request.\n\nThis works in OpenAI TypeScript SDK as long as you use `any` type to avoid type errors. [source](https://openrouter.ai/docs/use-cases/usage-accounting)\n\n```ts\nconst openaiResponse = await openai.chat.completions.create({\n  model,\n  messages,\n  usage: {\n    include: true,\n  },\n} as any);\n```\n\n## Temperature and sampling parameters\n\nTemperature range supported is different for different providers.\n\n- OpenAI models support temperature range of `[0, 2]`. [source](https://platform.openai.com/docs/api-reference/responses/create#responses-create-temperature)\n- Anthropic Claude models support temperature range of `[0, 1]`. [source](https://docs.anthropic.com/en/api/messages#body-temperature)\n- Google Gemini models support temperature range of `[0, 2]`. [source](https://ai.google.dev/api/generate-content#generationconfig)\n\nTemperature and sampling parameters are not supported by all models.\n\n- Some OpenAI models (e.g. `o3-mini` and `codex-mini-latest`) do not support temperature (and other sampling parameters like `top_p`, `top_k`, etc.), sending a temperature value will result in an error. [source](https://github.com/openai/openai-python/issues/2072)\n- Vercel AI SDK will always send a temperature value (default to `0` if you don't specify) to the models, even if the model does not support it. [source](https://ai-sdk.dev/docs/ai-sdk-core/settings#temperature) [issue](https://github.com/vercel/ai/issues/5609) [source code](https://github.com/vercel/ai/blob/05c2da505b4564074456ab9c544c9266cdd2244a/packages/ai/core/prompt/prepare-call-settings.ts#L101C35-L101C35)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparadite%2Fmodel-quirks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fparadite%2Fmodel-quirks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fparadite%2Fmodel-quirks/lists"}