{"id":28753172,"url":"https://github.com/zpg6/azure-ai-inference-plus","last_synced_at":"2025-07-18T04:37:41.057Z","repository":{"id":297496020,"uuid":"997001033","full_name":"zpg6/azure-ai-inference-plus","owner":"zpg6","description":"Enhanced wrapper that makes Azure AI Inference SDK simple and reliable with automatic retry, JSON validation, and reasoning separation.","archived":false,"fork":false,"pushed_at":"2025-06-09T01:14:55.000Z","size":76,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-17T00:48:03.887Z","etag":null,"topics":["azure-ai","azure-ai-foundry","azure-sdk-for-python"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/azure-ai-inference-plus/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zpg6.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-05T19:45:02.000Z","updated_at":"2025-06-09T01:14:58.000Z","dependencies_parsed_at":"2025-06-05T20:04:47.920Z","dependency_job_id":"ae51ff83-bc0e-4be7-abc4-f8d7e03f84c0","html_url":"https://github.com/zpg6/azure-ai-inference-plus","commit_stats":null,"previous_names":["zpg6/azure-ai-inference-plus"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/zpg6/azure-ai-inference-plus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpg6%2Fazure-ai-inference-plus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpg6%2Fazure-ai-inference-plus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpg6%2Fazure-ai-inference-plus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpg6%2Fazure-ai-inference-plus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zpg6","download_url":"https://codeload.github.com/zpg6/azure-ai-inference-plus/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zpg6%2Fazure-ai-inference-plus/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265701429,"owners_count":23813751,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-ai","azure-ai-foundry","azure-sdk-for-python"],"created_at":"2025-06-17T00:38:56.571Z","updated_at":"2025-07-18T04:37:41.038Z","avatar_url":"https://github.com/zpg6.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# azure-ai-inference-plus\n\n**The easier way to use Azure AI Inference SDK** ✨\n\n[![PyPI Version](https://img.shields.io/pypi/v/azure-ai-inference-plus)](https://pypi.org/project/azure-ai-inference-plus/)\n[![PyPI Downloads](https://img.shields.io/pypi/dm/azure-ai-inference-plus)](https://pypi.org/project/azure-ai-inference-plus/)\n[![License: MIT](https://img.shields.io/pypi/l/azure-ai-inference-plus)](https://opensource.org/licenses/MIT)\n[![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)\n\nEnhanced wrapper that makes Azure AI Inference SDK simple and reliable with **automatic retry**, **JSON validation**, and **reasoning separation**.\n\n## Why Use This Instead?\n\n✅ **Reasoning separation** - automatically splits thinking from output (`.content` and `.reasoning`)  \n✅ **Automatic retries** - never lose requests to transient failures  \n✅ **JSON that works** - guaranteed valid JSON or automatic retry  \n✅ **One import** - no need for multiple Azure SDK imports  \n✅ **100% compatible** - drop-in replacement for Azure AI Inference SDK\n\n## 🛡️ Handles Real-World LLM Issues\n\nAutomatic retries for the errors you actually encounter in production:\n\n```\n🔄 Service overloaded (timeouts)     → Auto-retry with backoff\n🔄 Rate limits (429)                 → Smart retry timing\n🔄 Azure service hiccups (5xx)       → Exponential backoff\n🔄 Invalid JSON responses            → Re-request clean JSON\n🔄 Network timeouts                  → Multiple quick attempts\n```\n\n**Just works.** No manual error handling needed.\n\n## Installation\n\n```bash\npip install azure-ai-inference-plus\n```\n\nSupports Python 3.11+\n\n## Quick Start\n\n```python\nfrom azure_ai_inference_plus import ChatCompletionsClient, SystemMessage, UserMessage\n\n# Uses environment variables: AZURE_AI_ENDPOINT, AZURE_AI_API_KEY\nclient = ChatCompletionsClient()\n\nresponse = client.complete(\n    messages=[\n        SystemMessage(content=\"You are a helpful assistant.\"),\n        UserMessage(content=\"What's the capital of France?\"),\n    ],\n    max_tokens=100,\n    model=\"Codestral-2501\"\n)\n\nprint(response.choices[0].message.content)\n# \"The capital of France is Paris...\"\n```\n\n**Or with manual credentials (everything from one import!):**\n\n```python\nfrom azure_ai_inference_plus import ChatCompletionsClient, SystemMessage, UserMessage, AzureKeyCredential\n\nclient = ChatCompletionsClient(\n    endpoint=\"https://your-resource.services.ai.azure.com/models\",\n    credential=AzureKeyCredential(\"your-api-key\")\n)\n```\n\n## 🎯 Key Features\n\n### 🧠 Automatic Reasoning Separation\n\n**Game changer for reasoning models like DeepSeek-R1** - automatically separates thinking from output:\n\n```python\nresponse = client.complete(\n    messages=[\n        SystemMessage(content=\"You are a helpful assistant.\"),\n        UserMessage(content=\"What's 2+2? Think step by step.\"),\n    ],\n    model=\"DeepSeek-R1\",\n    reasoning_tags=[\"\u003cthink\u003e\", \"\u003c/think\u003e\"]  # ✨ Auto-separation\n)\n\n# Clean output without reasoning clutter\nprint(response.choices[0].message.content)\n# \"2 + 2 equals 4.\"\n\n# Access the reasoning separately\nprint(response.choices[0].message.reasoning)\n# \"Let me think about this step by step. 2 + 2 is a basic addition...\"\n```\n\n### ✅ Guaranteed Valid JSON\n\nNo more JSON parsing errors - automatic validation and retry.\n\n**Simple JSON (standard models like GPT-4o):**\n\n```python\nresponse = client.complete(\n    messages=[\n        SystemMessage(content=\"You are a helpful assistant that returns JSON.\"),\n        UserMessage(content=\"Give me Tokyo info as JSON with keys: name, country, population\"),\n    ],\n    max_tokens=500,\n    model=\"gpt-4o\",\n    response_format=\"json_object\"  # ✨ Auto-validation + retry\n)\n\n# Always valid JSON, no try/catch needed!\nimport json\ndata = json.loads(response.choices[0].message.content)  # ✅ Works perfectly\n```\n\n**JSON with reasoning models (like DeepSeek-R1):**\n\n```python\nresponse = client.complete(\n    messages=[\n        SystemMessage(content=\"You are a helpful assistant that returns JSON.\"),\n        UserMessage(content=\"Give me Paris info as JSON with keys: name, country, population\"),\n    ],\n    max_tokens=2000,  # More tokens needed for reasoning + JSON\n    model=\"DeepSeek-R1\",\n    response_format=\"json_object\",  # ✨ Clean JSON guaranteed\n    reasoning_tags=[\"\u003cthink\u003e\", \"\u003c/think\u003e\"]  # Required for reasoning separation\n)\n\n# Pure JSON - reasoning automatically stripped\ndata = json.loads(response.choices[0].message.content)  # {\"name\": \"Paris\", ...}\n\n# But reasoning is still accessible\nthinking = response.choices[0].message.reasoning  # \"Let me think about Paris...\"\n```\n\n_Note: JSON responses are automatically cleaned of markdown wrappers (like \\`\\`\\`json blocks) for reliable parsing._\n\n### 🔄 Smart Automatic Retries\n\nBuilt-in retry with exponential backoff - no configuration needed:\n\n```python\n# Automatically retries on failures (including timeouts) - just works!\nresponse = client.complete(\n    messages=[UserMessage(content=\"Tell me a joke\")],\n    model=\"Phi-4\"\n)\n```\n\n### ⚙️ Custom Retry Configuration\n\n```python\nfrom azure_ai_inference_plus import RetryConfig\n\n# Override default behavior (with smart timeout strategy)\nclient = ChatCompletionsClient(\n    connection_timeout=100.0,  # Better: 100s + retries vs 300s timeout\n    retry_config=RetryConfig(max_retries=5, delay_seconds=2.0)\n)\n```\n\n### 📢 Retry Callbacks (Optional Observability)\n\nGet notified when retries happen - perfect for logging and monitoring:\n\n```python\nfrom azure_ai_inference_plus import RetryConfig\n\ndef on_chat_retry(attempt, max_retries, exception, delay):\n    print(f\"🔄 Chat retry {attempt}/{max_retries}: {type(exception).__name__} - waiting {delay:.1f}s\")\n\ndef on_json_retry(attempt, max_retries, message):\n    print(f\"📝 JSON retry {attempt}/{max_retries}: {message}\")\n\n# Add callbacks to your retry config\nclient = ChatCompletionsClient(\n    retry_config=RetryConfig(\n        max_retries=3,\n        on_chat_retry=on_chat_retry,    # Called for general failures\n        on_json_retry=on_json_retry     # Called for JSON validation failures\n    )\n)\n\n# Now you'll see retry notifications:\n# 🔄 Chat retry 1/3: HttpResponseError - waiting 1.0s\n# 📝 JSON retry 2/3: Retry 2 after JSON validation failed\n```\n\n**Why callbacks?** The library doesn't print anything by default (clean for production), but callbacks let you add your own logging, metrics, or notifications exactly how you want them.\n\n## 🚀 Embeddings Too\n\n```python\nfrom azure_ai_inference_plus import EmbeddingsClient\n\nclient = EmbeddingsClient()\nresponse = client.embed(\n    input=[\"Hello world\", \"Python is great\"],\n    model=\"text-embedding-3-large\"\n)\n```\n\n## Environment Setup\n\nCreate a `.env` file:\n\n```bash\nAZURE_AI_ENDPOINT=https://your-resource.services.ai.azure.com/models\nAZURE_AI_API_KEY=your-api-key-here\n```\n\n## Migration from Azure AI Inference SDK\n\n**2 simple steps:**\n\n1. `pip install azure-ai-inference-plus`\n2. Change your import:\n\n   ```python\n   # Before\n   from azure.ai.inference import ChatCompletionsClient\n   from azure.ai.inference.models import SystemMessage, UserMessage\n   from azure.core.credentials import AzureKeyCredential\n\n   # After\n   from azure_ai_inference_plus import ChatCompletionsClient, SystemMessage, UserMessage, AzureKeyCredential\n   ```\n\nThat's it! Your existing code works unchanged with automatic retries and JSON validation.\n\n### Manual Credential Setup\n\n```python\nfrom azure_ai_inference_plus import ChatCompletionsClient, AzureKeyCredential\n\nclient = ChatCompletionsClient(\n    endpoint=\"https://your-resource.services.ai.azure.com/models\",\n    credential=AzureKeyCredential(\"your-api-key\")\n)\n```\n\n## Examples\n\nCheck out the [`examples/`](examples/) directory for complete demonstrations:\n\n- [`basic_usage.py`](examples/basic_usage.py) - Reasoning separation, JSON validation, retry features, and timeout strategy\n- [`embeddings_example.py`](examples/embeddings_example.py) - Embeddings with retry and credential setup\n- [`callbacks_example.py`](examples/callbacks_example.py) - Retry callbacks for logging and monitoring\n\nAll examples show real-world usage patterns and advanced features.\n\n## License\n\n[MIT](./LICENSE)\n\n## Contributing\n\nContributions are welcome! Whether it's bug fixes, feature additions, or documentation improvements, we appreciate your help in making this project better. For major changes or new features, please open an issue first to discuss what you would like to change.\n\n## Related Projects\n\n- [langchain-azure-ai-inference-plus](https://github.com/zpg6/langchain-azure-ai-inference-plus) - **The easier way to use Azure AI Inference SDK with LangChain ✨**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzpg6%2Fazure-ai-inference-plus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzpg6%2Fazure-ai-inference-plus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzpg6%2Fazure-ai-inference-plus/lists"}