{"id":13640058,"url":"https://github.com/promptslab/promptify","last_synced_at":"2026-10-17T04:08:00.290Z","repository":{"id":65489276,"uuid":"577429312","full_name":"promptslab/Promptify","owner":"promptslab","description":"Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research","archived":false,"fork":false,"pushed_at":"2026-03-27T01:09:45.000Z","size":2917,"stargazers_count":4642,"open_issues_count":61,"forks_count":362,"subscribers_count":51,"default_branch":"main","last_synced_at":"2026-10-14T07:57:55.501Z","etag":null,"topics":["chatgpt","chatgpt-api","chatgpt-python","gpt-3","gpt-3-prompts","gpt-4","gpt-4-api","gpt3-library","large-language-models","machine-learning","nlp","openai","prompt-engineering","prompt-toolkit","prompt-tuning","prompt-versioning","prompting","prompts","promptversioning","transformers"],"latest_commit_sha":null,"homepage":"https://discord.gg/m88xfYMbK6","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/promptslab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-12-12T18:06:32.000Z","updated_at":"2026-10-13T20:56:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"bcd537c8-5d96-4f04-9e5e-139f7a4cb33c","html_url":"https://github.com/promptslab/Promptify","commit_stats":{"total_commits":561,"total_committers":14,"mean_commits":40.07142857142857,"dds":0.124777183600713,"last_synced_commit":"27a53fa8e8f2a4d90f887d06ece65a44466f873a"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/promptslab/Promptify","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptslab%2FPromptify","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptslab%2FPromptify/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptslab%2FPromptify/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptslab%2FPromptify/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/promptslab","download_url":"https://codeload.github.com/promptslab/Promptify/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/promptslab%2FPromptify/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":344836268,"owners_count":38332409,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-10-03T21:59:58.778Z","status":"online","status_checked_at":"2026-10-17T02:00:06.547Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatgpt","chatgpt-api","chatgpt-python","gpt-3","gpt-3-prompts","gpt-4","gpt-4-api","gpt3-library","large-language-models","machine-learning","nlp","openai","prompt-engineering","prompt-toolkit","prompt-tuning","prompt-versioning","prompting","prompts","promptversioning","transformers"],"created_at":"2024-08-02T01:01:07.376Z","updated_at":"2026-10-17T04:08:00.280Z","avatar_url":"https://github.com/promptslab.png","language":"Python","funding_links":[],"categories":["Prompts"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003cimg width=\"110px\" src=\"https://raw.githubusercontent.com/promptslab/Promptify/main/assets/logo.png\"\u003e\n\u003ch1\u003ePromptify\u003c/h1\u003e\u003c/div\u003e\n\u003c!-- \n\u003ch2 align=\"center\"\u003ePromptify\u003c/h2\u003e --\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cp align=\"center\"\u003eTask-based NLP engine with Pydantic structured outputs, built-in evaluation, and LiteLLM as the universal LLM backend. Think \"scikit-learn for LLM-powered NLP\".\n\u003c/p\u003e\n\u003c/p\u003e\n\n \u003ch4 align=\"center\"\u003e\n  \u003ca href=\"https://github.com/promptslab/Promptify/blob/main/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/License-Apache_2.0-blue.svg\" alt=\"Promptify is released under the Apache 2.0 license.\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/promptify/\"\u003e\n    \u003cimg src=\"https://badge.fury.io/py/Promptify.svg\" alt=\"PyPI version\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"http://makeapullrequest.com\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square\" alt=\"http://makeapullrequest.com\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://discord.gg/m88xfYMbK6\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/Discord-Community-orange\" alt=\"Community\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"#\"\u003e\n    \u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"colab\" /\u003e\n  \u003c/a\u003e\n\u003c/h4\u003e\n\n\u003cimg width=\"910px\" src=\"https://raw.githubusercontent.com/promptslab/Promptify/main/assets/dark.png\"\u003e\n\n## Installation\n\n### With pip\n\nRequires Python 3.9+.\n\n```bash\npip install promptify\n```\n\nor\n\n```bash\npip install git+https://github.com/promptslab/Promptify.git\n```\n\nFor evaluation metrics support:\n```bash\npip install promptify[eval]\n```\n\n## Quick Tour\n\n### 3-Line NER\n\n```python\nfrom promptify import NER\n\nner = NER(model=\"gpt-4o-mini\", domain=\"medical\")\nresult = ner(\"The patient is a 93-year-old female with a medical history of chronic right hip pain, osteoporosis, hypertension, depression, and chronic atrial fibrillation admitted for evaluation and management of severe nausea and vomiting and urinary tract infection\")\n```\n\n**Output:**\n```python\nNERResult(entities=[\n    Entity(text=\"93-year-old\", label=\"AGE\"),\n    Entity(text=\"chronic right hip pain\", label=\"CONDITION\"),\n    Entity(text=\"osteoporosis\", label=\"CONDITION\"),\n    Entity(text=\"hypertension\", label=\"CONDITION\"),\n    Entity(text=\"depression\", label=\"CONDITION\"),\n    Entity(text=\"chronic atrial fibrillation\", label=\"CONDITION\"),\n    Entity(text=\"severe nausea and vomiting\", label=\"SYMPTOM\"),\n    Entity(text=\"urinary tract infection\", label=\"CONDITION\"),\n])\n```\n\n### Classification\n\n```python\nfrom promptify import Classify\n\nclf = Classify(model=\"gpt-4o-mini\", labels=[\"positive\", \"negative\", \"neutral\"])\nresult = clf(\"Amazing product! Best purchase I've ever made.\")\n# Classification(label=\"positive\", confidence=0.95)\n```\n\n### Question Answering\n\n```python\nfrom promptify import QA\n\nqa = QA(model=\"gpt-4o-mini\")\nanswer = qa(\"Einstein was born in Ulm in 1879.\", question=\"Where was Einstein born?\")\n# Answer(answer=\"Ulm\", evidence=\"Einstein was born in Ulm\", confidence=0.98)\n```\n\n### Custom Task with Any Pydantic Schema\n\n```python\nfrom promptify import Task\nfrom pydantic import BaseModel\n\nclass MovieReview(BaseModel):\n    sentiment: str\n    rating: float\n    key_themes: list[str]\n\ntask = Task(model=\"gpt-4o\", output_schema=MovieReview, instruction=\"Analyze this movie review.\")\nreview = task(\"Nolan's best work. Stunning visuals but the plot drags.\")\n# MovieReview(sentiment=\"mostly positive\", rating=7.5, key_themes=[\"visuals\", \"pacing\"])\n```\n\n### Any Provider - Just Change the Model String\n\n```python\nner_openai = NER(model=\"gpt-4o-mini\")\nner_claude = NER(model=\"claude-sonnet-4-20250514\")\nner_local  = NER(model=\"ollama/llama3\")\n```\n\n### Batch Processing\n\n```python\nresults = ner.batch([\"text1\", \"text2\", \"text3\"], max_concurrent=10)\n```\n\n### Async Support\n\n```python\nresult = await ner.acall(\"Patient has diabetes\")\n```\n\n### Built-in Evaluation\n\n```python\nfrom promptify.eval import evaluate\n\nscores = evaluate(task=ner, dataset=labeled_data, metrics=[\"precision\", \"recall\", \"f1\"])\n# {\"precision\": 0.92, \"recall\": 0.88, \"f1\": 0.90}\n```\n\n## Features\n\n- **2-3 lines of code** for any NLP task -no training data required\n- **Pydantic structured outputs** -type-safe results, not raw strings\n- **Any LLM provider** via LiteLLM -OpenAI, Anthropic, Google, Ollama, Azure, and 100+ more\n- **Built-in tasks** -NER, Classification (binary/multiclass/multilabel), QA, Summarization, Relation Extraction, SQL Generation, and more\n- **Custom tasks** -bring your own Pydantic schema for any structured output\n- **Few-shot examples** -easily add examples to improve accuracy\n- **Domain specialization** -pass `domain=\"medical\"` or any domain for context-aware prompts\n- **Batch processing** -async concurrency under the hood for processing multiple texts\n- **Async support** -native `await` support with `acall()`\n- **Evaluation framework** -precision, recall, F1, accuracy, exact match, ROUGE metrics\n- **Safe parser** -fallback JSON completion for providers without native structured outputs (no `eval()`)\n- **Cost tracking** -built-in token usage and cost monitoring via `get_cost_summary()`\n\n### Supported NLP Tasks\n\n| Task | Class | Output Schema |\n|------|-------|---------------|\n| Named Entity Recognition | `NER` | `NERResult` (list of `Entity`) |\n| Binary Classification | `Classify` | `Classification` |\n| Multiclass Classification | `Classify` | `Classification` |\n| Multilabel Classification | `Classify(multi_label=True)` | `MultiLabelResult` |\n| Question Answering | `QA` | `Answer` |\n| Summarization | `Summarize` | `Summary` |\n| Relation Extraction | `ExtractRelations` | `ExtractionResult` |\n| Tabular Extraction | `ExtractTable` | `ExtractionResult` |\n| Question Generation | `GenerateQuestions` | list of `GeneratedQuestion` |\n| SQL Generation | `GenerateSQL` | `SQLQuery` |\n| Text Normalization | `NormalizeText` | normalized text |\n| Topic Modelling | `ExtractTopics` | list of topics |\n| Custom Task | `Task` | any Pydantic `BaseModel` |\n\n## Community\n\u003cdiv align=\"center\"\u003e\nIf you are interested in Prompt-Engineering, LLMs, and NLP, please consider joining \u003ca href=\"https://discord.gg/m88xfYMbK6\"\u003ePromptsLab\u003c/a\u003e\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\u003cimg alt=\"Join us on Discord\" src=\"https://img.shields.io/discord/1069129502472556587?color=5865F2\u0026logo=discord\u0026logoColor=white\"\u003e\n\u003c/div\u003e\n\n\n\n```\n\n@misc{Promptify2022,\n  title = {Promptify: Structured Output from LLMs},\n  author = {Pal, Ankit},\n  year = {2022},\n  howpublished = {\\url{https://github.com/promptslab/Promptify}},\n  note = {Prompt-Engineering components for NLP tasks in Python}\n}\n\n```\n\n## 💁 Contributing\n\nWe welcome any contributions to our open source project, including new features, improvements to infrastructure, and more comprehensive documentation. \nPlease see the [contributing guidelines](contribute.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpromptslab%2Fpromptify","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpromptslab%2Fpromptify","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpromptslab%2Fpromptify/lists"}