https://github.com/cleanlab/cleanlab-tlm
Python client library for Cleanlab Trustworthy Language Model
https://github.com/cleanlab/cleanlab-tlm
Last synced: about 2 months ago
JSON representation
Python client library for Cleanlab Trustworthy Language Model
- Host: GitHub
- URL: https://github.com/cleanlab/cleanlab-tlm
- Owner: cleanlab
- License: mit
- Created: 2025-01-29T19:03:39.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-04-10T23:12:06.000Z (about 2 months ago)
- Last Synced: 2025-04-10T23:31:46.518Z (about 2 months ago)
- Language: Python
- Size: 200 KB
- Stars: 11
- Watchers: 2
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Cleanlab Trustworthy Language Model (TLM) - Reliability and explainability added to every LLM output
[](https://github.com/cleanlab/cleanlab-tlm/actions/workflows/ci.yml) [](https://pypi.org/project/cleanlab-tlm) [](https://pypi.org/project/cleanlab-tlm)
In one line of code, Cleanlab TLM adds real-time evaluation of every response in GenAI, RAG, LLM, and Agent systems.
## Setup
This tutorial requires a TLM API key. Get one [here](https://tlm.cleanlab.ai/).
```console
export CLEANLAB_TLM_API_KEY=
```
Install the package:
```console
pip install cleanlab-tlm
```
## Usage
To get started, copy the code below to try your own prompt or score existing prompt/response pairs with ease.
```python
from cleanlab_tlm import TLM
tlm = TLM(options={"log": ["explanation"], "model": "gpt-4o-mini"}) # GPT, Claude, etc.
out = tlm.prompt("What's the third month of the year alphabetically?")
print(out)
```
TLM returns a dictionary containing `response`, `trustworthiness_score`, and any requested optional fields like `explanation`.
```json
{
"response": "March.",
"trustworthiness_score": 0.4590804375945598,
"explanation": "Found an alternate response: December"
}
```
## Why TLM?
- **Trustworthiness Scores**: Each response comes with a trustworthiness score, helping you [reliably](https://cleanlab.ai/blog/trustworthy-language-model/) gauge the likelihood of hallucinations.
- **Higher accuracy**: Rigorous [benchmarks](https://cleanlab.ai/blog/trustworthy-language-model/) show TLM consistently produces more accurate results than other LLMs like o3/o1, GPT 4o, and Claude.
- **Scalable API**: Designed to handle large datasets, TLM is suitable for most enterprise applications, including data extraction, tagging/labeling, Q&A (RAG), and more.
## Documentation
Comprehensive documentation along with tutorials and examples can be found [here](https://help.cleanlab.ai/tlm).
## License
`cleanlab-tlm` is distributed under the terms of the [MIT](https://spdx.org/licenses/MIT.html) license.