https://github.com/crypto-artisan/openai-token-counter-with-tiktoken

counter the number of tokens with tiktoken
https://github.com/crypto-artisan/openai-token-counter-with-tiktoken

Last synced: 2 months ago
JSON representation

counter the number of tokens with tiktoken

• Host: GitHub
• URL: https://github.com/crypto-artisan/openai-token-counter-with-tiktoken
• Owner: crypto-artisan
• Created: 2024-02-17T06:52:12.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-02-17T06:52:35.000Z (over 1 year ago)
• Last Synced: 2025-01-19T22:23:14.491Z (4 months ago)
• Language: Python
• Size: 2.93 KB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
# Open AI Token Counter Sample Script

Generally, token counts vary upon the various model.

[Visit this link for futher details](https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken)