https://github.com/simonpierreboucher/embedding
A robust Python tool for generating embeddings from text files using OpenAI's API. This tool processes text files, splits them into chunks while preserving context headers, and generates embeddings using OpenAI's models, saving both text and embeddings in structured formats.
https://github.com/simonpierreboucher/embedding
api-rate-limiting automated-text-analysis context-preservation data-preprocessing embeddings-generation error-handling json-and-npy-formats machine-learning metadata-management natural-language-processing openai-api python-tool text-chunking text-embedding yaml-configuration
Last synced: 28 days ago
JSON representation
A robust Python tool for generating embeddings from text files using OpenAI's API. This tool processes text files, splits them into chunks while preserving context headers, and generates embeddings using OpenAI's models, saving both text and embeddings in structured formats.
- Host: GitHub
- URL: https://github.com/simonpierreboucher/embedding
- Owner: simonpierreboucher
- Created: 2024-11-13T18:07:52.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2024-11-13T18:16:11.000Z (5 months ago)
- Last Synced: 2024-11-13T19:25:04.202Z (5 months ago)
- Topics: api-rate-limiting, automated-text-analysis, context-preservation, data-preprocessing, embeddings-generation, error-handling, json-and-npy-formats, machine-learning, metadata-management, natural-language-processing, openai-api, python-tool, text-chunking, text-embedding, yaml-configuration
- Language: Python
- Homepage:
- Size: 10.7 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md