{"id":24562511,"url":"https://github.com/mendableai/llmstxt-generator","last_synced_at":"2025-06-12T20:06:14.522Z","repository":{"id":264929210,"uuid":"891163629","full_name":"mendableai/llmstxt-generator","owner":"mendableai","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-31T19:34:44.000Z","size":291,"stargazers_count":412,"open_issues_count":5,"forks_count":42,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-12T17:57:39.406Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://llmstxt.firecrawl.dev/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mendableai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-19T20:55:56.000Z","updated_at":"2025-06-12T12:42:00.000Z","dependencies_parsed_at":"2025-01-30T17:29:32.703Z","dependency_job_id":"efeff2fb-bd65-44ec-ba86-f328da17ad8e","html_url":"https://github.com/mendableai/llmstxt-generator","commit_stats":null,"previous_names":["mendableai/llmstxt-generator"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mendableai/llmstxt-generator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Fllmstxt-generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Fllmstxt-generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Fllmstxt-generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Fllmstxt-generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mendableai","download_url":"https://codeload.github.com/mendableai/llmstxt-generator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mendableai%2Fllmstxt-generator/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259522022,"owners_count":22870449,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-23T09:02:28.733Z","updated_at":"2025-06-12T20:06:14.517Z","avatar_url":"https://github.com/mendableai.png","language":"TypeScript","funding_links":[],"categories":["TypeScript","A01_文本生成_文本对话"],"sub_categories":["大语言对话模型及数据"],"readme":"# llms.txt Generator 🚀\n\nGenerate consolidated text files from websites for LLM training and inference. Powered by [@firecrawl_dev](https://twitter.com/firecrawl_dev) for web crawling and GPT-4-mini for text processing.\n\n## Features\n- Crawls websites and combines content into a single text file\n- Generates both standard (`llms.txt`) and full (`llms-full.txt`) versions\n- Web interface and API access available\n- No API key required for basic usage\n\n## Usage\n\n### Web Interface\nVisit [llmstxt.firecrawl.dev](https://llmstxt.firecrawl.dev) to generate files through the browser.\n\n### API Endpoint\n```\nGET https://llmstxt.firecrawl.dev/[YOUR_URL_HERE]\n```\n\nNote: Processing may take several minutes due to crawling and LLM operations.\n\n## Local Development\n\n### Prerequisites\nCreate a `.env` file with the following variables:\n```\nFIRECRAWL_API_KEY=\nSUPABASE_URL=\nSUPABASE_KEY=\nOPENAI_API_KEY=\n```\n\n### Installation\n```bash\nnpm install\nnpm run dev\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmendableai%2Fllmstxt-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmendableai%2Fllmstxt-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmendableai%2Fllmstxt-generator/lists"}