{"id":23552483,"url":"https://github.com/xlisp/ai-any-text-clusterer","last_synced_at":"2025-04-28T13:46:52.281Z","repository":{"id":39452589,"uuid":"237601809","full_name":"xlisp/ai-any-text-clusterer","owner":"xlisp","description":"AI any text or file clusterer \u0026 sorting","archived":false,"fork":false,"pushed_at":"2024-10-31T15:20:24.000Z","size":2627,"stargazers_count":14,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-30T10:11:24.648Z","etag":null,"topics":["ai","classifier","embedded","kmeans","machine-learning","ml","nomic-embed-text"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xlisp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-01T11:02:40.000Z","updated_at":"2025-02-17T15:46:18.000Z","dependencies_parsed_at":"2024-09-06T17:10:52.860Z","dependency_job_id":"b7cc0878-f674-42b2-a165-f28d30a634d7","html_url":"https://github.com/xlisp/ai-any-text-clusterer","commit_stats":null,"previous_names":["xlisp/emacs-spark-nlp","xlisp/ai-any-text-clusterer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xlisp%2Fai-any-text-clusterer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xlisp%2Fai-any-text-clusterer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xlisp%2Fai-any-text-clusterer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xlisp%2Fai-any-text-clusterer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xlisp","download_url":"https://codeload.github.com/xlisp/ai-any-text-clusterer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251321850,"owners_count":21570811,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","classifier","embedded","kmeans","machine-learning","ml","nomic-embed-text"],"created_at":"2024-12-26T11:11:20.003Z","updated_at":"2025-04-28T13:46:52.271Z","avatar_url":"https://github.com/xlisp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AI any text clusterer \u0026 sorting\n* Why: When your log files are very large and there are so many of them, and you don't know where to start, you can use ai-any-text-clusterer to classify your logs or files, git logs and other texts, so that you can clearly see where the relevant information is.\n\n## Feature\n\n- [x] File clustering (matching rules)\n- [x] Filter markdown todos or other pattern\n- [x] Git log clustering\n- [ ] process status clustering (pipe as input)\n- [ ] Log clustering\n- [ ] File sorting\n- [ ] Visualization operate \u0026 process\n- [ ] Support GPT ask for search or sorting\n\n## Init\n\n* Setup python env\n```sh\nconda create -n ai-any-text-clusterer python=3.11\nconda activate ai-any-text-clusterer\npoetry install\n```\n* [Ollama](https://ollama.com/) run embed model\n```sh\nollama run nomic-embed-text\n```\n\n## Usage\n* command\n```sh\n# function name: find_files_with_chinese_names, get_todo_items, get_git_log, get_pattern_items ...\n# When there are a lot of content or files, it is recommended that the n_clusters value is larger, such as 20. When there are fewer files, the n_clusters value is recommended to be 5\npython ai_any_text_clusterer.py \u003cfunction_name\u003e \u003cindex_file_name\u003e \u003cn_clusters\u003e \u003cwork_path\u003e\n```\n* run filter markdown\n\n```sh\n$ prunp ai_any_text_clusterer.py get_pattern_items  get_pattern_items.index 20 /Users/clojure/Documents/my_markdown_notes \"^.*(?:Breakthrough|Revolution).*$\"\n\n```\n* run git log clusterer\n```sh\n$ ai-any-text-clusterer  main @ poetry run python ai_any_text_clusterer.py get_git_log get_git_log.index 5 /Users/clojure/Desktop/ai-any-text-clusterer\nLoading embeddings from FAISS index...\nGroup 1:\n  [f9816067d95d20ba18cf3e8238ca3c0be252866e] Add git log\n  [9712fbd2c4611230b666dd3a02ee0bd76d832c68] https://ollama.com/ install embed model\n  [b519b481cf68833a116eb79e250a22b8ca02c2e5] Filter markdown todos\n  [338e72e4e42eef9900eba7749dc88dee302d735e] Add visualization.gif\n\nGroup 2:\n  [c23c23bfa72af712153fdd0596f19472acfb7f2b] Add Usage\n  [99e49f6f0b165e5c6058ca9b7828170e9006ac73] Add why\n\nGroup 3:\n  [631d8cd8c50a2989100a951c06e3bfb5cc6d8fea] Add function_name and index_file_name\n  [7b26ecf07645522e40375779fd1e720a9f293a9a] add find_files_with_chinese_names\n  [b1b515937cccfa5dff9b482e79bfa6dd14f96078] Add files classifier\n\nGroup 4:\n  [c5bd5863e0ade7a558b1bf8eb8367416e7305df3] rename\n  [31bef97ccda56f8b1989bc555ca4fd7a9d537f53] rename\n  [55d4057cbd775beb910363e6902dbab534a65000] rename\n\nGroup 5:\n  [c9f8e2351e0c456feb7c30754d261ba42505ec87] Add setup, use poetry\n```\n\n## Visualization\n\n![](./visualization.gif)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxlisp%2Fai-any-text-clusterer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxlisp%2Fai-any-text-clusterer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxlisp%2Fai-any-text-clusterer/lists"}