{"id":21456142,"url":"https://github.com/cworld1/novel-analysis","last_synced_at":"2025-03-17T03:25:57.471Z","repository":{"id":244184198,"uuid":"780951610","full_name":"cworld1/novel-analysis","owner":"cworld1","description":"A simple project for analyzing Chinese novels","archived":false,"fork":false,"pushed_at":"2024-10-23T15:46:18.000Z","size":5841,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-23T20:40:55.817Z","etag":null,"topics":["data-analysis","novel"],"latest_commit_sha":null,"homepage":"https://cworld1.github.io/novel-analysis/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cworld1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-02T13:24:16.000Z","updated_at":"2024-10-23T15:46:31.000Z","dependencies_parsed_at":"2024-06-13T09:57:22.719Z","dependency_job_id":"8a47fd0c-b585-47d4-8fbe-566723caa691","html_url":"https://github.com/cworld1/novel-analysis","commit_stats":null,"previous_names":["cworld1/novel-analysis"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fnovel-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fnovel-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fnovel-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fnovel-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cworld1","download_url":"https://codeload.github.com/cworld1/novel-analysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243966716,"owners_count":20376108,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","novel"],"created_at":"2024-11-23T05:14:34.147Z","updated_at":"2025-03-17T03:25:57.441Z","avatar_url":"https://github.com/cworld1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Novel Analysis\n\n\u003c!-- [![GitHub stars](https://img.shields.io/github/stars/cworld1/novel-analysis?style=flat-square)](https://github.com/cworld1/novel-analysis/stargazers)\n[![GitHub commit activity](https://img.shields.io/github/commit-activity/y/cworld1/novel-analysis?label=commits\u0026style=flat-square)](https://github.com/cworld1/novel-analysis/commits)\n[![GitHub license](https://img.shields.io/github/license/cworld1/novel-analysis?style=flat-square)](https://github.com/cworld1/novel-analysis/blob/main/LICENSE) --\u003e\n\nA simple project for analyzing Chinese novels. All data is crawled from [红袖读书](https://www.hongxiu.com/)\n\n## Main technology stack\n\n- Python: Flask, Pyecharts, statsmodels\n- Node.js: Bun, Vite, React, Ant Design\n- Database: Json, CSV\n\n## Screenshots\n\nMain UI for the dashboard:\n\n![alt text](assets/PixPin_2024-06-17_17-24-10.png)\n![alt text](assets/PixPin_2024-06-17_17-26-48.png)\n![alt text](assets/PixPin_2024-06-17_17-27-13.png)\n\n\u003cdetails\u003e\n\u003csummary\u003eMore screenshots\u003c/summary\u003e\n\n![alt text](assets/PixPin_2024-06-17_17-24-39.png)\n![alt text](assets/PixPin_2024-06-17_17-25-09.png)\n![alt text](assets/PixPin_2024-06-17_17-25-22.png)\n![alt text](assets/PixPin_2024-06-17_17-25-37.png)\n![alt text](assets/PixPin_2024-06-17_17-25-53.png)\n![alt text](assets/PixPin_2024-06-17_17-26-09.png)\n\n\u003c/details\u003e\n\n## Project Structure\n\n- `anal`: Analysis scripts\n- `data`: Data directory\n- `database`: database script\n- `crawl`: Crawler scripts\n- `fetch`: Basic info for api scripts\n- `web`: Frontend dashboard\n\n## Data Constructed\n\nData can be crawled from the script in the following steps. After running the crawler script, the data will be saved in the `data` directory.\n\nThe repository [Novel Data](https://github.com/cworld1/novel-data) also contains the data, which has been a submodule part of this project.\n\n## Local Development\n\nClone the repository:\n\n```shell\ngit clone https://github.com/cworld1/novel-analysis.git\n# If you wang to also get data, you can also clone the submodule\n# git clone https://github.com/cworld1/novel-analysis.git --recurse-submodules\ncd novel-analysis\n```\n\n### Crawl \u0026 Analysis Server\n\nEnvironment requirements:\n\n- [Python](https://www.python.org/downloads/): 3.8+\n\n1. Create a virtual environment:\n\n   ```shell\n   python -m venv .venv\n   source .venv/bin/activate\n   ```\n\n   Or you can use tools for your own:\n\n   - [virtualenv](https://virtualenv.pypa.io/en/latest/)\n   - [pipenv](https://pipenv.pypa.io/en/latest/)\n   - [poetry](https://python-poetry.org/)\n   - [conda](https://docs.conda.io/en/latest/)\n   - [uv](https://github.com/astral-sh/uv/)\n\n2. Install the dependencies:\n\n   ```shell\n   pip install -r requirements.txt\n   ```\n\n3. Run crawler script (if needed)\n\n   ```shell\n   python ./crawl/main.py\n   ```\n\n4. Run the server script\n\n   ```shell\n   python ./api.py\n   # or run: python -m flask --app ./api.py run\n   ```\n\n### Display Dashboard\n\nEnvironment requirements:\n\n- [Nodejs](https://nodejs.org/): 18.0.0+\n- [Bun](https://bunjs.dev/): 1.0.0+\n\n2. Install dependencies:\n\n   ```shell\n   bun install\n   ```\n\n3. Start the development server:\n\n   ```shell\n   bun dev\n   ```\n\n   This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.\n\n4. Some useful commands:\n\n   - `bun build`: Bundles the app into static files for production.\n   - `bun lint`: Lints the project for potential errors.\n   - `bun preview`: Preview the production build locally.\n\n## Contributions\n\nTo spend more time coding and less time fiddling with whitespace, this project uses code conventions and styles to encourage consistency. Code with a consistent style is easier (and less error-prone!) to review, maintain, and understand.\n\n### Be consistent\n\nIf the style guide is not explicit about a particular situation, the cardinal rule is to **be consistent**. For example, take a look at the surrounding code and follow its lead, or look for similar cases elsewhere in the codebase.\n\n### Expanding the ESLint configuration\n\nIf you are developing a production application, we recommend updating the configuration to enable type aware lint rules:\n\n- Configure the top-level `parserOptions` property like this:\n\n```js\nexport default {\n  // other rules...\n  parserOptions: {\n    ecmaVersion: \"latest\",\n    sourceType: \"module\",\n    project: [\"./tsconfig.json\", \"./tsconfig.node.json\"],\n    tsconfigRootDir: __dirname,\n  },\n};\n```\n\n- Replace `plugin:@typescript-eslint/recommended` to `plugin:@typescript-eslint/recommended-type-checked` or `plugin:@typescript-eslint/strict-type-checked`\n- Optionally add `plugin:@typescript-eslint/stylistic-type-checked`\n- Install [eslint-plugin-react](https://github.com/jsx-eslint/eslint-plugin-react) and add `plugin:react/recommended` \u0026 `plugin:react/jsx-runtime` to the `extends` list\n\n## Thanks\n\n- [红袖读书](https://www.hongxiu.com/)\n- [文心中文心理分析系统 (Text Mind)](http://ccpl.psych.ac.cn/textmind/)\n- [Jieba](https://github.com/fxsjy/jieba/)\n\n## License\n\nThis project is licensed under the GPL 3.0 License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcworld1%2Fnovel-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcworld1%2Fnovel-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcworld1%2Fnovel-analysis/lists"}