{"id":37076214,"url":"https://github.com/wanglaoshi/wanglaoshi-pypi","last_synced_at":"2026-01-14T08:58:05.075Z","repository":{"id":259832132,"uuid":"871452472","full_name":"WangLaoShi/wanglaoshi-pypi","owner":"WangLaoShi","description":"Useful tools for DA ML DL","archived":false,"fork":false,"pushed_at":"2025-12-04T08:01:40.000Z","size":5427,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-07T15:40:55.728Z","etag":null,"topics":["data-analysis","deep-learning","machine-learning","unitility"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/wanglaoshi/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WangLaoShi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-10-12T02:32:54.000Z","updated_at":"2025-12-04T08:01:43.000Z","dependencies_parsed_at":"2024-10-28T08:18:06.661Z","dependency_job_id":"d2c55007-f4af-4c9a-adc9-edfaa788b0b2","html_url":"https://github.com/WangLaoShi/wanglaoshi-pypi","commit_stats":null,"previous_names":["wanglaoshi/wanglaoshi-pypi"],"tags_count":24,"template":false,"template_full_name":null,"purl":"pkg:github/WangLaoShi/wanglaoshi-pypi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WangLaoShi%2Fwanglaoshi-pypi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WangLaoShi%2Fwanglaoshi-pypi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WangLaoShi%2Fwanglaoshi-pypi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WangLaoShi%2Fwanglaoshi-pypi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WangLaoShi","download_url":"https://codeload.github.com/WangLaoShi/wanglaoshi-pypi/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WangLaoShi%2Fwanglaoshi-pypi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28414727,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-analysis","deep-learning","machine-learning","unitility"],"created_at":"2026-01-14T08:58:04.577Z","updated_at":"2026-01-14T08:58:05.069Z","avatar_url":"https://github.com/WangLaoShi.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 王老师 WangLaoShi\n\n## 项目介绍\n\n总结一些在学习过程中的知识点，以及一些学习资料。\n\n## 项目结构\n\n```\nWangLaoShi\n├── README.md\n├── wanglaoshi\n│   ├── version.py\n```\n\n## 项目版本\n\n- 0.0.1 初始化版本，项目开始\n- 0.0.2 增加列表输出\n- 0.0.3 增加字典输出,使用 Rich 输出\n- 0.0.4 实现 JupyterNotebook 环境创建\n- 0.0.5 增加几个有用的库\n- 0.0.6 修改获取 version 的方法\n- 0.0.7 增加获取当前安装包的版本号，增加获取当前每一个安装包最新版本的方法\n- 0.0.8 增加对数据文件的基本分析的部分\n- 0.0.9 增加 jinja2 的模板输出的 Analyzer\n- 0.10.0 增加 no_waring,字体获取，安装字体\n- 0.10.6 增加 Analyzer 的使用部分(需要 statsmodels)\n- 0.10.7 增加 MLDL 部分(需要 sklearn,torch)\n- 0.10.10 增加分析结果 Render notebook 部分\n- 0.10.13 修复分析结果\n- 0.11.1 增加 static 和 template 修改 html 报告生成\n- 0.11.2 添加 js 的引用\n\n## 安装方式\n\n### 1. 源码安装方式\n\n* 检出项目\n* 进入项目目录\n* 执行`python setup.py install`\n* 安装完成\n\n### 2. pip安装方式\n\n```shell\npip install wanglaoshi\n```\n\n## 使用方法\n\n### 1. 创建新的环境\n    \n```python\nfrom wanglaoshi import JupyterEnv as JE\nJE.jupyter_kernel_list()\nJE.install_kernel()\n# 按照提示输入环境名称\n```\n### 2. 获取当前环境常用库版本\n    \n```python\nfrom wanglaoshi import VERSIONS as V\nV.check_all_versions()\n```\n### 3. 获取当前环境所有库\n\n```python\nfrom wanglaoshi import VERSIONS as V\nV.check_all_installed()\n```\n### 4. 获取当前环境所有库最新版本\n\n```python\nfrom wanglaoshi import VERSIONS as V\nV.check_all_installed_with_latest()\n```\n\n### 5. 得到一个数据文件的基本的分析页面\n\n#### 示例调用\n\n```python\n\"\"\"\nDataAnalyzer 使用示例\n这个示例展示了如何使用 DataAnalyzer 进行数据分析并生成报告\n\"\"\"\n\nimport pandas as pd\nimport numpy as np\nfrom wanglaoshi.Analyzer import DataAnalyzer\nimport os\n\ndef create_sample_data():\n    \"\"\"创建示例数据\"\"\"\n    # 创建随机数据\n    np.random.seed(42)\n    n_samples = 1000\n    \n    # 数值型数据\n    data = {\n        'age': np.random.normal(35, 10, n_samples),  # 年龄\n        'income': np.random.lognormal(10, 1, n_samples),  # 收入\n        'height': np.random.normal(170, 10, n_samples),  # 身高\n        'weight': np.random.normal(65, 15, n_samples),  # 体重\n        'satisfaction': np.random.randint(1, 6, n_samples),  # 满意度评分\n    }\n    \n    # 添加一些缺失值\n    for col in ['age', 'income', 'height', 'weight']:\n        mask = np.random.random(n_samples) \u003c 0.05  # 5%的缺失值\n        data[col][mask] = np.nan\n    \n    # 添加一些异常值\n    data['income'][np.random.choice(n_samples, 5)] = data['income'].max() * 2\n    \n    # 创建分类数据\n    data['gender'] = np.random.choice(['男', '女'], n_samples)\n    data['education'] = np.random.choice(['高中', '本科', '硕士', '博士'], n_samples)\n    data['occupation'] = np.random.choice(['工程师', '教师', '医生', '销售', '其他'], n_samples)\n    \n    # 创建时间数据\n    dates = pd.date_range(start='2023-01-01', periods=n_samples, freq='D')\n    data['date'] = dates\n    \n    return pd.DataFrame(data)\n\ndef basic_analysis_demo():\n    \"\"\"基础分析示例\"\"\"\n    print(\"=== 基础分析示例 ===\")\n    \n    # 创建示例数据\n    df = create_sample_data()\n    print(\"\\n数据预览:\")\n    print(df.head())\n    \n    # 创建分析器实例\n    analyzer = DataAnalyzer(df)\n    \n    # 基本统计分析\n    print(\"\\n基本统计信息:\")\n    basic_stats = analyzer.basic_statistics()\n    print(basic_stats)\n    \n    # 正态性检验\n    print(\"\\n正态性检验结果:\")\n    normality_test = analyzer.normality_test()\n    print(normality_test)\n    \n    # 缺失值分析\n    print(\"\\n缺失值分析:\")\n    missing_analysis = analyzer.missing_value_analysis()\n    print(missing_analysis)\n    \n    # 异常值分析\n    print(\"\\n异常值分析:\")\n    outlier_analysis = analyzer.outlier_analysis()\n    print(outlier_analysis)\n    \n    # 重复值分析\n    print(\"\\n重复值分析:\")\n    duplicate_analysis = analyzer.duplicate_analysis()\n    print(duplicate_analysis)\n\ndef advanced_analysis_demo():\n    \"\"\"高级分析示例\"\"\"\n    print(\"\\n=== 高级分析示例 ===\")\n    \n    # 创建示例数据\n    df = create_sample_data()\n    analyzer = DataAnalyzer(df)\n    \n    # 相关性分析\n    print(\"\\n相关性分析:\")\n    correlation_matrix = analyzer.correlation_analysis()\n    print(correlation_matrix)\n    \n    # 多重共线性分析\n    print(\"\\n多重共线性分析:\")\n    multicollinearity = analyzer.multicollinearity_analysis()\n    print(multicollinearity)\n    \n    # 主成分分析\n    print(\"\\n主成分分析:\")\n    pca_analysis = analyzer.pca_analysis()\n    print(pca_analysis)\n\ndef visualization_demo():\n    \"\"\"可视化示例\"\"\"\n    print(\"\\n=== 可视化示例 ===\")\n    \n    # 创建示例数据\n    df = create_sample_data()\n    analyzer = DataAnalyzer(df)\n    \n    # 分布图\n    print(\"\\n生成分布图...\")\n    for column in ['age', 'income', 'height', 'weight']:\n        print(f\"\\n{column} 的分布图:\")\n        img_base64 = analyzer.plot_distribution(column)\n        print(f\"图片已生成，base64长度: {len(img_base64)}\")\n    \n    # 相关性热图\n    print(\"\\n生成相关性热图...\")\n    heatmap_base64 = analyzer.plot_correlation_heatmap()\n    print(f\"热图已生成，base64长度: {len(heatmap_base64)}\")\n\ndef report_generation_demo():\n    \"\"\"报告生成示例\"\"\"\n    print(\"\\n=== 报告生成示例 ===\")\n    \n    # 创建示例数据\n    df = create_sample_data()\n    analyzer = DataAnalyzer(df)\n    \n    # 生成HTML报告\n    print(\"\\n生成分析报告...\")\n    analyzer.generate_report(\"analysis_report.html\")\n    print(\"报告已生成: analysis_report.html\")\n\ndef file_analysis_demo():\n    \"\"\"文件分析示例\"\"\"\n    print(\"\\n=== 文件分析示例 ===\")\n    \n    # 创建示例数据文件\n    print(\"\\n创建示例数据文件...\")\n    data_dir = \"example_data\"\n    os.makedirs(data_dir, exist_ok=True)\n    \n    # 创建CSV文件\n    df1 = create_sample_data()\n    df1.to_csv(os.path.join(data_dir, \"sample_data1.csv\"), index=False)\n    \n    # 创建另一个CSV文件，使用不同的数据\n    df2 = create_sample_data()  # 使用不同的随机种子\n    df2.to_csv(os.path.join(data_dir, \"sample_data2.csv\"), index=False)\n    \n    # 创建Excel文件\n    df3 = create_sample_data()  # 使用不同的随机种子\n    df3.to_excel(os.path.join(data_dir, \"sample_data3.xlsx\"), index=False)\n    \n    print(f\"示例数据文件已创建在 {data_dir} 目录下\")\n    \n    # 分析单个文件\n    print(\"\\n分析单个文件示例:\")\n    print(\"分析 sample_data1.csv...\")\n    from wanglaoshi import analyze_data\n    analyze_data(\n        os.path.join(data_dir, \"sample_data1.csv\"),\n        \"analysis_report_single.html\"\n    )\n    print(\"单个文件分析报告已生成: analysis_report_single.html\")\n    \n    # 分析多个文件\n    print(\"\\n分析多个文件示例:\")\n    print(\"分析目录下的所有数据文件...\")\n    from wanglaoshi import analyze_multiple_files\n    analyze_multiple_files(data_dir, \"reports\")\n    print(\"多个文件的分析报告已生成在 reports 目录下\")\n    \n    # 清理示例文件\n    print(\"\\n清理示例文件...\")\n    import shutil\n    shutil.rmtree(data_dir)\n    print(\"示例数据文件已清理\")\n\ndef notebook_demo():\n    \"\"\"Jupyter Notebook示例\"\"\"\n    print(\"\\n=== Jupyter Notebook示例 ===\")\n    print(\"\"\"\n    在Jupyter Notebook中使用:\n    \n    ```python\n    import pandas as pd\n    from wanglaoshi import DataAnalyzer\n    \n    # 创建或加载数据\n    df = pd.DataFrame(...)\n    \n    # 创建分析器实例\n    analyzer = DataAnalyzer(df)\n    \n    # 在notebook中显示分析报告\n    analyzer.analyze_notebook()\n    ```\n    \"\"\")\n\ndef main():\n    \"\"\"主函数\"\"\"\n    print(\"DataAnalyzer 使用示例\\n\")\n    \n    # 运行各个示例\n    basic_analysis_demo()\n    advanced_analysis_demo()\n    visualization_demo()\n    report_generation_demo()\n    file_analysis_demo()\n    notebook_demo()\n\nif __name__ == \"__main__\":\n    main() \n```\n\n#### 数据分析示例\n\n`analyzer_demo.py` 提供了完整的数据分析示例，包含以下功能：\n\n1. **基础分析示例**：\n   - 数据预览和基本统计信息\n   - 正态性检验\n   - 缺失值分析\n   - 异常值分析\n   - 重复值分析\n\n2. **高级分析示例**：\n   - 相关性分析：计算变量间的相关系数矩阵\n   - 多重共线性分析：使用VIF检测变量间的多重共线性\n   - 主成分分析：降维和特征提取\n\n3. **可视化示例**：\n   - 分布图：展示数值变量的分布情况\n   - 相关性热图：直观展示变量间的相关关系\n\n4. **报告生成示例**：\n   - 生成HTML格式的分析报告\n   - 包含所有分析结果和可视化图表\n\n5. **文件分析示例**：\n   - 支持分析单个数据文件\n   - 支持批量分析多个数据文件\n   - 支持CSV、Excel、JSON等多种格式\n\n6. **Jupyter Notebook示例**：\n   - 在Jupyter环境中使用DataAnalyzer\n   - 交互式数据分析和可视化\n\n\n### 6. 取消错误输出\n\n```python\nfrom wanglaoshi import JupyterEnv as JE\nJE.no_warning()\n```\n\n### 7. Wget 功能\n\n基本功能：\n - 支持从 URL 下载文件\n - 自动从 URL 提取文件名\n - 支持指定输出目录和自定义文件名\n - 显示下载进度条\n\n使用方法：\n\n```python\nfrom WebGetter import Wget\n\n# 创建下载器实例\ndownloader = Wget(\n    url='https://example.com/file.zip',\n    output_dir='./downloads',\n    filename='custom_name.zip'\n)\n\n# 开始下载\ndownloader.download()\n```\n\n## 8. 字体安装\n\n```python\n# 这里用的是 SimHei 字体，可以根据自己的需要更改\nfrom wanglaoshi import JupyterFont as JF\nJF.matplotlib_font_init()\n```\n\n## 9. 批量数据分析（适合比赛）\n\n```python\nfrom wanglaoshi import Analyzer as A\nimport seaborn as sns\nimport pandas as pd\n\n# 获取示例数据集\n# 方法1：使用seaborn自带的数据集\ntips = sns.load_dataset('tips')  # 餐厅小费数据集\ntips.to_csv('tips.csv', index=False)\n\n# 方法2：使用sklearn自带的数据集\nfrom sklearn.datasets import load_iris\niris = load_iris()\niris_df = pd.DataFrame(iris.data, columns=iris.feature_names)\niris_df['target'] = iris.target\niris_df.to_csv('iris.csv', index=False)\n\n# 创建测试文件夹\nimport os\nos.makedirs('test_data', exist_ok=True)\n\n# 将数据集移动到测试文件夹\nimport shutil\nshutil.move('tips.csv', 'test_data/tips.csv')\nshutil.move('iris.csv', 'test_data/iris.csv')\n\n# 分析数据集\nA.analyze_multiple_files('test_data', output_dir='reports')\n```\n\n批量分析功能特点：\n- 支持多种数据格式（CSV、Excel、JSON）\n- 自动生成每个数据文件的详细分析报告\n- 异常值分析包含：\n  - Z-score方法：识别极端和中度异常值\n  - IQR方法：提供数据分布特征和异常值范围\n  - 综合建议：基于两种方法的结果给出处理建议\n- 报告包含可视化图表和详细的解释说明\n\n分析完成后，您可以在 `reports` 目录下找到生成的分析报告：\n- `tips_report.html`：餐厅小费数据集的分析报告\n- `iris_report.html`：鸢尾花数据集的分析报告\n\n## 10. MLDL (单独安装 torch，pip install torch)\n\n```python\n\"\"\"使用示例\"\"\"\nfrom MLDL import *\n# 1. 数据预处理\npreprocessor = DataPreprocessor()\ndf = pd.DataFrame({\n    'A': [1, 2, np.nan, 4, 5],\n    'B': ['a', 'b', 'a', 'c', 'b']\n})\ndf_processed = preprocessor.handle_missing_values(df, method='mean')\ndf_encoded = preprocessor.encode_categorical(df_processed, ['B'])\n\n# 2. 特征工程\nengineer = FeatureEngineer()\ndf_features = engineer.create_polynomial_features(df_encoded, ['A'], degree=2)\n\n# 3. 机器学习模型\nml_model = MLModel('logistic')\nX = df_features[['A', 'A_power_2']]\ny = df_features['B']\nml_model.train(X, y)\nmetrics = ml_model.evaluate()\nprint(\"ML模型评估结果:\", metrics)\n\n# 4. 深度学习模型\ndl_model = DLModel(input_size=2, hidden_size=4, output_size=3)\nX_tensor = torch.FloatTensor(X.values)\ny_tensor = torch.LongTensor(y.values)\ndl_model.train(X_tensor, y_tensor, epochs=100)\n\n# 5. 模型评估\nevaluator = ModelEvaluator()\ny_pred = ml_model.predict(X)\nevaluator.plot_confusion_matrix(y, y_pred)\n```\n\n## 11. render notebook\n\n```python\n# 方法1：使用工具函数\nfrom wanglaoshi import analyze_notebook\nimport pandas as pd\n\ndf = pd.read_csv('your_data.csv')\nanalyze_notebook(df)\n\n# 方法2：使用类方法\nfrom wanglaoshi import DataAnalyzer\nimport pandas as pd\n\ndf = pd.read_csv('your_data.csv')\nanalyzer = DataAnalyzer(df)\nanalyzer.analyze_notebook()\n```\n\n\n## 建议的版本对照关系\n\n1. numpy https://numpy.org/news/\n2. pandas https://pandas.pydata.org/pandas-docs/stable/whatsnew/index.html\n3. sklearn https://scikit-learn.org/stable/whats_new.html","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwanglaoshi%2Fwanglaoshi-pypi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwanglaoshi%2Fwanglaoshi-pypi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwanglaoshi%2Fwanglaoshi-pypi/lists"}