{"id":14959636,"url":"https://github.com/shure-dev/logllm","last_synced_at":"2026-03-03T11:38:29.869Z","repository":{"id":254036334,"uuid":"845295423","full_name":"shure-dev/logllm","owner":"shure-dev","description":"Automatically extract ML experimental conditions from your Python scripts with GPT4, and save them via WandB.","archived":false,"fork":false,"pushed_at":"2024-08-30T01:21:50.000Z","size":892,"stargazers_count":1,"open_issues_count":28,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-31T03:31:39.836Z","etag":null,"topics":["chatgpt","gpt4","large-language-models","llm","llmops","logging","machine-learning","mlflow","mlops","wandb"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shure-dev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-21T01:14:07.000Z","updated_at":"2024-08-27T16:21:06.000Z","dependencies_parsed_at":"2024-08-26T11:33:31.536Z","dependency_job_id":"686ff0d1-03ef-49d6-b6e8-f568e608d08f","html_url":"https://github.com/shure-dev/logllm","commit_stats":null,"previous_names":["shure-dev/llm-powered-wandb","shure-dev/logllm","shure-dev/log-llm"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shure-dev%2Flogllm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shure-dev%2Flogllm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shure-dev%2Flogllm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shure-dev%2Flogllm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shure-dev","download_url":"https://codeload.github.com/shure-dev/logllm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238008512,"owners_count":19401266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatgpt","gpt4","large-language-models","llm","llmops","logging","machine-learning","mlflow","mlops","wandb"],"created_at":"2024-09-24T13:20:18.153Z","updated_at":"2025-10-24T16:32:14.323Z","avatar_url":"https://github.com/shure-dev.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003cdiv align=\"center\"\u003e\n\n[🌐 **Project Website**](https://logllm.tiiny.site/) | [💬 **Discord Community**](https://discord.gg/3xvUV6xcKW)\n\n# 🚀 **LogLLM** 🚀\n\n**LLM-Enhanced ML Experiment Logging System**\n\n`Ever found yourself lost in a maze of changing experiment conditions in your early ML scripts? 😵 No worries—here’s the solution you’ve been looking for! ✨`\n\n![image](./images/logllm-overview.png)\n\n\n\u003c/div\u003e\n\n---\n\n## ✨ **Features**\n\n**🔍 Automatic Logging**: Effortlessly extracts code from Jupyter Notebook files using GPT4o, saving the logs to Weights \u0026 Biases (W\u0026B) for seamless tracking and analysis.\n\n**💬 Natural Language Queries**: Effortlessly extract and display information from WandB logs using natural language queries. Simply ask in plain language, and get insights directly. For example, you could ask, \"What was the most effective method using decision trees so far?\" and receive the relevant data instantly.\n\n## ⚙️ **Installation**\n\nTo install the package, run the following command in your terminal:\n\n```bash\ngit clone https://github.com/shure-dev/logllm.git\npip install -e .\n```\n\n```bash\nexport OPENAI_API_KEY=\"your-openai-api-key\"\nwandb login\n```\n\nThis command installs the package in editable mode, allowing you to modify the code and see changes without reinstalling.\n\n\n## 🚀 **Usage**\n\nHere’s a simplified example of how to use the package:\n\n### 🔍 Automatic Logging\n\n**Sample Notebook Script: `sample-script.ipynb`**\n\n```python\n###\n# Your machine learning script goes here.\n# Experimental conditions are extracted from here.\n###\n\nfrom logllm import log_llm\n\nnotebook_path = \"sample-script.ipynb\"  # The target file to log\n\nlog_llm(notebook_path)\n```\n\n### 💬 Natural Language Queries\n- developing now,,,\n\n\n## 🧠 **How It Works: Simple and Powerful!**\n\n**LLM**(`Our Prompt` + `Your ML Script`) = `Extracted Experimental Conditions`\n\n### **Our Prompt:**\n\n```plaintext\nYou are an advanced machine learning experiment designer.\nExtract all experimental conditions and results for logging via W\u0026B API.\nAdd your original parameters in your JSON response if you want to log other parameters.\nExtract all information you can find in the given script as int, bool, or float values.\nIf you cannot describe conditions with int, bool, or float values, use a list of natural language.\nGive advice to improve accuracy.\nIf you use natural language, answers should be very short.\nDo not include information already provided in param_name_1 for `condition_as_natural_langauge`.\nOutput JSON schema example:\nThis is just an example, make changes as you see fit.\n{{\n    \"method\": \"str\",\n    \"dataset\": \"str\",\n    \"task\": \"str\",\n    \"is_advanced_method\": bool,\n    \"is_latest_method\": \"\",\n    \"accuracy\": \"\",\n    \"other_param_here\": \"\",\n    \"other_param_here\": \"\",\n    ...\n    \"condition_as_natural_langauge\": [\"Small dataset.\"],\n    \"advice_to_improve_acc\": [\"Use a bigger dataset.\", \"Use a simpler model.\"]\n}}\n```\n\n### **Your ML Script: `svc-sample.ipynb`**\n\n```python\nfrom sklearn import datasets\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.svm import SVC\nfrom sklearn.metrics import accuracy_score\n\niris = datasets.load_iris()\n\nX = iris.data[iris.target != 2]\ny = iris.target[iris.target != 2]\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\nmodel = SVC(kernel='linear')\nmodel.fit(X_train, y_train)\n\ny_pred = model.predict(X_test)\naccuracy = accuracy_score(y_test, y_pred)\n\nprint(f\"Accuracy: {accuracy:.2f}\")\n```\n\n### **Extracted Experimental Conditions:**\n\n```json\n{\n    \"method\": \"SVC\",\n    \"dataset\": \"Iris\",\n    \"task\": \"classification\",\n    \"is_advanced_method\": false,\n    \"is_latest_method\": \"\",\n    \"accuracy\": 1.00,\n    \"kernel\": \"linear\",\n    \"test_size\": 0.2,\n    \"random_state\": 42,\n    \"condition_as_natural_langauge\": [\n        \"Using linear kernel on SVC model.\",\n        \"Excluding class 2 from Iris dataset.\",\n        \"Splitting data into 80% training and 20% testing.\"\n    ],\n    \"advice_to_improve_acc\": [\n        \"Confirm dataset consistency.\",\n        \"Consider cross-validation for validation.\"\n    ]\n}\n```\n\n[📄 **Check the Demo Code**](https://github.com/shure-dev/logllm/blob/main/demos/svc-sample.ipynb)\n\n---\n\n## 🤝 **Contributing**\n\nContributions are welcome! If you have suggestions or improvements, please feel free to submit an issue or a pull request.\n\n## 📜 **License**\n\nThis project is licensed under the **MIT License**.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshure-dev%2Flogllm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshure-dev%2Flogllm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshure-dev%2Flogllm/lists"}