{"id":25492347,"url":"https://github.com/bernlewal/aicodementor","last_synced_at":"2025-09-13T00:41:10.937Z","repository":{"id":276692143,"uuid":"924384086","full_name":"BernLeWal/AICodeMentor","owner":"BernLeWal","description":"AI Code Mentor is a runtime environment designed for building applications with integrated autonomous AI agents.","archived":false,"fork":false,"pushed_at":"2025-08-28T13:35:00.000Z","size":8166,"stargazers_count":5,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-28T20:55:17.609Z","etag":null,"topics":["ai-agent","ai-agents","prompt-toolkit","runtime-environment","workflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BernLeWal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-29T22:39:06.000Z","updated_at":"2025-08-28T13:35:03.000Z","dependencies_parsed_at":"2025-06-24T11:29:23.899Z","dependency_job_id":"e6946bdd-5dc3-4c69-abac-4263b8c3d7ad","html_url":"https://github.com/BernLeWal/AICodeMentor","commit_stats":null,"previous_names":["bernlewal/aicodementor"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/BernLeWal/AICodeMentor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernLeWal%2FAICodeMentor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernLeWal%2FAICodeMentor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernLeWal%2FAICodeMentor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernLeWal%2FAICodeMentor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BernLeWal","download_url":"https://codeload.github.com/BernLeWal/AICodeMentor/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BernLeWal%2FAICodeMentor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274902526,"owners_count":25371078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-12T02:00:09.324Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agent","ai-agents","prompt-toolkit","runtime-environment","workflow"],"created_at":"2025-02-18T22:29:33.527Z","updated_at":"2025-09-13T00:41:10.922Z","avatar_url":"https://github.com/BernLeWal.png","language":"Python","readme":"# AI Code Mentor\n\nAI Code Mentor is a **runtime environment** designed for building applications with **integrated autonomous AI agents**. These applications are defined as **workflows**, written in Markdown files (`.wf.md`), which are interpreted and executed by the Code Mentor system.\n\nThe AI agents (multiple instances are supported) generate and execute commands (e.g., Bash shell commands) directly, feeding the output back to the AI for analysis and iteration. This allows AI agents to **self-improve their tasks** when necessary. Currently integrated are various cloud-platform tools, like OpenAI, Google Gemini, Anthropic Claude (with OpenAI's platform as default setting) and Huggingface Transformer based LLMs which can be executed self-hosted. \n\n## Features \u0026 Benefits\n\n### Key Features\n- **Local Execution**: Data remains local and is processed inside a **Docker container**, avoiding unnecessary cloud uploads.\n- **Autonomous Execution**: AI-generated commands are executed automatically within the container, eliminating the need for manual copy-pasting.\n- **Multi-Agent Support**: Multiple AI agents can operate simultaneously within the same workflow, distributing tasks efficiently.\n- **Structured Workflow Definition**: Workflows define clear goals, paths, branches, and alternative solutions, providing better AI context compared to free-text prompting.\n- **Scalability \u0026 Automation**: Execute workflows on multiple targets, e.g., grading 100+ source code submissions automatically.\n- **Traceability \u0026 Transparency**: All AI-generated commands and their results are logged, providing visibility into the execution process.\n\n\u003c!--ChatGPT's **Canvas is still limited**. It can execute commands also, but will do that in the cloud and is limited (currently) when it comes to deal with multiple files and complex execution environments.--\u003e\n\n### Example Use Case: Automatic Grading of Software Development Assignments\n\nAI Code Mentor enables **autonomous evaluation of source code projects** based on program requirements and specifications. The AI agents:\n1. Analyze student submissions.\n2. Provide feedback.\n3. Assign grades automatically.\n4. Iterate if necessary to refine evaluations.\n\nThis is particularly useful for software development educators, reducing manual grading effort while maintaining fairness and transparency.  \nFor more details see [AI CodeMentor – Automating the Evaluation of Programming Assignments](./docs/pitch/educators-pitch.md).\n\n---\n\n## Screenshots\n\n### Workflow Execution for a REST Service Evaluation\n![](./docs/Screenshot_run_workflow.png)\n\n### Console Output of an AI Code Mentor Run\n![](./docs/screenshot_run_console_output.png)\n\n---\n\n## Implementation Status\n\n🚀Latest News in version: 0.2.5\n- The **Simple web UI** to run and manage workflows and history, and also includes a simple editor docs see [docker/codementor/README.md](docker/codementor/README.md)\n- Workflow history and logs can be retrieved via REST API\n- Introduced **Shellbox** - a docker container dedicated to run shell commands, connected via SSH\n- Added RESTful API (see [api/openapi.yaml](app/api/openapi.yaml) ) and server mode (--server CLI param)\n\nVersion history see [app/version.py](./app/version.py)\n\n### Comming Enhancements\n- **layered context** to better support variables and configs, e.g. change LLM-model within workflows\n- support for deployment on **k8s/Open Shift**\n- **Python** command support\n\n### Future Ideas\n- add separate statistics object for workflow runner\n- **create agent** instead of \"Prompt: System\"\n- Enhance AI agent feedback loops for **self-improvement**: Adapt temperature Setting: start with 0, increase on IMPROVE path\n- Workflow Validation\n- Implement command execution **whitelists/blacklists** and a reputation mechanism for security.\n- Develop a **collaboration model** for AI agents.\n- Realize MCQ based on www.github.com/chaoshu1201/lo-mcq-gen , contact Shu for contrib\n- LLMs to add: Perplexity AI, SanaAI, Ms Copilot\n- Implement **PLaG** technique (see [docs/literature/PLaG.md](./docs/literature/PLaG.md)) as a sample - should fit perfectly to CodeMentor\n- CodeRunner Integration? (as docker-container)\n\n---\n\n## Research Questions\n\n### 1. How can AI Code Mentor achieve **full AI autonomy**?\n- Follow Google's [AI Agent definition](https://www.kaggle.com/whitepaper-agents):\n  - Integration with **external systems**.\n  - **Session-based interactions** with multi-turn inference.\n  - **Native tool integration** within the AI agent architecture.\n  - AI agents **learn from feedback** without external intervention.\n\n### 2. How does AI-driven software development differ from traditional methods?\n- AI-generated output is inherently **non-deterministic**, requiring flexible error-handling mechanisms.\n- Unlike structured programming, AI-driven execution lacks **formal grammars**, requiring **robust parsers** for interpreting agent outputs.\n- AI-generated outputs should be **iteratively improved** rather than statically parsed.\n\n---\n\n## Usage\n\n### Prerequisites\n- Install **Docker**: [Get Docker](https://www.docker.com/products/docker-desktop)\n- You need to create an account and credentials on the cloud AI vendor of your choice (e.g. OpenAI, Google or Anthropic): see [docs/setup.md](docs/setup.md)\n- If you want to run the local AI-agents on your CUDA environment, see [docs/setup_cuda.md](docs/setup_cuda.md)\n- Create an .env file in the docker/ directory, based on the [docker/.env.sample](./docker/.env.sample) file.\n\n### Running AI Code Mentor\n\n**Run Docker Image:**  \n  The most easy way is to run it out of the box using the prepared Docker Image from Docker-Hub, see [docker/README.md](docker/README.md)\n  ```sh\n  docker run --rm -e OPENAI_API_KEY=\u003csk-yourkeyhere\u003e codepunx/codementor [options]\n  ```\n  or use the prepared docker-compose:\n  ```sh\n  docker compose up -d\n  ```\n\n  Remarks: This will use the default settings with OpenAI Platform as AI-agent and run AICodeMentor as Server (http://localhost:5000)\n  - SwaggerUI: http://localhost:5000/docs\n\n  You may use the browser as REST-Client: http://localhost:5000/\n\n\n### Command-Line Options:\n| Option | Description |\n|--------|-------------|\n|-h, --help | Show help message |\n|-v, --version | Display version info |\n|--verbose | Show log output in console |\n|--server | Run the REST server (default in docker) |\n\n### Arguments:\n- ```\u003cworkflow-file.md\u003e```: Markdown file defining the workflow (e.g., [workflows/check-toolchain.wf.md](workflows/check-toolchain.wf.md)).\n- ```[\u003ckey=value\u003e ...]```: Optional key-value parameters passed to the workflow.\n\n\n### Logging\n- The console (```stdout```/```stderr```) is reserved for CLI output.\n- Logs are stored in: [log/codementor.log](log/codementor.log)\n- Docker logs remain empty unless running with ```--verbose``` or in ```--server``` mode.\n\n\n### Create your own Workflows\n\nSee the [docs/tutorial](./docs/tutorial/README.md) to get started with creating your own worklfows\n\n---\n\n## Contents\n\n```\n.\n├── app                         # The CodeMentor sources (Python)\n├── artwork                     # Logos, etc.\n├── docker                      # Docker-Environment in which the CodeMentor will run\n│   ├── codementor              # - CodeMentor can execute BASH (and Python) commands\n│   └── codementor-java         # - CodeMentor can execute BASH commands and has a Java21+Maven environment\n├── docs                        # Documentation\n├── log                         # AI CodeMentor will output the application logs and history into that directory\n├── test                        # Unit-Tests for the Python application\n└── workflows                   # Contains the workflow files, which AI CodeMentor will execute\n```\n\n---\n\n## Development\n\nsee [CONTRIBUTION.md](CONTRIBUTION.md)\n\n### Setup\n\n**Install Python \u0026 Dependencies**\n  ```sh\n  sudo apt install python3 python3-pip -y\n  pip install virtualenv\n  python -m virtualenv .venv\n  source .venv/bin/activate\n  pip install -r requirements.txt\n  ```\n\n### Running \u0026 Debugging\n\n1. Set ```PYTHONPATH``` before running:\n\n    - Linux (Bash):\n      ```sh\n      export PYTHONPATH=$(pwd)\n      ```\n\n    - Windows (PowerShell):\n      ```powershell\n      $env:PYTHONPATH = (Get-Location).Path\n      ```\n\n2. Run the application with a workflow file:\n\n    ```sh\n    python app/main.py workflows/tutorial/lesson1.wf.md\n    ```\n\n3. Help \u0026 Usage:\n\n    ```sh\n    python app/main.py -h\n    ```\n\n4. Run with REST-Server and Web-Frontend:\n\n    ```sh\n    python app/main.py --server\n    ```\n\n    Server-URL: http://localhost:5000/  \n    see [docker/codementor/README.md](docker/codementor/README.md) for instructions\n\n5. Run the Test-Suite\n  \n    Pytest is used to run the unit-tests, start it from the project root directory:\n    ```sh\n    pytest\n    ```\n\n### Sample Workflow Execution\n\n```sh\npython app/main.py workflows/source-eval/paperless-sprint1.wf.md REPO_URL=https://github.com/BernLeWal/fhtw-bif5-swkom-paperless.git\n```\n\n---\n\n## Documentation\n\nFor software architecture and implementation details, see [docs/README.md](docs/README.md).\n\nUser documentation and Tutorials are here: [docs/tutorial](./docs/tutorial/README.md).\n\n---\n\n## License\n\nThis project is licensed under the **[AGPLv3](./LICENSE)** open-source license.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbernlewal%2Faicodementor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbernlewal%2Faicodementor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbernlewal%2Faicodementor/lists"}