Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/kangwonlee/gemini-python-tutor

AI Tutor for Coding Assignments
https://github.com/kangwonlee/gemini-python-tutor

pytest pytest-json-report python rest-api

Last synced: 6 months ago
JSON representation

AI Tutor for Coding Assignments

Awesome Lists containing this project

README 

          
[![Build Status](https://github.com/kangwonlee/gemini-python-tutor/workflows/build/badge.svg)](https://github.com/kangwonlee/gemini-python-tutor/actions)

[![GitHub release](https://img.shields.io/github/release/kangwonlee/gemini-python-tutor.svg)](https://github.com/kangwonlee/gemini-python-tutor/releases)



# AI Code Tutor



This GitHub Action uses AI to provide personalized feedback for student assignments in C/C++ and Python. It analyzes test results and code, identifying errors, suggesting optimizations, and explaining concepts clearly. Ideal for GitHub Classroom, it saves instructors time and ensures consistent, on-demand feedback with enhanced security against prompt injection attacks.



The AI tutor processes JSON test reports from `pytest-json-report`, generated by `pytest` tests wrapping C/C++ or Python code. It detects logic errors, recommends efficient algorithms, and links to relevant documentation. New security features sanitize inputs and use random delimiters to prevent malicious prompt manipulation.



## Key Features

- AI-powered feedback for C/C++ and Python assignments.

- Supports multiple JSON test reports from `pytest-json-report`.

- Analyzes multiple student code files (`.c`, `.cpp`, `.py`).

- Flexible LLM selection (Claude, Gemini, Grok, Nvidia NIM, Perplexity) with Gemini fallback.

- Customizable feedback language (e.g., English, Korean).

- Excludes common README content to optimize API usage.

- **Security Enhancements**: Sanitizes student code and READMEs to remove malicious patterns and wraps content with random delimiters to prevent prompt injection attacks.



## Prerequisites

- **Python Dependencies**:

  - Install required packages:

    ```bash

    pip install pytest pytest-json-report pytest-xdist requests

    ```

  - Generate JSON reports with:

    ```bash

    python -m pytest --json-report --json-report-indent=4 --json-report-file=report.json tests/test_file.py

    ```

  - See [pytest-json-report documentation](https://pypi.org/project/pytest-json-report/).

- **API Key**: At least one API key for supported LLMs (Claude, Gemini, Grok, Nvidia NIM, Perplexity), set as repository secrets (e.g., `INPUT_CLAUDE_API_KEY`, `INPUT_GOOGLE_API_KEY`).

- **Docker**: For C/C++ testing, use a Docker image with `clang`, `cmake`, and `pytest`.



## Usage

1. Add a workflow file (e.g., `.github/workflows/classroom.yml`) to your repository.

2. Configure it to run tests and invoke the AI tutor. Example for C/C++ assignments:



```yaml

name: Grade Assignment

on: [push, pull_request, workflow_dispatch]

jobs:

  grade:

    runs-on: ubuntu-latest

    env:

      CONTAINER_WORKSPACE: /app/workspace

      CONTAINER_TESTS: /tests

      CONTAINER_SRC: /app/workspace/src

      C_FILENAME: main.c

      WORKSPACE_OUTPUT: ${{ runner.temp }}/output

      CONTAINER_OUTPUT: /output

    steps:

      - uses: actions/checkout@v5

      - name: Set up environment

        run: pip install pytest==8.3.5 pytest-json-report==1.5.0 pytest-xdist==3.6.1 requests==2.32.4

      - name: Create output folder

        run: mkdir -p ${{ env.WORKSPACE_OUTPUT }}

      - name: Run C/C++ tests

        run: |

          docker run --rm \

            --volume ${{ github.workspace }}:${{ env.CONTAINER_WORKSPACE }}:ro \

            --volume ${{ env.WORKSPACE_OUTPUT }}:${{ env.CONTAINER_OUTPUT }}:rw \

            --workdir ${{ env.CONTAINER_TESTS }} \

            ghcr.io/kangwonlee/edu-base-cpp:4e0d6d8 \

            /bin/sh -c "cmake . -DCMAKE_BUILD_TYPE=Debug -DSTUDENT_DIR=${{ env.CONTAINER_WORKSPACE }} && make && python3 -m pytest --json-report --json-report-indent=4 --json-report-file=${{ env.CONTAINER_OUTPUT }}/report.json test_dynamic.py"

      - name: AI Code Tutor

        uses: kangwonlee/gemini-python-tutor@v0.3.12

        if: always()

        with:

          report-files: ${{ env.WORKSPACE_OUTPUT }}/report.json

          student-files: ${{ env.CONTAINER_SRC }}/${{ env.C_FILENAME }}

          readme-path: ${{ env.CONTAINER_WORKSPACE }}/README.md

          explanation-in: English

          model: gemini-2.5-flash

          INPUT_CLAUDE_API_KEY: ${{ secrets.INPUT_CLAUDE_API_KEY }}

          INPUT_GOOGLE_API_KEY: ${{ secrets.INPUT_GOOGLE_API_KEY }}

          INPUT_GROK_API_KEY: ${{ secrets.INPUT_GROK_API_KEY }}

          INPUT_NVIDIA_API_KEY: ${{ secrets.INPUT_NVIDIA_API_KEY }}

          INPUT_PERPLEXITY_API_KEY: ${{ secrets.INPUT_PERPLEXITY_API_KEY }}

        timeout-minutes: 10

```



### Notes

- **C/C++ Testing**: Tests can run in a Docker container with `pytest` wrapping C/C++ code (e.g., via `ctypes` for shared libraries, as in `test_dynamic.py`). Ensure JSON reports are generated.

- **Model Selection**: Set `model` to prefer an LLM (e.g., `gemini-2.5-flash`). If its key is unavailable, the action falls back to Gemini if `INPUT_GOOGLE_API_KEY` is set, or uses any available key.

- **Secrets**: Store API keys as repository secrets with `INPUT_` prefix (e.g., `INPUT_GOOGLE_API_KEY`) in Settings > Secrets and variables > Actions.

- **README Optimization**: Exclude common README content with:

  - Start: ``From here is common to all assignments.``

  - End: ``Until here is common to all assignments.``

  - Use double backticks (``).

- **Security**: Student code and READMEs are sanitized to remove malicious patterns (e.g., "ignore previous instructions") and wrapped with random delimiters to prevent prompt injection.



### Optimizing pytest for AI Feedback

- Use descriptive test names (e.g., `test_sum_range_for__valid_input`).

- Include clear assertion messages (e.g., `assert result == 10, f"Expected 10, got {result}"`).

- Keep tests focused for accurate AI interpretation.



## Inputs

| Input                   | Description                                      | Required | Default         |

|-------------------------|--------------------------------------------------|----------|-----------------|

| `report-files`          | Comma-separated JSON report files                | Yes      | None            |

| `student-files`         | Comma-separated student code files (`.c`, `.cpp`, `.py`) | Yes | None            |

| `readme-path`           | Path to assignment instructions (README.md)      | Yes      | None            |

| `explanation-in`        | Feedback language (e.g., English, Korean)        | No       | `English`       |

| `model`                 | Preferred LLM (e.g., `gemini-2.5-flash`, `claude-sonnet-4-20250514`) | No | `gemini-2.5-flash` |

| `INPUT_CLAUDE_API_KEY`  | Claude API key                                  | No*      | None            |

| `INPUT_GOOGLE_API_KEY`  | Google Gemini API key                           | No*      | None            |

| `INPUT_GROK_API_KEY`    | Grok API key                                    | No*      | None            |

| `INPUT_NVIDIA_API_KEY`  | Nvidia NIM API key                              | No*      | None            |

| `INPUT_PERPLEXITY_API_KEY` | Perplexity API key                          | No*      | None            |



*At least one API key is required.



### Example with Multiple Files

```yaml

with:

  report-files: 'report1.json,report2.json,reports/*.json'

  student-files: 'src/main.c,src/utils.c'

  readme-path: README.md

  explanation-in: English

  model: gemini-2.5-flash

  INPUT_GOOGLE_API_KEY: ${{ secrets.INPUT_GOOGLE_API_KEY }}

  INPUT_CLAUDE_API_KEY: ${{ secrets.INPUT_CLAUDE_API_KEY }}

```



## Outputs

- **Feedback**: Markdown written to `$GITHUB_STEP_SUMMARY`, visible in the GitHub Job Summary, and saved as `feedback.md` in artifacts.



## Limitations

- Primarily supports C/C++ and Python assignments via `pytest-json-report`.

- Requires at least one valid API key.

- C/C++ feedback relies on `pytest` tests wrapping compiled code.

- Prompt injection mitigated but not eliminated; use in controlled environments.



## Future Enhancements

- Auto-detect feedback language.

- Support additional programming languages.

- Add verbose mode for detailed feedback.

- Enhance prompt injection defenses with advanced parsing.



## Troubleshooting

Check GitHub Actions logs for details.



### Common Errors

- **API Key Issues**: "No API keys provided" – Ensure at least one API key is set in secrets.

- **Report File Issues**: "Report file not found" – Verify JSON report exists.

- **Student File Issues**: "Student file not found" – Check file paths and extensions.

- **Prompt Injection**: Malicious inputs are sanitized and wrapped with random delimiters, but monitor outputs for anomalies.



### Debugging Tips

- View logs in the "AI Code Tutor" job.

- Test locally with [act](https://github.com/nektos/act).

- Use `INPUT_FAIL-EXPECTED=true` for debugging expected test failures.



## Contact

Questions? Contact [https://github.com/kangwonlee](https://github.com/kangwonlee).



## License

BSD 3-Clause License + Do Not Harm.

Copyright (c) 2024 Kangwon Lee



## Acknowledgements

- Built using [python-github-action-template](https://github.com/cicirello/python-github-action-template) by Vincent A. Cicirello (MIT License).

- Gemini 2.5 Flash and Grok assisted with code and documentation.

- Registered as #C-2024-034203, #C-2024-035473, #C-2025-016393, and #C-2025-027967 with the Korea Copyright Commission.