https://github.com/actions/ai-inference
An action for calling AI models with GitHub Models
https://github.com/actions/ai-inference
Last synced: about 1 month ago
JSON representation
An action for calling AI models with GitHub Models
- Host: GitHub
- URL: https://github.com/actions/ai-inference
- Owner: actions
- License: mit
- Created: 2025-04-03T20:27:55.000Z (12 months ago)
- Default Branch: main
- Last Pushed: 2025-04-14T18:22:14.000Z (11 months ago)
- Last Synced: 2025-04-14T19:33:13.721Z (11 months ago)
- Language: TypeScript
- Homepage:
- Size: 1.22 MB
- Stars: 43
- Watchers: 2
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: CODEOWNERS
- Security: SECURITY.md
- Support: SUPPORT.md
Awesome Lists containing this project
- awesome-continuous-ai - actions/ai-inference
README
# AI Inference in GitHub Actions
[](https://github.com/super-linter/super-linter)
[](https://github.com/actions/typescript-action/actions/workflows/check-dist.yml)
[](https://github.com/actions/typescript-action/actions/workflows/codeql-analysis.yml)
Use AI models from [GitHub Models](https://github.com/marketplace/models) in
your workflows.
## Usage
Create a workflow to use the AI inference action:
```yaml
name: 'AI inference'
on: workflow_dispatch
jobs:
inference:
permissions:
models: read
runs-on: ubuntu-latest
steps:
- name: Test Local Action
id: inference
uses: actions/ai-inference@v1
with:
prompt: 'Hello!'
- name: Print Output
id: output
run: echo "${{ steps.inference.outputs.response }}"
```
### Using a prompt file
You can also provide a prompt file instead of an inline prompt. The action
supports both plain text files and structured `.prompt.yml` files:
```yaml
steps:
- name: Run AI Inference with Text File
id: inference
uses: actions/ai-inference@v1
with:
prompt-file: './path/to/prompt.txt'
```
### Using GitHub prompt.yml files
For more advanced use cases, you can use structured `.prompt.yml` files that
support templating, custom models, and JSON schema responses:
```yaml
steps:
- name: Run AI Inference with Prompt YAML
id: inference
uses: actions/ai-inference@v1
with:
prompt-file: './.github/prompts/sample.prompt.yml'
input: |
var1: hello
var2: ${{ steps.some-step.outputs.output }}
var3: |
Lorem Ipsum
Hello World
file_input: |
var4: ./path/to/long-text.txt
var5: ./path/to/config.json
```
#### Simple prompt.yml example
```yaml
messages:
- role: system
content: Be as concise as possible
- role: user
content: 'Compare {{a}} and {{b}}, please'
model: openai/gpt-4o
```
#### Prompt.yml with JSON schema support
```yaml
messages:
- role: system
content: You are a helpful assistant that describes animals using JSON format
- role: user
content: |-
Describe a {{animal}}
Use JSON format as specified in the response schema
model: openai/gpt-4o
responseFormat: json_schema
jsonSchema: |-
{
"name": "describe_animal",
"strict": true,
"schema": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "The name of the animal"
},
"habitat": {
"type": "string",
"description": "The habitat the animal lives in"
}
},
"additionalProperties": false,
"required": [
"name",
"habitat"
]
}
}
```
Variables in prompt.yml files are templated using `{{variable}}` format and are
supplied via the `input` parameter in YAML format. Additionally, you can
provide file-based variables via `file_input`, where each key maps to a file
path.
### Using a system prompt file
In addition to the regular prompt, you can provide a system prompt file instead
of an inline system prompt:
```yaml
steps:
- name: Run AI Inference with System Prompt File
id: inference
uses: actions/ai-inference@v1
with:
prompt: 'Hello!'
system-prompt-file: './path/to/system-prompt.txt'
```
### Read output from file instead of output
This can be useful when model response exceeds actions output limit
```yaml
steps:
- name: Test Local Action
id: inference
uses: actions/ai-inference@v1
with:
prompt: 'Hello!'
- name: Use Response File
run: |
echo "Response saved to: ${{ steps.inference.outputs.response-file }}"
cat "${{ steps.inference.outputs.response-file }}"
```
### Using custom headers
You can include custom HTTP headers in your API requests, which is useful for integrating with API Management platforms, adding tracking information, or routing requests through custom gateways.
#### YAML format (recommended for multiple headers)
```yaml
steps:
- name: AI Inference with Azure APIM
id: inference
uses: actions/ai-inference@v1
with:
prompt: 'Analyze this code for security issues...'
endpoint: ${{ secrets.APIM_ENDPOINT }}
token: ${{ secrets.APIM_KEY }}
custom-headers: |
Ocp-Apim-Subscription-Key: ${{ secrets.APIM_SUBSCRIPTION_KEY }}
serviceName: code-review-workflow
env: production
team: security
computer: github-actions
```
#### JSON format (alternative for compact syntax)
```yaml
steps:
- name: AI Inference with Custom Headers
id: inference
uses: actions/ai-inference@v1
with:
prompt: 'Hello!'
custom-headers: '{"X-Custom-Header": "value", "X-Team": "engineering", "X-Request-ID": "${{ github.run_id }}"}'
```
#### Use cases for custom headers
- **API Management**: Integrate with Azure APIM, AWS API Gateway, Kong, or other API management platforms
- **Request tracking**: Add correlation IDs, request IDs, or workflow identifiers
- **Rate limiting**: Include quota or tier information for custom rate limiting
- **Multi-tenancy**: Identify teams, services, or environments
- **Observability**: Add metadata for logging, monitoring, and debugging
- **Routing**: Control request routing through custom gateways or load balancers
**Header name requirements**: Header names must follow the HTTP token syntax defined in RFC 7230 (which permits underscores). For maximum compatibility with intermediaries and tooling, we recommend using only alphanumeric characters and hyphens.
**Security note**: Always use GitHub secrets for sensitive header values like API keys, tokens, or passwords. The action automatically masks common sensitive headers (containing `key`, `token`, `secret`, `password`, or `authorization`) in logs.
### GitHub MCP Integration (Model Context Protocol)
This action now supports **read-only** integration with the GitHub-hosted Model
Context Protocol (MCP) server, which provides access to GitHub tools like
repository management, issue tracking, and pull request operations.
#### Authentication
You can authenticate the MCP server with **either**:
1. **Personal Access Token (PAT)** – user-scoped token
2. **GitHub App Installation Token** (`ghs_…`) – short-lived, app-scoped token
> The built-in `GITHUB_TOKEN` is **not** accepted by the MCP server.
> Using a **GitHub App installation token** is recommended in most CI environments because it is short-lived and least-privilege by design.
#### Enabling MCP in the action
Set `enable-github-mcp: true` and provide a token via `github-mcp-token`.
```yaml
steps:
- name: AI Inference with GitHub Tools
id: inference
uses: actions/ai-inference@v1.2
with:
prompt: 'List my open pull requests and create a summary'
enable-github-mcp: true
token: ${{ secrets.USER_PAT }} # or a ghs_ installation token
```
If you want, you can use separate tokens for the AI inference endpoint
and the GitHub MCP server:
```yaml
steps:
- name: AI Inference with Separate MCP Token
id: inference
uses: actions/ai-inference@v1.2
with:
prompt: 'List my open pull requests and create a summary'
enable-github-mcp: true
token: ${{ secrets.GITHUB_TOKEN }}
github-mcp-token: ${{ secrets.USER_PAT }} # or a ghs_ installation token
```
#### Configuring GitHub MCP Toolsets
By default, the GitHub MCP server provides a standard set of tools (`context`, `repos`, `issues`, `pull_requests`, `users`). You can customize which toolsets are available by specifying the `github-mcp-toolsets` parameter:
```yaml
steps:
- name: AI Inference with Custom Toolsets
id: inference
uses: actions/ai-inference@v2
with:
prompt: 'Analyze recent workflow runs and check security alerts'
enable-github-mcp: true
token: ${{ secrets.USER_PAT }}
github-mcp-toolsets: 'repos,issues,pull_requests,actions,code_security'
```
**Available toolsets:**
See: [Tool configuration](https://github.com/github/github-mcp-server/blob/main/README.md#tool-configuration)
When MCP is enabled, the AI model will have access to GitHub tools and can
perform actions like searching issues and PRs.
## Inputs
Various inputs are defined in [`action.yml`](action.yml) to let you configure
the action:
| Name | Description | Default |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------ |
| `token` | Token to use for inference. Typically the GITHUB_TOKEN secret | `github.token` |
| `prompt` | The prompt to send to the model | N/A |
| `prompt-file` | Path to a file containing the prompt (supports .txt and .prompt.yml formats). If both `prompt` and `prompt-file` are provided, `prompt-file` takes precedence | `""` |
| `input` | Template variables in YAML format for .prompt.yml files (e.g., `var1: value1` on separate lines) | `""` |
| `file_input` | Template variables in YAML where values are file paths. The file contents are read and used for templating | `""` |
| `system-prompt` | The system prompt to send to the model | `"You are a helpful assistant"` |
| `system-prompt-file` | Path to a file containing the system prompt. If both `system-prompt` and `system-prompt-file` are provided, `system-prompt-file` takes precedence | `""` |
| `model` | The model to use for inference. Must be available in the [GitHub Models](https://github.com/marketplace?type=models) catalog | `openai/gpt-4o` |
| `endpoint` | The endpoint to use for inference. If you're running this as part of an org, you should probably use the org-specific Models endpoint | `https://models.github.ai/inference` |
| `max-tokens` | The max number of tokens to generate | 200 |
| `temperature` | The sampling temperature to use (0-1) | `""` |
| `top-p` | The nucleus sampling parameter to use (0-1) | `""` |
| `enable-github-mcp` | Enable Model Context Protocol integration with GitHub tools | `false` |
| `github-mcp-token` | Token to use for GitHub MCP server (defaults to the main token if not specified). | `""` |
| `custom-headers` | Custom HTTP headers to include in API requests. Supports both YAML format (`header1: value1`) and JSON format (`{"header1": "value1"}`). Useful for API Management platforms, rate limiting, and request tracking. | `""` |
## Outputs
The AI inference action provides the following outputs:
| Name | Description |
| --------------- | ----------------------------------------------------------------------- |
| `response` | The response from the model |
| `response-file` | The file path where the response is saved (useful for larger responses) |
## Required Permissions
In order to run inference with GitHub Models, the GitHub AI inference action
requires `models` permissions.
```yml
permissions:
contents: read
models: read
```
## Publishing a New Release
This project includes a helper script, [`script/release`](./script/release)
designed to streamline the process of tagging and pushing new releases for
GitHub Actions. For more information, see
[Versioning](https://github.com/actions/toolkit/blob/master/docs/action-versioning.md)
in the GitHub Actions toolkit.
GitHub Actions allows users to select a specific version of the action to use,
based on release tags. This script simplifies this process by performing the
following steps:
1. **Retrieving the latest release tag:** The script starts by fetching the most
recent SemVer release tag of the current branch, by looking at the local data
available in your repository.
1. **Prompting for a new release tag:** The user is then prompted to enter a new
release tag. To assist with this, the script displays the tag retrieved in
the previous step, and validates the format of the inputted tag (vX.X.X). The
user is also reminded to update the version field in package.json.
1. **Tagging the new release:** The script then tags a new release and syncs the
separate major tag (e.g. v1, v2) with the new release tag (e.g. v1.0.0,
v2.1.2). When the user is creating a new major release, the script
auto-detects this and creates a `releases/v#` branch for the previous major
version.
1. **Pushing changes to remote:** Finally, the script pushes the necessary
commits, tags and branches to the remote repository. From here, you will need
to create a new release in GitHub so users can easily reference the new tags
in their workflows.
## License
This project is licensed under the terms of the MIT open source license. Please
refer to [MIT](./LICENSE.txt) for the full terms.
## Contributions
Contributions are welcome! See the [Contributor's Guide](CONTRIBUTING.md).