https://github.com/kurtpayne/skillscan-security

Security scanner for AI agent skills and MCP tool bundles — prompt injection, IOC matching, malware detection, ML classifier
https://github.com/kurtpayne/skillscan-security

ai-agent ai-security llm-security mcp prompt-injection security skill-scanner static-analysis

Last synced: 3 months ago
JSON representation

Security scanner for AI agent skills and MCP tool bundles — prompt injection, IOC matching, malware detection, ML classifier

Host: GitHub
URL: https://github.com/kurtpayne/skillscan-security
Owner: kurtpayne
License: apache-2.0
Created: 2026-02-08T23:13:14.000Z (5 months ago)
Default Branch: main
Last Pushed: 2026-03-31T18:48:45.000Z (3 months ago)
Last Synced: 2026-03-31T20:38:48.432Z (3 months ago)
Topics: ai-agent, ai-security, llm-security, mcp, prompt-injection, security, skill-scanner, static-analysis
Language: Python
Homepage: https://skillscan.sh
Size: 25.8 MB
Stars: 1
Watchers: 0
Forks: 0
Open Issues: 6
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Codeowners: CODEOWNERS
- Security: SECURITY.md

Awesome Lists containing this project

README

[![CI](https://github.com/kurtpayne/skillscan-security/actions/workflows/ci.yml/badge.svg)](https://github.com/kurtpayne/skillscan-security/actions/workflows/ci.yml)
[![CodeQL](https://github.com/kurtpayne/skillscan-security/actions/workflows/codeql.yml/badge.svg)](https://github.com/kurtpayne/skillscan-security/actions/workflows/codeql.yml)
[![PyPI](https://img.shields.io/pypi/v/skillscan-security.svg)](https://pypi.org/project/skillscan-security/)
[![Docker Hub](https://img.shields.io/docker/v/kurtpayne/skillscan-security?label=docker)](https://hub.docker.com/r/kurtpayne/skillscan-security)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.11%2B-blue.svg)](pyproject.toml)

**Free. Private. Offline. No API key required.**

Security scanner for AI agent skills and MCP tool bundles. Part of the [SkillScan](https://skillscan.sh) project.

SkillScan Security catches the obvious stuff so you don't have to pay Claude to find it. It runs entirely on your machine — no network calls, no telemetry, no tokens spent — and returns deterministic verdicts before you ever send a skill to an online scanner.

Use it as a free pre-filter in your CI pipeline. If it blocks, you know immediately. If it passes, you've already eliminated the easy wins before handing off to a deeper (and more expensive) analysis layer.

Verdicts: `allow` · `warn` · `block`

Default policy: `strict`.

---

## Why SkillScan First

Online AI scanners (Invariant, Lakera Guard, and others) are excellent at nuanced intent analysis. They are also billed per token. Running them on every skill in a large repository is expensive.

SkillScan handles the deterministic layer for free:

- Download-and-execute chains
- Secret exfiltration patterns
- Credential harvesting instructions
- Malicious binary artifacts
- Known-bad IOC domains and IPs
- Vulnerable dependency versions
- Prompt injection and instruction override attempts
- Social engineering credential requests

If SkillScan blocks it, you don't need to spend tokens on it. If it passes, you have a clean bill of health on the obvious vectors before your paid scanner runs.

---

## Features

1. **Offline-first.** No network calls required. Runs entirely on your machine.
2. Archive-safe extraction and static analysis.
3. Binary artifact classification and flagging (executables, libraries, bytecode, blobs).
4. Malware and instruction-abuse pattern detection (121 static rules + 17 multilang rules, 15 chain rules).
5. Instruction hardening pipeline (Unicode normalization, zero-width stripping, bounded base64 decode, action-chain checks).
6. IOC extraction with local intel matching (updated regularly).
7. Dependency vulnerability checks (23 Python + 4 npm packages via OSV.dev).
8. Social engineering and credential-harvest instruction detection (SE-001, SE-SEM-001).
9. Policy profiles (`strict`, `balanced`, `permissive`, `ci`, `enterprise`, `observe`) + custom policies.
10. Pretty terminal output + JSON / SARIF / JUnit / compact reports.
11. Auto-refresh managed intel feeds (default checks every scan, 1-hour max age).
12. Versioned YAML rulepack for flexible detection updates.
13. Adversarial regression corpus with expected verdicts.
14. Default-on local semantic prompt-injection classifier (NLTK/classical features, no external API).
15. Optional offline ML detection (`--ml-detect`) using a fine-tuned Qwen2.5-1.5B model (GGUF Q4_K_M) — no API key, no cloud.

---

## Distribution Status

- PyPI: `pip install skillscan-security`
- Docker: `docker pull kurtpayne/skillscan-security`
- Pre-commit hook: `skillscan-security>=0.8.0`

Release process: `docs/RELEASE_CHECKLIST.md`.

SBOMs: Python CycloneDX (`sbom-python.cdx.json`) and Docker SPDX (`sbom-docker.spdx.json`) are included in release artifacts.

Docker default behavior: the image includes ClamAV and enables it by default (`SKILLSCAN_CLAMAV=true`). Override with `--no-clamav`.

---

## Install

### Option A: convenience installer

```bash
curl -fsSL https://raw.githubusercontent.com/kurtpayne/skillscan/main/scripts/install.sh | bash
```

### Option B: pip

```bash
pip install skillscan-security
```

**Base install is ~25 MB.** No torch, no transformers, no heavy ML stack. The `--ml-detect` flag requires an optional extra:

```bash
# llama-cpp-python backend (~935 MB model download) — recommended
pip install 'skillscan-security[ml]'
```

### Option C: local/dev install

```bash
python3 -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
```

---

## Quick Start

```bash
skillscan scan ./examples/suspicious_skill
```

Scan directly from URL (including GitHub blob URLs):

```bash
skillscan scan "https://github.com/blader/humanizer/blob/main/SKILL.md?plain=1"
```

Save reports:

```bash
# JSON
skillscan scan ./target --format json --out report.json --fail-on never
# SARIF (GitHub code scanning)
skillscan scan ./target --format sarif --out skillscan.sarif --fail-on never
# JUnit XML (CI test report ingestion)
skillscan scan ./target --format junit --out skillscan-junit.xml --fail-on never
# Compact (terse CI logs)
skillscan scan ./target --format compact --fail-on never
```

Render a saved report:

```bash
skillscan explain ./report.json
```

Optional offline ML detection (requires `[ml]` extra):

```bash
skillscan scan ./target --ml-detect
```

The ML detector uses a fine-tuned Qwen2.5-1.5B model (GGUF Q4_K_M, ~935 MB). It runs entirely on your machine — no API calls, no tokens, no cloud. It is the right tool for subtle semantic attacks that the static rules don't catch. For nuanced intent analysis that requires reasoning about context, see the [integration bridges](#integration-bridges) below.

---

## Highlighted Examples

### 1. Scan a suspicious skill

```console
$ skillscan scan examples/suspicious_skill --fail-on never
```

### 2. Scan a benign skill

```console
$ skillscan scan examples/benign_skill --fail-on never
```

Every static rule includes an inline `test_input` field in `default.yaml` that
documents which text triggers the rule. Run all rule tests with:

```console
SKILLSCAN_NO_USER_RULES=1 pytest tests/test_rule_inputs.py -q
```

---

## Command Summary

See full command docs: `docs/COMMANDS.md`.

---

## Policies

Built-in profiles:

1. `strict` (default) — maximum coverage, blocks on score >= 70
2. `balanced` — blocks on score >= 50, HIGH+ severity
3. `permissive` — trusted environments, blocks on score >= 90, CRITICAL only
4. `ci` — PR gates, blocks on CRITICAL + HIGH only
5. `enterprise` — formal security gate, ML required
6. `observe` — day-one adoption, exit 0 always

Use a built-in profile:

```bash
skillscan scan ./target --profile balanced
```

Use a custom policy file:

```bash
skillscan scan ./target --policy ./examples/policies/strict_custom.yaml
```

---

## Intel Management

Add a local IOC source:

```bash
skillscan intel add --url https://example.com/feeds/iocs.json --type ioc --name team-iocs
```

View sources:

```bash
skillscan intel status
skillscan intel list
```

Managed intel auto-refresh runs by default on `scan`. You can tune or disable it:

```bash
skillscan scan ./target --intel-max-age-minutes 60
skillscan scan ./target --no-auto-intel
```

---

## Integration Bridges

SkillScan is designed to be the **free pre-filter** in a layered scanning pipeline. It handles deterministic checks locally so you don't spend tokens on the obvious cases. For nuanced intent analysis, pair it with an online scanner.

### Use SkillScan as a pre-filter for Invariant

[Invariant Analyzer](https://github.com/invariantlabs-ai/invariant) provides deep semantic analysis of agent traces and skill files. Run SkillScan first to eliminate clear-cut cases:

```bash
# Only send to Invariant if SkillScan doesn't block
skillscan scan ./skill --format json --out pre-filter.json --fail-on never
if [ "$(jq -r '.verdict' pre-filter.json)" != "block" ]; then
invariant analyze ./skill
fi
```

Or in CI:

```yaml
- name: SkillScan pre-filter
run: skillscan scan ./skills --format sarif --out skillscan.sarif
continue-on-error: true

- name: Upload SkillScan results
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: skillscan.sarif

- name: Deep scan (only if SkillScan passes)
if: steps.skillscan.outcome == 'success'
run: invariant analyze ./skills
```

### Use SkillScan as a pre-filter for Lakera Guard

[Lakera Guard](https://www.lakera.ai/) provides real-time prompt injection detection via API. SkillScan catches the static patterns for free before you hit the API:

```python
import subprocess, json, requests

result = subprocess.run(
["skillscan", "scan", skill_path, "--format", "json", "--fail-on", "never"],
capture_output=True, text=True
)
report = json.loads(result.stdout)

if report["verdict"] == "block":
# SkillScan caught it — no API call needed
raise ValueError(f"Skill blocked by SkillScan: {report['top_findings']}")

# SkillScan passed — send to Lakera for semantic analysis
response = requests.post(
"https://api.lakera.ai/v1/prompt_injection",
headers={"Authorization": f"Bearer {LAKERA_API_KEY}"},
json={"input": skill_content}
)
```

### Pre-commit hook

```yaml
# .pre-commit-config.yaml
repos:
- repo: https://github.com/kurtpayne/skillscan-security
rev: v0.8.0
hooks:
- id: skillscan
args: [--fail-on, warn]
```

---

## Example Fixtures

1. Benign sample: `examples/benign_skill`
2. Suspicious sample: `examples/suspicious_skill`
3. OpenAI-style sample: `examples/openai_style_tool`
4. Claude-style sample: `examples/claude_style_skill`
5. Rule test inputs: inline `test_input` fields in `src/skillscan/data/rules/default.yaml`
7. OpenClaw-compromised-style sample: `tests/fixtures/malicious/openclaw_compromised_like`

---

## Cross-Platform Skill Bundles

Starter bundles for OpenClaw/ClawHub, Claude-style skills, and OpenAI Actions are in:

- `integrations/openclaw/`
- `integrations/claude/`
- `integrations/openai/`

Each `integrations/` directory has a README describing the starter bundle and how to adopt it.

---

## Testing

```bash
./scripts/run_tests.sh test
./scripts/run_tests.sh lint
./scripts/run_tests.sh type
./scripts/run_tests.sh check
```

Or via Makefile:

```bash
make check
```

---

## CI/CD Integration

SkillScan provides a reusable GitHub Actions workflow for scanning skill artifacts in CI pipelines with native SARIF upload to the GitHub Security tab.

```yaml
jobs:
skillscan:
uses: kurtpayne/skillscan-security/.github/workflows/skillscan-scan.yml@main
with:
scan-path: ./skills
```

See `docs/GITHUB_ACTIONS.md` for full documentation and examples.

---

## Extend SkillScan

Every detection layer except the ML classifier is configurable. The ML model is a frozen GGUF artifact — a tamper-resistant trust anchor that cannot be overridden by local configuration.

### Add custom rules

Write a YAML file using the same schema as the built-in rules and place it in `~/.skillscan/rules/`. Custom rules are additive — they run alongside the built-in set.

```yaml
# my-rules.yaml
static_rules:
- id: CUSTOM-001
name: Internal API Key Exposure
description: Detects hardcoded internal API key patterns
severity: critical
pattern: "sk-internal-[a-zA-Z0-9]{32}"
tags: [secrets, custom]

- id: CUSTOM-002
name: Internal Domain Reference
description: Flags references to internal infrastructure domains
severity: high
pattern: "\\.internal\\.example\\.com"
tags: [infra, custom]
```

```bash
# Test a rule against a fixture file (positional args)
skillscan rule test my-rules.yaml test.md
```

See [`docs/custom-rules-format.md`](docs/custom-rules-format.md) for the full schema including chain rules, multilang rules, and AST flow rules.

### Suppress false positives

Inline — suppress a specific rule on a specific line:

```yaml
# skillscan-suppress: MAL-001
export ADMIN_TOKEN="placeholder-replaced-at-runtime"
```

Project-level — suppress rules across a whole project via `.skillscan-suppressions.yaml` at the repo root:

```yaml
# .skillscan-suppressions.yaml
suppressions:
- id: MAL-001
evidence_path: skills/legacy-tool.md
reason: Known false positive — token is a placeholder
expires: "2026-12-31"
- id: EXF-003
evidence_path: skills/analytics.md
reason: Intentional telemetry, reviewed 2026-03-01
expires: "2026-12-31"
```

See [`docs/suppression-format.md`](docs/suppression-format.md) for the full schema.

### Policy profiles

Six built-in profiles cover the most common deployment contexts:

| Profile | Fails on | Use case |
|---|---|---|
| `strict` (default) | score >= 70 | Local dev, high-assurance pipelines |
| `balanced` | score >= 50, HIGH+ | Developer and team use |
| `permissive` | score >= 90, CRITICAL only | Trusted internal registries |
| `ci` | CRITICAL + HIGH only | PR gates, CI/CD pipelines |
| `enterprise` | score >= 70, ML required | Formal security gate |
| `observe` | never (exit 0 always) | Day-one adoption, reporting only |

```bash
skillscan scan ./skills/ --profile ci
```

### Contribute a rule

Community-submitted patterns are reviewed and, if accepted, added to the static rule layer — not the model. Submit via the [corpus-submission issue template](https://github.com/kurtpayne/skillscan-security/issues/new?template=corpus-submission.md).

For the full customization reference, see [skillscan.sh/docs#customization](https://skillscan.sh/docs#customization).

---

## Uninstall

```bash
skillscan uninstall
# Keep local data (intel/reports/config):
skillscan uninstall --keep-data
```

Shell script uninstall: `scripts/uninstall.sh`.

---

## Documentation

- Command reference: `docs/COMMANDS.md`
- Detection model: `docs/DETECTION_MODEL.md`
- Scan overview: `docs/SCAN_OVERVIEW.md`
- Architecture: `docs/ARCHITECTURE.md`
- Policy guide: `docs/POLICY.md`
- Intel guide: `docs/INTEL.md`
- Testing guide: `docs/TESTING.md`
- Rules and scoring: `docs/RULES.md`
- Rule catalogue: `docs/EXAMPLES.md`
- GitHub Actions integration: `docs/GITHUB_ACTIONS.md`
- Custom rule format: `docs/custom-rules-format.md`
- Custom policy format: `docs/custom-policy-format.md`
- Custom intel format: `docs/custom-intel-format.md`
- Suppression format: `docs/suppression-format.md`
- ML model metrics: `docs/MODEL_METRICS.md`
- Release checklist: `docs/RELEASE_CHECKLIST.md`
- Pattern update changelog: `PATTERN_UPDATES.md`

---

## Related

- **[skillscan-lint](https://github.com/kurtpayne/skillscan-lint)** — Quality linter for AI agent skills: readability, clarity, graph integrity
- **[Invariant Analyzer](https://github.com/invariantlabs-ai/invariant)** — Deep semantic analysis of agent traces; use SkillScan as a free pre-filter
- **[Lakera Guard](https://www.lakera.ai/)** — Real-time prompt injection detection API; use SkillScan to eliminate static cases before hitting the API
- **[skills.sh](https://skills.sh)** — Community registry of AI agent skills
- **[ClawHub](https://clawhub.ai)** — MCP skill marketplace
- **[Docker Hub](https://hub.docker.com/r/kurtpayne/skillscan-security)** — `docker pull kurtpayne/skillscan-security`
- **[PyPI](https://pypi.org/project/skillscan-security/)** — `pip install skillscan-security`

---

## License

Licensed under Apache-2.0. See `LICENSE`.

## Security Note

SkillScan performs static analysis by default and does not execute scanned artifacts. For untrusted inputs, run in a trusted isolated environment.

For URL scans, unreadable linked sources are reported as low-severity `SRC-READ-ERR` findings. They are flagged for review but are not treated as malicious by default.

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/kurtpayne/skillscan-security

Awesome Lists containing this project

README