https://github.com/jlevy/superanalyze
https://github.com/jlevy/superanalyze
Last synced: 9 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/jlevy/superanalyze
- Owner: jlevy
- License: agpl-3.0
- Created: 2025-08-10T20:49:25.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-08-11T00:38:19.000Z (9 months ago)
- Last Synced: 2025-08-11T01:14:28.108Z (9 months ago)
- Language: Python
- Size: 227 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# superanalyze
Analyze claims, rigor, and evidence in a doc.
## Motivation
We are now inundated with more content than ever in text form.
And it is even harder to tell what to trust.
Say you have a document that makes an argument or claim.
It might be a deep research report.
Or it might be a transcript of a talk, or anything else where someone makes claims or
takes a stance on a topic.
Superanalyze provides deep analysis of long documents, making it possible to break down
AI or human-written reports or transcripts systematically, to evaluate claims, rigor,
and evidence.
## Approach
Superanalyze is an experiment at building a this kind of tool.
It uses a sequence of LLM prompts (and some embeddings) to analyze:
- What are the key claims
- What parts of the document relate to those claims (this is done by paragraph)
- Researching related content using a thinking LLM with web search, and inserting
annotations on each
- Evaluating the rigor of the claims and if the evidence supports it
- Visualizing these in the document in a way that lets you drill down and see the
assessments and whether you believe them
The design is based on three key beliefs:
1. **Automatic, easier deep analysis:** While establishing absolute truth of statements
is very hard, there *are *many statements are arguments that are easy to evaluate for
rigor or evidence or counter-arguments, and this is the kind of thing that should be
done automatically and easily within tools.
2. **Using a consistent structure and clear rubrics:** Just asking an AI tool to
evaluate the strength or rigor of arguments can help, but it’s not consistent and
it’s different for every person and every prompt.
We need systematic analyses that break down the process of analyzing a document into
key steps that can be verified and each run fairly reliably.
Rubrics (like 1-to-4 scores on clarity or depth, for example) can help with this.
3. **Exposing full details to the reader:** The only way to share this is to give the
reader both an overview of the assessment and an ability to drill down into the
document and its sources.
## How It Works
This is built on [kash](https://www.github.com/jlevy/kash) and its
[kash-docs](https://www.github.com/jlevy/kash-docs) kit of tools for docs and analysis.
It’s a good use case for this framework as it has lots of actions that do individual
portions like summarizing, chunking, handling footnotes, etc, but none had been stitched
together properly for a deeper analysis of document claims.
This has let us convert PDFs and transcribe YouTube videos and then analyze them as
clean Markdown with annotations.
## Status
New!
We’re still experimenting with UI on visualizing them, but it seems promising.
Factuality is a tough thing to measure, but we now have partial support for crawling all
linked resources and analyzing strength of evidence.
This part isn’t fully done yet but will be soon.
## Usage
### Key Setup
See the `env.template` to set up keys (default model is OpenAI).
### Basic Usage
It should work on most file types, including text, HTML, and PDF.
```bash
uv run superanalyze somefile.pdf
```
## MCP Server
Run as an MCP server for integration with other tools.
Once it is in your path:
```bash
# Run as stdio MCP server
superanalyze --mcp
# Run as SSE MCP server at 127.0.0.1:4440
superanalyze --sse
# View MCP server logs
superanalyze --logs
```
Note: Both `--sse` and `--logs` automatically enable MCP mode, so you don’t need to
specify `--mcp` explicitly.
### Claude Desktop Configuration
For Claude Desktop, a config like this should work (adjusted to use your appropriate
home folder):
```json
{
"mcpServers": {
"superanalyze": {
"command": "/Users/levy/.local/bin/superanalyze",
"args": ["--mcp"]
}
}
}
```
## Project Docs
For how to install uv and Python, see [installation.md](installation.md).
For development workflows, see [development.md](development.md).
For instructions on publishing to PyPI, see [publishing.md](publishing.md).
* * *
*This project was built from
[simple-modern-uv](https://github.com/jlevy/simple-modern-uv).*