https://github.com/puzzlet-ai/agentmark
Markdown for the AI era
https://github.com/puzzlet-ai/agentmark
agents llms observability opentelemetry prompt prompt-engineering prompt-management
Last synced: 10 days ago
JSON representation
Markdown for the AI era
- Host: GitHub
- URL: https://github.com/puzzlet-ai/agentmark
- Owner: puzzlet-ai
- License: mit
- Created: 2024-10-16T15:06:49.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2025-03-29T01:11:46.000Z (18 days ago)
- Last Synced: 2025-03-29T01:32:57.663Z (18 days ago)
- Topics: agents, llms, observability, opentelemetry, prompt, prompt-engineering, prompt-management
- Language: TypeScript
- Homepage: https://docs.puzzlet.ai/agentmark/
- Size: 15.7 MB
- Stars: 139
- Watchers: 3
- Forks: 5
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.md
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-llmops - AgentMark - Safe Markdown-based Agents |  | (LLMOps / Observability)
README
AgentMark
Markdown for the AI Era
---
## Overview
Develop type-safe prompts and agents using readable Markdown and JSX.
### Features
AgentMark supports:
1. Markdown: 📝
1. Type Safety: 🛡️
1. Unified model config: 🔗
1. JSX components, props, & plugins: 🧩
1. Loops, Conditionals, and Filter Functions: ♻️
1. Custom SDK Adapters: 🛠️
1. JSON Output: 📦
1. Tools & Agents: 🕵️
1. Text, Object, and Image output. Audio/Video coming soon.
Read our [docs](https://docs.puzzlet.ai/agentmark/) to learn more.
## Getting Started
Below is a basic example to help you get started with AgentMark:
`example.prompt.mdx`
```mdx
---
name: basic-prompt
model:
name: gpt-4o-mini
test_settings:
props:
num: 3
---
You are a math expert
What's 2 + {props.num}?
```
## Models
By default, AgentMark doesn't support any models or calling any LLM providers. Instead, we format the input of your prompt through an adapter to match the input of the SDK you're using.
### Supported Adapters
| Adapter | Supported | Supports Type-Safety |
|-----------|-----------|-----------|
| Default | ✅ | ✅ |
| Custom | ✅ | ✅ |
| Vercel (Recommended) | ✅ | ✅ |
| Mastra | ⚠️ Coming Soon | ⚠️ |
| LangChain | ⚠️ Coming Soon | ❌ |
| OpenAI Compatible | ⚠️ Coming Soon | ❌ |
Want to add support for another adapter? Open an [issue](https://github.com/puzzlet-ai/agentmark/issues).
### Supported Prompt Types
| Prompt Type | Supported |
|-----------|-----------|
| Object | ✅ |
| Text | ✅ |
| Image | ✅ |
| Audio | ⚠️ Coming Soon |
| Video | ⚠️ Coming Soon |
### Custom Adapters
Refer to our [docs](https://docs.puzzlet.ai/agentmark/) to learn how to add custom adapter support.
## Language Support
We plan on providing support for AgentMark across a variety of languages.
| Language | Support Status |
|----------|---------------|
| TypeScript | ✅ Supported |
| JavaScript | ✅ Supported |
| Python | ⚠️ Coming Soon |
| Others | Need something else? [Open an issue](https://github.com/puzzlet-ai/agentmark/issues) |
## Running AgentMark
You can run AgentMark using any of the following methods:
### 1. VSCode Extension
Run .prompt.mdx files directly within your VSCode editor. Note: You can test props by using `test_settings` in your prompts.
[Download the VSCode Extension](https://marketplace.visualstudio.com/items?itemName=puzzlet.agentmark)
### 2. FileLoader
Run AgentMark files from your file system. Below is a sample implementation using the Vercel adapter to generate an object:
```ts
import { VercelAdapter, VercelModelRegistry, FileLoader, createAgentMark } from "../src";
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
// Import/Register any Vercel compatible models
const modelRegistry = new VercelModelRegistry();
modelRegistry.registerModel(['gpt-4o', 'gpt-4o-mini'], (name: string, options: any) => {
return openai(name);
});
// Specify a file loader + vercel adapter
const fileLoader = new FileLoader('./puzzlet/templates');
const adapter = new VercelAdapter(modelRegistry);
const agentMark = createAgentMark({
loader: fileLoader,
adapter,
});
const prompt = await agentMark.loadObjectPrompt('test/math2.prompt.mdx');
const props = {
num1: 2,
num2: 3,
};
// Adapt to the Vercel SDK
const vercelInput = await prompt.format(props);
// Call the Vercel SDK directly
const result2 = await generateObject(vercelInput);
console.log(result2.object);
```
### 3. Puzzlet Integration
Puzzlet is a platform for managing, versioning, and monitoring your LLM prompts in production, with built-in observability, evaluations, prompt management, alerts, and more.
```ts
// Specify the puzzlet loader instead of file loader
const puzzletLoader = new PuzzletLoader({
apiKey: process.env.PUZZLET_API_KEY!,
appId: process.env.PUZZLET_APP_ID!,
baseUrl: process.env.PUZZLET_BASE_URL!,
});
const agentMark = createAgentMark({
loader: puzzletLoader,
// rest stays the same...
});
```
## Type Safety
AgentMark & Puzzlet supports automatic type generation from your prompt schemas. Define input (`input_schema`) and output (`schema`) types in your prompt files:
```mdx
---
name: math-addition
model:
name: gpt-4o
schema:
type: "object"
properties:
sum:
type: "number"
description: "The sum of the two numbers"
required: ["sum"]
input_schema:
type: "object"
properties:
num1:
type: "number"
description: "First number to add"
num2:
type: "number"
description: "Second number to add"
required: ["num1", "num2"]
---
You are a helpful math assistant that performs addition.
```
Then generate types using the CLI:
```bash
# From local files
npx @puzzlet/cli generate-types --root-dir ./prompts > puzzlet.types.ts
# From local Puzzlet server
npx @puzzlet/cli generate-types --local 9002 > puzzlet.types.ts
```
Use the generated types with FileLoader:
```ts
import { VercelAdapter, VercelModelRegistry, FileLoader, createAgentMark } from "../src";
import { generateObject } from 'ai';
import { openai } from '@ai-sdk/openai';
import PuzzletTypes from './puzzlet.types';
const modelRegistry = new VercelModelRegistry();
modelRegistry.registerModel(['gpt-4o', 'gpt-4o-mini'], (name: string, options: any) => {
return openai(name);
});
// Add the puzzlet types
const fileLoader = new FileLoader('./puzzlet/templates');
const adapter = new VercelAdapter(modelRegistry);
const agentMark = createAgentMark({
loader: fileLoader,
adapter,
});
const prompt = await agentMark.loadObjectPrompt('test/math2.prompt.mdx');
const props = {
num1: 2,
num2: 3,
};
const vercelInput = await prompt.format(props);
const result2 = await generateObject(vercelInput);
console.log(result2.object.sum);
```
Or with Puzzlet:
```ts
// ...
const puzzletLoader = new PuzzletLoader({
apiKey: process.env.PUZZLET_API_KEY!,
appId: process.env.PUZZLET_APP_ID!,
baseUrl: process.env.PUZZLET_BASE_URL!,
});
// Rest stays the same...
```
AgentMark is also type-safe within markdown files. Read more [here](https://puzzlet-ai.github.io/templatedx/docs/type-safety).
## Contributing
We welcome contributions! Please check out our [contribution guidelines](https://github.com/puzzlet-ai/agentmark/blob/main/CONTRIBUTING.md) for more information.
## Community
Join our community to collaborate, ask questions, and stay updated:
- [Discord](https://discord.gg/P2NeMDtXar)
- [Issues](https://github.com/puzzlet-ai/agentmark/issues)
- [Discussions](https://github.com/puzzlet-ai/agentmark/discussions)
## License
This project is licensed under the [MIT License](https://github.com/puzzlet-ai/agentmark/blob/main/LICENSE).