https://github.com/runtime-org/runtime
Deterministic skills-based browser agent
https://github.com/runtime-org/runtime
browser-ai llm puppeteer
Last synced: 28 days ago
JSON representation
Deterministic skills-based browser agent
- Host: GitHub
- URL: https://github.com/runtime-org/runtime
- Owner: runtime-org
- License: mit
- Created: 2025-07-12T13:18:17.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-08-30T23:26:58.000Z (2 months ago)
- Last Synced: 2025-08-31T01:09:28.983Z (2 months ago)
- Topics: browser-ai, llm, puppeteer
- Language: TypeScript
- Homepage: https://anytime.sh
- Size: 2.37 MB
- Stars: 41
- Watchers: 0
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-tauri - Runtime - AI taskmate for web & office tools. (Applications / Productivity)
README
Skills-based browser agent
Runtime allows you to interact with a wide range of Chromium-based browsers, including Chrome, Edge and soon (Firefox, Brave, Dia, Comet, etc). Our goal is to provide a fast, reliable, and affordable way to work with your office tools and the web. No need to install a separate AI browser to access AI capabilities.
No vendor lock-in or API dependencies.
Demo: showing Runtime is 3x faster than Comet. To learn more about our technic, please see our [skills method](/SKILLS.md).
---
What is our skills method?
Previous AI browser agents (such as Comet, Dia, Genspark, browser-use, or browserbase) rely heavily on the DOM at execution time and manipulate it using heuristic methods. This approach makes processing requests slow.
We took a different approach: instead of manipulating the DOM at runtime/render-time, we use the tag attributes present in the DOM after the initial rendering by Blink. These attributes serve as gateways to locate elements such as `button`, `a`, `div`, `span`, and others.
We call these "skills." Every website has a set of predefined actions that a human can perform. For example, on amazon.com, the basic functions a user might perform include:
- searching for products,
- reading product information (such as title, price, discount, delivery time, rating, comments, etc.),
- adding items to the cart,
- viewing the cart,
- tracking deliveries,
- and more.
By leveraging predefined skills instead of interacting with the live DOM, we achieve significant speed improvements. This approach also removes the need for the LLM to reason about which button to click or which element to hover over.
Important: While we recognize that this method is not infinitely scalable, we have developed a solution to address scalability challenges and will be sharing more details soon. Follow us for updates!
A single skills of amazon.* are:
```json
{
"name": "search_products",
"description": "Search for a product on Amazon. this skill will take the user query as input, and perform the search on amazon.com and it will return the list of results of products.",
"input": { "text": "string" },
"output": "results",
"steps": [
{ "action": "navigate_to_url", "url": "https://www.amazon.com" },
{ "action": "wait_for_selector", "selector": "#twotabsearchtextbox" },
{ "action": "click", "selector": "#twotabsearchtextbox" },
{ "action": "type", "selector": "#twotabsearchtextbox", "input_key": "text" },
{ "action": "press_enter" },
{ "action": "wait_for_selector", "selector": "div[data-component-type='s-search-result'][data-asin]:not([data-asin=''])" },
{ "action": "scroll_down", "times": 3 },
{
"action": "extract_list",
"selector": "div[data-component-type='s-search-result'][data-asin]:not([data-asin=''])",
"schema": {
"asin": "@data-asin",
"title": "[data-cy='title-recipe'] a h2::text",
"price": "[data-cy='price-recipe'] .a-row [aria-describedby='price-link'] .a-price .a-offscreen::text",
"link": "[data-cy='title-recipe'] a::href"
},
"output_key": "results"
}
]
}
```
----
### Key Features
- Reliable, hand-crafted action recipes for popular sites (Amazon, Gmail, Google, Notion, and more coming) ensure consistent results
- Actions are performed in your own logged-in browser session—no need for separate profiles or repeated logins
- Research mode: gather, rank, and cite sources efficiently in "tabs" tab.
- Automate multi-step tasks like triaging email, reading and replying to mail.
- Instantly send results to Apple Notes.
---
## What can Runtime do?
#### Search email, reply to an email?
#### Search for notion, and summarize the content
#### Do research about vegan restaurant in Berlin
---
## Quick Start
```bash
# 1. Prerequisites
brew install node rust # or from nodejs.org & rust-lang.org
# 2. Clone
git clone git@github.com:runtime-org/runtime.git
cd runtime
# 3. Configure (Gemini key)
echo 'VITE_GEMINI_API_KEY=' > .env
# 4. Install & run
npm install
npm run tauri dev
```
---
### Contributing
If you want to contribute, you are welcome! Please open an issue or submit a PR if you want us to generate a skills for your website or web app.
### Citation
If you use this project in your research, please cite it as follows:
```bibtex
@software{runtime2025,
author = {Tito and Mile and Sam},
title = {Runtime},
year = {2025},
publisher = {GitHub},
url = {[https://github.com/runtime-org/runtime](https://github.com/runtime-org/runtime)}
}
```