{"id":48614126,"url":"https://github.com/tzafon/lightcone","last_synced_at":"2026-05-21T01:13:33.322Z","repository":{"id":343520578,"uuid":"1152777590","full_name":"tzafon/lightcone","owner":"tzafon","description":"Lightcone: SDK for computer use agents","archived":false,"fork":false,"pushed_at":"2026-05-18T21:53:28.000Z","size":17353,"stargazers_count":15,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-05-18T23:59:27.091Z","etag":null,"topics":["agent","automation","computer-use","computer-vision","desktop-automation","gui-automation","llm","vision-language-model"],"latest_commit_sha":null,"homepage":"https://lightcone.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tzafon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-08T12:22:42.000Z","updated_at":"2026-05-18T05:31:44.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tzafon/lightcone","commit_stats":null,"previous_names":["tzafon/lightcone"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tzafon/lightcone","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzafon%2Flightcone","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzafon%2Flightcone/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzafon%2Flightcone/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzafon%2Flightcone/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tzafon","download_url":"https://codeload.github.com/tzafon/lightcone/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tzafon%2Flightcone/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33283729,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-20T15:12:43.734Z","status":"ssl_error","status_checked_at":"2026-05-20T15:12:42.300Z","response_time":356,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","automation","computer-use","computer-vision","desktop-automation","gui-automation","llm","vision-language-model"],"created_at":"2026-04-09T01:04:16.892Z","updated_at":"2026-05-21T01:13:33.310Z","avatar_url":"https://github.com/tzafon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# Lightcone\n\n### The API for Northstar, a vision-language model by Tzafon\n\n\u003cimg src=\"northstar.svg\" alt=\"Northstar\" width=\"160\"\u003e\n\n**Northstar CUA Fast** — trained with GUI reinforcement learning\n\n[Docs](https://docs.lightcone.ai) | [API Reference](https://docs.lightcone.ai/api) | [Model](https://huggingface.co/Tzafon/Northstar-CUA-Fast) | [Pricing](https://docs.tzafon.ai/pricing) | [X (Twitter)](https://x.com/tzafon_company)\n\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)\n[![PyPI - tzafon](https://img.shields.io/pypi/v/tzafon?label=tzafon\u0026color=blue)](https://pypi.org/project/tzafon/)\n[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://python.org)\n[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Northstar--CUA--Fast-yellow)](https://huggingface.co/Tzafon/Northstar-CUA-Fast)\n\n\u003c/div\u003e\n\n---\n\nNorthstar sees screens and acts on them. Give it a screenshot, it decides where to click, what to type, when to scroll. Give it a task in plain language, it operates a computer from start to finish — opening apps, navigating between pages, filling forms, reading results.\n\nIt recovers from mistakes, generalizes across desktop environments, and outperforms open-source models at twice its size. Built for computer-use loops where every step is a model call.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"assets/3.gif\" alt=\"Northstar searching for a product on a Linux desktop\" width=\"720\"\u003e\n\u003c/div\u003e\n\n| | |\n|---|---|\n| **Context** | 64K tokens |\n| **Training** | GUI reinforcement learning |\n| **Input** | Text + screenshot |\n| **Output** | GUI actions — `click`, `type`, `scroll`, `key`, `drag`, ... |\n| **Coordinates** | 0–999 normalized — denormalize to pixels in your code |\n| **Pricing** | $1/M input · $5/M output ([details](https://docs.tzafon.ai/pricing)) |\n\n---\n\n## Quickstart\n\n### Install\n\n```bash\npip install tzafon\n```\n\n### Give Northstar a task\n\n```python\nimport os\nfrom tzafon import Lightcone\n\nclient = Lightcone(api_key=os.environ[\"TZAFON_API_KEY\"])\n\nfor event in client.agent.tasks.start_stream(\n    instruction=\"Go to wikipedia.org, search for 'Alan Turing', and tell me the first sentence\",\n    kind=\"desktop\",\n):\n    print(event)\n```\n\nNorthstar spins up a computer, opens a browser, searches Wikipedia, reads the article, and reports back. You just described what you wanted.\n\n---\n\n## CUA Loop\n\nFor full control, build the loop yourself. Northstar looks at a screenshot, decides the next action, you execute it, feed back the result:\n\n```python\nimport os\nfrom tzafon import Lightcone\n\nclient = Lightcone(api_key=os.environ[\"TZAFON_API_KEY\"])\n\nTOOL = {\"type\": \"computer_use\", \"display_width\": 1280, \"display_height\": 720, \"environment\": \"desktop\"}\nTASK = \"Open the terminal, run 'uname -a', then run 'df -h' and report the results\"\n\nwith client.computer.create(kind=\"desktop\") as computer:\n    screenshot_url = computer.get_screenshot_url(computer.screenshot())\n\n    response = client.responses.create(\n        model=\"tzafon.northstar-cua-fast\",\n        tools=[TOOL],\n        input=[{\"role\": \"user\", \"content\": [\n            {\"type\": \"input_text\", \"text\": TASK},\n            {\"type\": \"input_image\", \"image_url\": screenshot_url, \"detail\": \"auto\"},\n        ]}],\n    )\n\n    while True:\n        computer_call = next(\n            (o for o in (response.output or []) if o.type == \"computer_call\"), None\n        )\n        if not computer_call:\n            break\n\n        action = computer_call.action\n        if action.type == \"click\":\n            computer.click(action.x, action.y)\n        elif action.type == \"type\":\n            computer.type(action.text)\n        elif action.type in (\"key\", \"keypress\"):\n            computer.hotkey(action.keys)\n        elif action.type == \"scroll\":\n            computer.scroll(dx=action.scroll_x or 0, dy=action.scroll_y or 0)\n        elif action.type == \"navigate\":\n            computer.navigate(action.url)\n        elif action.type in (\"terminate\", \"done\", \"answer\"):\n            break\n        # ... see examples/ for full action handling\n\n        computer.wait(1)\n        screenshot_url = computer.get_screenshot_url(computer.screenshot())\n        response = client.responses.create(\n            model=\"tzafon.northstar-cua-fast\",\n            previous_response_id=response.id,\n            tools=[TOOL],\n            input=[{\n                \"type\": \"computer_call_output\",\n                \"call_id\": computer_call.call_id,\n                \"output\": {\"type\": \"input_image\", \"image_url\": screenshot_url, \"detail\": \"auto\"},\n            }],\n        )\n```\n\n---\n\n## Try it\n\nRun Northstar against a live enterprise app and see every step:\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd align=\"center\"\u003e\u003cimg src=\"assets/1.gif\" alt=\"Northstar logging in and adding an employee in OrangeHRM\" width=\"420\"\u003e\u003cbr\u003e\u003csub\u003eOrangeHRM — login \u0026 add employee\u003c/sub\u003e\u003c/td\u003e\n\u003ctd align=\"center\"\u003e\u003cimg src=\"assets/2.gif\" alt=\"Northstar creating a contact record in SuiteCRM\" width=\"420\"\u003e\u003cbr\u003e\u003csub\u003eSuiteCRM — create contact record\u003c/sub\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n```bash\nexport TZAFON_API_KEY=\"your-api-key\"\nuv run python -m examples.harness.evaluate\n```\n\nWith annotated screenshots saved to a directory:\n\n```bash\nuv run --with Pillow --with httpx python -m examples.harness.evaluate --screenshots steps/\n```\n\nReliability check (3 runs):\n\n```bash\nuv run python -m examples.harness.evaluate --runs 3\n```\n\nCustom target:\n\n```bash\nuv run python -m examples.harness.evaluate \\\n  --url \"https://any-web-app.com/login\" \\\n  --instruction \"Log in with user/pass, then do something\"\n```\n\n---\n\n## Examples\n\n| Example | Description |\n|---|---|\n| [`desktop.py`](examples/desktop.py) | Northstar operates a desktop — opens terminal, runs commands, reads output |\n| [`simple.py`](examples/simple.py) | Minimal browser CUA loop |\n| [`shell.py`](examples/shell.py) | Mixes Northstar with direct shell commands |\n| [`competitor_research.py`](examples/competitor_research.py) | Two-phase: Northstar explores, then extracts structured data |\n| [`persistent_session.py`](examples/persistent_session.py) | Persistent state for authenticated workflows |\n| [`streaming.py`](examples/streaming.py) | FastAPI SSE endpoint wrapping the CUA loop |\n| [`interactive.py`](examples/interactive.py) | Human-in-the-loop — pauses for CAPTCHAs, 2FA, ambiguity |\n| [`multi_tab.py`](examples/multi_tab.py) | Multi-tab comparison across sites |\n| [`visualize.py`](examples/visualize.py) | Save annotated screenshots showing every decision Northstar makes |\n| [`monitor.py`](examples/monitor.py) | Screenshot-only observer for monitoring |\n\n```bash\nexport TZAFON_API_KEY=\"your-api-key\"\npython examples/desktop.py\n```\n\n---\n\n## Supported Actions\n\n`click` · `double_click` · `triple_click` · `right_click` · `drag` · `type` · `key` · `scroll` · `hscroll` · `navigate` (browser only) · `wait` · `terminate`\n\nVia the **Responses API** (`/v1/responses`), coordinates are scaled to viewport pixels and responses are structured — no parsing required. Multi-turn conversations are managed server-side via `previous_response_id`.\n\n---\n\n## SDKs\n\n| SDK | Install | Source |\n|---|---|---|\n| Python | `pip install tzafon` | [`sdks/python`](sdks/python) |\n| TypeScript | `npm install @tzafon/lightcone` | [`sdks/typescript`](sdks/typescript) |\n\n---\n\n## OSWorld Benchmark (pass@1, 50 steps)\n\nEvaluated on [OSWorld](https://os-world.github.io/) — 369 real-world desktop tasks.\n\n| Domain | UI-TARS 2 | Qwen3 Flash | **Northstar CUA Fast** |\n|---|---|---|---|\n| Chrome | 62.96% | 56.43% | **55.30%** |\n| Thunderbird | 73.33% | 66.67% | **62.40%** |\n| LibreOffice Writer | 60.87% | 56.52% | **56.94%** |\n| OS | 41.67% | 54.17% | **46.26%** |\n| VLC | 49.94% | 34.41% | **43.87%** |\n| **Overall** | **53.1%** | 41.6% | 37.01% |\n\n\u003e Northstar CUA Fast is competitive with open-source models on single-app tasks. Using the EVOCUA agent harness: EVOCUA-8B averages 32.5% vs Northstar CUA Fast (RL) at 37.0%. See our [research blog](https://www.tzafon.ai/blog/training-vlm-for-cua) for training details.\n\n---\n\n## License\n\nThe code in this repository is released under the [Apache License 2.0](LICENSE).\n\n## Citation\n\n```bibtex\n@misc{tzafon2026northstarcuafast,\n    title={Northstar CUA Fast: Lightweight Computer-Use Agent Model},\n    author={Tzafon Team},\n    year={2026},\n    url={https://github.com/tzafon/lightcone},\n}\n```\n\n## Contact\n\nQuestions or feedback? Reach out at **support@tzafon.ai** or open an issue.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftzafon%2Flightcone","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftzafon%2Flightcone","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftzafon%2Flightcone/lists"}