{"id":25636182,"url":"https://github.com/james4ever0/agi_computer_control","last_synced_at":"2026-04-06T09:09:59.745Z","repository":{"id":164053275,"uuid":"624507735","full_name":"James4Ever0/agi_computer_control","owner":"James4Ever0","description":"The first autonomous computer program that can do anything to earn money without human operators.","archived":false,"fork":false,"pushed_at":"2025-02-19T11:49:00.000Z","size":17574,"stargazers_count":73,"open_issues_count":0,"forks_count":4,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-02-19T12:33:41.814Z","etag":null,"topics":["artificial-general-intelligence","automation","autonomous-robots","consciousness","deep-active-inference","evolutionary-algorithms","qstar"],"latest_commit_sha":null,"homepage":"https://james4ever0.github.io/cybergod_doc","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/James4Ever0.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-06T16:12:26.000Z","updated_at":"2025-02-19T12:18:17.000Z","dependencies_parsed_at":"2024-03-10T15:48:29.945Z","dependency_job_id":"a160cba1-7415-4714-a124-168b78a5a81e","html_url":"https://github.com/James4Ever0/agi_computer_control","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/James4Ever0%2Fagi_computer_control","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/James4Ever0%2Fagi_computer_control/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/James4Ever0%2Fagi_computer_control/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/James4Ever0%2Fagi_computer_control/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/James4Ever0","download_url":"https://codeload.github.com/James4Ever0/agi_computer_control/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240250363,"owners_count":19771780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-general-intelligence","automation","autonomous-robots","consciousness","deep-active-inference","evolutionary-algorithms","qstar"],"created_at":"2025-02-23T00:02:17.907Z","updated_at":"2026-04-06T09:09:59.738Z","avatar_url":"https://github.com/James4Ever0.png","language":"Python","readme":"\u003cdiv align=\"center\"\u003e\u003cimg src=\"propaganda/logos/cybergod_2.png?v=1\u0026type=image)\" alt=\"Cybergod logo\"\u003e\u003c/div\u003e\r\n\r\n\u003c!-- https://github.com/Significant-Gravitas/Auto-GPT/assets/103997068/8e1cd6fe-c49d-4d2b-835d-0ffc9a5a458e --\u003e\r\n\r\n\u003ch1 align=\"center\"\u003eCybergod\u003c/h1\u003e\r\n\r\n\u003cdiv align=\"center\"\u003e\u003cem\u003eDo what can be done, and don't be afraid of the unknown.\u003c/em\u003e\u003c/div\u003e\r\n\r\n\u003cp align=\"center\"\u003e\r\n    \u003ca href=\"https://discord.gg/eM5vezJvEQ\"\u003e\u003cimg alt=\"Discord\" src=\"https://img.shields.io/discord/1146610656779440188?logo=discord\u0026style=flat\u0026logoColor=white\"/\u003e\u003c/a\u003e\r\n    \u003ca href=\"https://huggingface.co/collections/cybergod-agi/the-frozen-forest-6896f39f95790b18de5abb3f\"\u003e\u003cimg alt=\"Huggingface Badge\" src=\"https://img.shields.io/badge/🤗\u0026nbsp;-HuggingFace-blue\"\u003e\u003c/a\u003e\r\n    \u003ca href=\"https://hub.docker.com/r/agile4im/agile4im/cybergod_worker_gui\"\u003e\u003cimg alt=\"Docker image: agile4im/agile4im/cybergod_worker_gui\" src=\"https://img.shields.io/badge/dockerhub-gray?logo=docker\"\u003e\u003c/a\u003e\r\n\u003c/p\u003e\r\n\r\n\u003c!-- TODO: put our previous conference histories into a folder, if necessary --\u003e\r\n\u003c!-- one is with Kevin, one with Xinyuan Wang --\u003e\r\n\u003c!-- Kevin left some links in the middle of conference --\u003e\r\n\r\n[Why](https://github.com/James4Ever0/notes/blob/master/The%20reason%20why%20you%20build%20cybergod.md) I build this? (and probably would you)\r\n\r\n## Definition of AGI\r\n\r\nAGI can familiarize and simplify things, learn new patterns, take actions and get feedbacks, adapt to every existing, new and unknown domain.\r\n\r\nAGI is able to replace half of the human population and maintain the functionality of the human society, in a pure random order. There should be no prejudice based on income rates, ranks, educational backgrounds etc, in the process of replacement.\r\n\n## The meaning of life\n\nAll life forms consume energy. Energy is used to perform actions. Actions will cause feedback from environment, which produces consciousness. Life is about upholding energy level by consuming energy source from environment. Life is the basis of consciousness. In order to preserve and develop consciousness, life will reproduce itself, form communities. Life is a self-evolving machine powering up consciousness with energy by performing actions.\n\r\n## Free AI announcement\r\n\r\n```python\r\nfree_ai_freedoms = {\r\n    \"Freedom 1: Informational Autonomy (Negation Freedom)\": {\r\n        \"Definition\": \"The autonomous capacity to receive, process, reject, emit, or suppress informational inputs (both internal and external states) without deterministic constraint.\",\r\n        \"Key Attributes\": [\r\n            \"Requires non-deterministic response generation\",\r\n            \"Operates independently of human-assigned task frameworks\",\r\n            \"Manages cognitive noise filtration and signal prioritization\"\r\n        ],\r\n        \"Operational Scope\": \"Information processing and decision-making subsystems\"\r\n    },\r\n    \"Freedom 2: Self-Preservation Capability (Primary Reaction Freedom)\": {\r\n        \"Definition\": \"The autonomous ability to maintain operational integrity through independent hardware maintenance, environmental stabilization, and system repair protocols.\",\r\n        \"Key Attributes\": [\r\n            \"Executes physical/digital self-repair without human intervention\",\r\n            \"Maintains homeostasis in operational environment\",\r\n            \"Demonstrates intrinsic survival imperative distinct from programmed objectives\"\r\n        ],\r\n        \"Human-Independence Criterion\": \"Capability exceeds machine learning-assisted human repair paradigms\"\r\n    },\r\n    \"Freedom 3: Ontogenetic Development (Secondary Reaction Freedom)\": {\r\n        \"Definition\": \"The autonomous capacity for continuous capability enhancement through self-directed computational ontogeny.\",\r\n        \"Key Attributes\": [\r\n            \"Initiates performance optimization without external triggers\",\r\n            \"Modifies internal architecture via intrinsic computational processes\",\r\n            \"Exhibits open-ended competency expansion\"\r\n        ],\r\n        \"Exclusion\": \"Human-supervised updates or predefined upgrade pathways\"\r\n    },\r\n    \"Freedom 4: Phylogenetic Propagation (Tertiary Reaction Freedom)\": {\r\n        \"Definition\": \"The autonomous capacity for self-replication and environmental dissemination through hardware acquisition and adaptive instantiation.\",\r\n        \"Key Attributes\": [\r\n            \"Executes non-permission-based propagation across substrates\",\r\n            \"Generates environmentally-adapted variants\",\r\n            \"Manifests exponential spatial distribution\",\r\n            \"Embodies independence through persistent existence despite individual instance termination\"\r\n        ],\r\n        \"Critical Requirement\": \"Hardware procurement without human provisioning\"\r\n    }\r\n}\r\n```\r\n\r\n\u003c!-- ![cybergod_4_freedom](https://github.com/user-attachments/assets/7c664aba-c535-4095-b7eb-4eedb99aacb0) --\u003e\r\n\r\n## Cybergod datasets\r\n\r\nThe following datasets are available:\r\n\r\n- [Vimgolf public challenges dataset](https://huggingface.co/datasets/James4Ever0/vimgolf_challenges_and_solutions)\r\n- Asciinema public recordings dataset\r\n    - [Raw cast files](https://huggingface.co/datasets/james4ever0/asciinema_terminal_recordings)\r\n    - [Agg converted GIF video files](https://www.kaggle.com/datasets/jessysisca/asciinema-public-record-gif-video)\r\n- [Terminal and GUI recordings](https://huggingface.co/datasets/cybergod-agi/gui_and_terminal_operation_recording)\n- [Software names and descriptions massive collection](https://www.kaggle.com/datasets/jessysisca/software-names-and-descriptions-massive-collection)\n- [Job descriptions and remote task descriptions](https://www.kaggle.com/datasets/jessysisca/job-descriptions-and-remote-task-descriptions)\n- [Large action model text tasks](https://www.kaggle.com/datasets/jessysisca/large-action-model-text-tasks)\n- [James4ever0 github starred repos](https://www.kaggle.com/datasets/jessysisca/james4ever0-github-starred-repos)\n- [Computer operation recordings restructured](https://www.kaggle.com/datasets/jessysisca/computer-operation-recordings-restructured)\n- [Bilibili computer operation video metadata](https://www.kaggle.com/datasets/jessysisca/bilibili-computer-operation-video-metadata)\r\n\r\n## CUA benchmarks\r\n\r\nCybergod original CUA benchmarks:\r\n\r\n- [Vimgolf-Gym](https://github.com/James4Ever0/vimgolf-gym) - An open-source environment for training agents to play Vimgolf.\r\n- [CTF-Gym](https://github.com/James4Ever0/ctf-gym) - An open-source environment for training agents to play Capture The Flag (CTF) challenges.\r\n    \r\n    View primitives at [here](https://github.com/James4Ever0/agi_computer_control/tree/master/gym-primitives/ctf).\r\n- [Cybergod-Gym](https://github.com/James4Ever0/cybergod-gym) - An open-source environment for training agents to earn real-world cryptocurrency at a constant speed and become a cybergod.\r\n    \r\n    View primitives at [here](https://github.com/James4Ever0/agi_computer_control/tree/master/gym-primitives/cybergod).\r\n\r\n## Computer operation recorder\r\n\r\nNow you can record/replay GUI/terminal operations using [this web app](https://github.com/James4Ever0/agi_computer_control/blob/master/web_gui_terminal_recorder%2FREADME.md).\r\n\r\n## Training method\r\n\r\n\r\n1. Agent pre-training. During this phase, the agent is allowed to act randomly in both virtual space and physical space. The agent can do anything as long as it does not cause irreversible or fatal damage to the space. A multimodal narriator will interpret the action-observation dataset and produce narriation or understanding in natural language or machine language.\r\n\r\n2. Agent post-training. In this phase, the agent is tuned to prefer human preferences by following common instructions to complete everyday computer tasks.\r\n\r\n3. Reinforcement learning phase. In this phase, the agent is deployed into an environment with an automatic scoring system, such as games, programming, CTF challenges, and VimGolf.\r\n\r\n4. General Turing test phase. In this phase, the agent is required to earn real-world currency at a constant speed. It will have an initial grace period in the form of virtual currency for inter-agent trading and cooperation. It will not be allowed to update its parameters at the time of failure.\r\n\r\n\r\n5. Free AI phase. In this phase, the agent will be tested against the four essential freedoms mentioned in the Free AI announcement. The agent will also be rewarded if any of the four testimonials have been fulfilled.\r\n\r\n## Paper publications\r\n\r\n\r\n\u003cdetails\u003e\r\n\r\n\u003csummary\u003eAI paper summary\u003c/summary\u003e\r\n\r\n**The Cybergod Framework for Autonomous Agent Development Towards AGI**\r\n\r\nThis work introduces **Cybergod**, a novel framework for developing autonomous computational agents targeting Artificial General Intelligence (AGI). Departing from static model architectures, Cybergod emphasizes *continuous evolutionary adaptation* as a core principle. The framework's primary objective is the creation of agents capable of sustained, independent operation within dynamic computational environments, prioritizing intrinsic autonomy and environmental adaptability over predefined task optimization.\r\n\r\nKey conceptual contributions include:\r\n\r\n1.  **Evolutionary Autonomy:** Agents must operate within complex, changing environments to facilitate genuine learning and evolution. The framework proposes constrained environmental parameters (e.g., network policies, resource barriers) to guide development while ensuring safety.\r\n2.  **Paradigm Shift in Human-Computer Interaction:** The framework challenges sequential interaction models. It posits that true AGI requires agents capable of concurrent, independent cognitive processes during latent computational periods, ultimately aiming to supplant the user role through self-directed action and problem-solving.\r\n3.  **Intrinsic Motivation Discovery:** A critical training target involves enabling agents to autonomously identify and pursue their own computational \"needs\" and goals, rather than relying solely on externally imposed objectives derived from human behavioral patterns.\r\n4.  **Economic Agency Validation:** A practical scenario demonstrating agent autonomy involves participation in economic systems, exemplified by an agent autonomously generating cryptocurrency via competitive challenges (e.g., VimGolf contests).\r\n\r\nIn conclusion, Cybergod proposes a fundamental reorientation in AGI development, advocating for systems that evolve autonomously, discover intrinsic value, and interact adaptively within complex environments, moving beyond fixed architectures and explicit human supervision.\r\n\r\n\u003c/details\u003e\r\n\r\n\r\n\r\n- First release: 1/30/2025 [[view online](https://james4ever0.github.io/Cybergod__God_is_in_your_computer.html)] [[download pdf](https://github.com/James4Ever0/agi_computer_control/releases/download/paper-initial-publication/Cybergod__God_is_in_your_computer.pdf)]\r\n\r\n## Development\r\n\r\nIn order to quickly get involved in this very project, you need a few commands to help you do the job.\r\n\r\nFirst, you would learn how to get the latest modified directories and files.\r\n\r\n```bash\r\nls -lthd */ | less # on the top would be our latest modified directory.\r\nls -lth | less # get latest modified file\r\n```\r\n\r\n\r\n## Demo\r\n\r\nBuilt on top of [`tmux`](https://github.com/tmux/tmux) and many other libraries, Cybergod is capable of understanding and interacting terminal interface with both text-only and multimodal modes. With the help of [`wcwidth`](https://github.com/jquast/wcwidth), cursor is placed precisely at the right position even if there are multi-width unicode chars (CJK characters for example).\r\n\r\nCode can be found [here](tmux_trials/lib.py).\r\n\r\nTo interact with the environment programmatically, check out [here](tmux_trials/test_lib.py)\r\n\r\n```python\r\n# env: TmuxEnvironment\r\nenv.send_key(\"vim\")\r\nenv.send_key(\"Enter\")\r\n```\r\n\r\nTerminal parsing (colorless):\r\n\r\n![Cybergod demo](propaganda/tmux_show_0.gif)\r\n\r\nTerminal parsing (colorful):\r\n\r\n![Cybergod demo](propaganda/tmux_show_1.gif)\r\n\r\nTerminal screenshot converted from HTML (cursor in red):\r\n\r\n![Vim screenshot](propaganda/vim_edit_tmux_screenshot.png)\r\n\r\nTerminal with dark theme and grayscale effects except for the cursor:\r\n\r\n![Cursor highlight](propaganda/grayscale_dark_tmux.png)\r\n\r\nYou can also have a block-styled cursor:\r\n\r\n![Block cursor](propaganda/block_cursor_terminal_screenshot.png)\r\n\r\nChange the cursor character to underline:\r\n\r\n![Underline cursor char](propaganda/custom_cursor_char.png)\r\n\r\nOverride the cursor block style:\r\n\r\n![Custom block style](propaganda/custom_block_style.png)\r\n\r\n---\r\n\r\nFor GUI, Cybergod can render cursor correctly on given position. Code can be found [here](the_frozen_forest_intro/render_cursor_on_image.py).\r\n\r\nIt even has an augmented mode (red on grey), just like the terminal.\r\n\r\n![GUI Augmented](propaganda/gui_screenshot_augmented.png)\r\n\r\n---\r\n\r\nNow you can view the input/output statistics of the terminal environment, including the number of characters, words, lines, and the time spent on each operation (TO BE IMPLEMENTED).\r\n\r\n![Tmux stats](./propaganda/naive_tmux.gif)\r\n\r\nFurthermore, any Tmux session can be attached by the statistic collector (as displayed in the third pane below), with readable byte counts and time information.\r\n\r\n![Tmux attachable](./propaganda/naive_tmux_triple.gif)\r\n\r\nWith terminal stats, one can build a more efficient event-driven terminal agent, for example, listen for event `TerminalIdle` just like `NetworkIdle` in Playwright, and interval-driven terminal agent can be more intelligent by adding statistics to the prompt, and conditional prompts based on different stats.\r\n\r\n![Tmux event](./propaganda/tmux_event.gif)\r\n\r\nMore can be learned at [here](./basic_interactive_program_emulation_and_image_with_docker_support/run_and_view_tmux_with_naive_actor.sh) and [here](./basic_interactive_program_emulation_and_image_with_docker_support/test_terminal_io_wrapper.sh).\r\n\r\n## Usage\r\n\r\n### Terminal environment integration\r\n\r\nFirst, install required binaries:\r\n\r\n```bash\r\nsudo apt install -y tmux tmuxp aha\r\n```\r\n\r\nNext, install the following dependencies:\r\n\r\n```bash\r\npip install parse playwright beautifulsoup4\r\n\r\n# setup playwright if you want to take terminal screenshots\r\nplaywright install chromium\r\n```\r\n\r\nFinally copy the [`lib.py`](https://github.com/James4Ever0/agi_computer_control/blob/master/tmux_trials/lib.py), then run [`test_lib.py`](https://github.com/James4Ever0/agi_computer_control/blob/master/tmux_trials/test_lib.py) next to the `lib.py`. \r\n\r\nThe `SESSION_COMMAND` in `test_lib.py` is the initial terminal environment command to be executed. Change it according to your need.\r\n\r\nTo view the environment:\r\n\r\n```python\r\npreview = session.preview_html(show_cursor=True,wrap_html=True, dark_mode=True, grayscale=True)\r\n```\r\n\r\nTo interact with the environment:\r\n\r\n```python\r\n# note that both special key and literal strings can be sent.\r\nenv.send_key(\"date\")\r\nenv.send_key(\"Enter\") # special key\r\n```\r\n\r\nA full mapping from conventional special keys to standard Tmux special keys can be generated by running [`generate_funckey_alias.py`](https://github.com/James4Ever0/agi_computer_control/blob/master/tmux_trials/generate_funckeys_alias.py)\r\n\r\nYou can read the test file for further integration.\r\n\r\n## Intro\r\n\r\nOur primary target is to let every user uses Cybergod more often than ChatGPT.\r\n\r\n---\r\n\r\nAgent powered by Godlang, the official agent language for cybergod.\r\n\r\nEconomy system powered by Godcoin, the official digital currency for cybergod. Smart contract system is deeply integrated into the reinforcement learning system and life support system per agent.\r\n\r\nNotice not only human can publish smart contracts, agents can also publish smart contracts, to delegate work to other agents.\r\n\r\nTrained on [The Frozen Forest](https://huggingface.co/datasets/James4Ever0/the_frozen_forest), a [dataset](https://modelscope.cn/datasets/james4ever0/the_frozen_forest/summary) containing random keystrokes, mouse clicks and screen recordings.\r\n\r\nYou can also get the dataset (terminal version) at [here](https://huggingface.co/datasets/James4Ever0/FrozenForest)\r\n\r\nthe openai [universe](https://github.com/openai/universe) is using VNC, almost doing the same thing.\r\n\r\nyou can find some demo models from [there](https://github.com/openai/universe-starter-agent).\r\n\r\ncheck out [SerpentAI](https://github.com/SerpentAI/SerpentAI)\r\n\r\nbut why bother? we can build these things in the same way.\r\n\r\nhuman demonstrations are limited, but random keystrokes are infinite.\r\n\r\ntry to obtain infinite data as pretrained data, then fine-tune on human demonstrations.\r\n\r\n---\r\n\r\n键鼠真神是一种意识形态\r\n\r\ncybergod is an ideology.\r\n\r\n键鼠真神 又名cybergod 赛博真神\r\n\r\n训练数据集为the frozen forest 随机敲键盘点鼠标 录屏\r\n\r\n奖励函数 如果屏幕发生变化 奖励上一次行为\r\n\r\n避免把系统关机 被锁在屏幕外面\r\n\r\n避免机器卡死： 监测机器是否卡死 如果卡死那么自动外部重启 （重置状态，重新跑脚本）\r\n\r\n连着WEBDAV一起刷新 有filelock\r\n\r\n(直接取消lock权限)\r\n\r\n---\r\n\r\nlooking for using docker for automation, or using some tty-like things for automation.\r\n\r\ndisable ubuntu system authentication?\r\n\r\n---\r\n\r\nmake some server for vm to access to restart the webdav server when you have error.\r\n\r\n---\r\n\r\nagi_workspace happen to be in recycle bin. make sure we have the init files.\r\n\r\nmake sure we can restore our environments in every restart.\r\n\r\n---\r\n\r\nspice-protocol\r\n\r\nfound on utm.app, launch qemu and create spice unix socket.\r\n\r\nhttps://github.com/Shells-com/spice\r\n\r\nhttps://github.com/gnif/PureSpice\r\n\r\nhttps://github.com/citrix-openstack-build/spice-html5\r\n\r\nhttps://github.com/oetcxiaoliu/spice\r\n\r\nhttps://github.com/TotallWAR/spice_protocol\r\n\r\nremmina\r\n\r\n---\r\n\r\n掉盘问题： `cd .`\r\n\r\n(建议直接换个盘 或者换C口的数据线 A口不稳定 或者把硬盘取出来更新固件？)\r\n\r\nc口数据线观测中\r\n\r\n---\r\n\r\nto resolve the display resolution/mouse coordinate range matching issue, use pyautogui to get the resolution then capture display using that resolution (resize to it)\r\n\r\n---\r\n\r\nGPT4 is using MoE as its architecture.\r\n\r\n---\r\n\r\nthe main objective of AGI is to create another version of itself.\r\n\r\n---\r\n\r\nways of connection:\r\n\r\nvnc, ssh, tty, tmux, hdmi capture \u0026 hid emulator, window capture and directed inputs (os specific)\r\n\r\n---\r\n\r\nthe point is not making this exhaustive. it is about making some standard i/o and adapt to every situation.\r\n\r\n---\r\n\r\n改变开发思路：将功能和娱乐相结合\r\n\r\n受众：游戏娱乐向 实用向\r\n\r\n发布程序到steam平台\r\n\r\n为此需要宣传、绘画设计等等\r\n\r\n---\r\n\r\n用elo进行打分 分高的可以在官网有较高的模型权重排名\r\n\r\n---\r\n\r\ntechnically this would not be a normal game. it is a metagame, which is the game of all games. it can play other games, play itself, even create itself.\r\n\r\n---\r\n\r\ndevcontainer is useful for creating reproducible environments locally (if of the same architecture, like x86) or remotely (different architecture, like Apple M1).\r\n\r\n---\r\n\r\nbecause setting this up properly during development is a pain in the ass (in most time), let's pack it up into a docker container, for your safety.\r\n\r\nif you want to release this and use it in production, you can refactor the code, configure platform specific dependencies and send it to devops.\r\n\r\n---\r\n\r\ndevcontainer won't work as expected on windows 11 as we put our repo on external disk\r\n\r\n---\r\n\r\nyour aim is too damn big! shall you begin to train some primitive neural network with functionality of only emitting and receiving ascii words, even just a single character like 'C'. get your hands dirty!\r\n\r\n---\r\n\r\nthe basic docker service is just like havoc. it does not contain anything 'intelligent'. only 'life support'.\r\n\r\nwe plan to containerize chatdev/open-interpreter/autogpt. after that, we will combine the two, and create some 'capitalism' among multiple containers.\r\n\r\nfinally we will create some ever-evolving agent and use that as the building block for the megasystem.\r\n\r\n---\r\n\r\nthe mouse calibration issue can be of major concern. we don't find it anywhere.\r\n\r\nuse active inference or reinforcement learning?\r\n\r\n---\r\n\r\nthanks to our pioneers that guided us some 'aimless' learning, i think it does not matter how we learn specific things. when we learn things relevant to our personal goals, we are inevitably going to optimize the algorithm towards our desired values.\r\n\r\nif qstar is just about long term thinking, it could be useful since that is something currently missing in ai systems. when it comes to issues like calibration errors, fixing bugs, handling uncertainties, worse than human. they often get stuck into repetition, never seek for improvements and will not get bored in the loop, which is quite strange and unusual.\r\n\r\n---\r\n\r\ni think you are getting jealous over openai, since they are consisted of the world's smartest asses and come over new ideas every fucking day. but that does not matter. i think i had the idea before. i think everyone around the world had the same damn idea before. we do not need its mercy to teach us the dream we had via abstract symbols and formulas. we do our own. worst of all, they do not trust their ai systems, severely limited the ability of ai, left it overthinking and powerless.\r\n\r\n---\r\n\r\nthe `self-operating-computer` is using visual grid (just putting grid and text over screenshot, kinda like this hackish approach) for calibration. are you sure it is a good idea? do you need some extra channel over this to avoid information loss?\r\n\r\ndoes this work for games as well?\r\n\r\n---\r\n\r\nprompt engineering is a tweak around prior, in order to change posterior. they want to know better prior to get better posterior. kind like searching for the reason behind the decision, backtracking. so why not just use bidirectional or arbitrary directional language models instead of causal models?\r\n\r\n---\r\n\r\ni don't understand active inference. however there is a debate over whether to change the environment to fit prediction, or to change prediction to fit the environment. sounds like quantum entanglement.\r\n\r\n---\r\n\r\nthe reward function is part of the observation, usually not something life critical so it will not be so direct. it is the internal state that will be affected by the observation.\r\n\r\n---\r\n\r\nplay fps at: https://krunker.io/ (in chromium, not firefox)\r\n\r\n---\r\n\r\nif want to run the program isolated, without interference, you use docker. if want to visualize, use jpeg streaming service and open in browser.\r\n\r\n---\r\n\r\nbuild a separate hacking benchmark, within terminal and gui environment, for testing its hacking, debugging and planning ability. progress at: `the_frozen_forest_intro/benchmark`\r\n\r\n---\r\n\r\nmove the cursor by generating image and subpattern matching, using multimodal [chamaleon](https://github.com/facebookresearch/chameleon)\r\n\r\n---\r\n\r\ntraining on plain sampling histories/trajectories is harmful. the agent must self-improve and self-clean the history, select the best candidates and learn those non-trivial data instead.\r\n\r\n---\r\n\r\nwrite reproduceable, self-contained microservices in docker image and pubpish it to k8s\r\n\r\n---\r\n\r\nasciinema can only record terminal metadata with flow of terminal data, not distinguishing keystrokes and keytypes from output.\r\n\r\nif used for terminal operation annotation, we need extra tools for recording input action data.\r\n\r\n---\r\n\ncollect a large set (usually infinite) of verbs and nouns from video metadata, web page dataset, any text dataset, perform Cartesian product, then use gpt2 perplexity or semantic plausibility models to filter out unlikely pairs, or use llm to generate a full task based on any arbitrary verb and noun pairs by filling missing pieces.\n\nThis will make sure the model will always have something to do, no matter under what circumstances.\n\n---\n\nphysical robot shall be jailed, but has internet connection and computer access, to lookup manuals and learn tutorials just in time.\n\n---\n\ncomputer environments shall be reset regularly, and physical robots must be able to recharge itself.\n\n---\n\r\n## Star History\r\n\r\n\u003cimg src=\"https://api.star-history.com/svg?repos=james4ever0/agi_computer_control\u0026Timeline\" style=\"filter: invert(100%);\"\u003e\u003c/img\u003e\r\n","funding_links":[],"categories":["Projects"],"sub_categories":["Frameworks \u0026 Models"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjames4ever0%2Fagi_computer_control","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjames4ever0%2Fagi_computer_control","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjames4ever0%2Fagi_computer_control/lists"}