{"id":19484971,"url":"https://github.com/opencog/rocca","last_synced_at":"2025-04-25T16:33:54.896Z","repository":{"id":38107274,"uuid":"281368515","full_name":"opencog/rocca","owner":"opencog","description":"Rational OpenCog Controlled Agent (ROCCA). Use OpenCog to control a rational agent in OpenAI Gym and Malmo environments.","archived":false,"fork":false,"pushed_at":"2023-04-05T16:26:21.000Z","size":1801,"stargazers_count":27,"open_issues_count":5,"forks_count":17,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-10-20T22:12:03.422Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-07-21T10:36:17.000Z","updated_at":"2023-10-21T12:38:16.849Z","dependencies_parsed_at":"2022-07-08T01:40:37.973Z","dependency_job_id":"67c4c9d4-1263-4088-8020-c9eba2b7fdb3","html_url":"https://github.com/opencog/rocca","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Frocca","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Frocca/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Frocca/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Frocca/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencog","download_url":"https://codeload.github.com/opencog/rocca/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224009959,"owners_count":17240651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T20:24:56.408Z","updated_at":"2024-11-10T20:24:57.570Z","avatar_url":"https://github.com/opencog.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rational OpenCog Controlled Agent\n\n\u003cp align=\"left\"\u003e\n   \u003ca href=\"https://github.com/opencog/rocca/actions\"\u003e\n      \u003cimg alt=\"CI Status\" src=\"https://github.com/opencog/rocca/actions/workflows/main.yml/badge.svg\"\u003e\n   \u003c/a\u003e\n   \u003ca href=\"https://github.com/psf/black\"\u003e\n      \u003cimg src=\"https://img.shields.io/badge/code%20style-black-000000.svg\" alt=\"Code style: black\" /\u003e\n   \u003c/a\u003e\n\u003c/p\u003e\n\n## Description\n\nRational OpenCog Controlled Agent, or ROCCA, is a project aiming at\ncreating an opencog agent that acts rationally in OpenAI Gym\nenvironments (including Minecraft via MineRL and Malmo).\n\nAt its core it relies on PLN (Probabilistic Logic Networks) for both\nlearning and planning.  In practice most of the learning is however\nhandled by the pattern miner, which can be seen as a specialized form\nof PLN reasoning.  Planning, the discovery of cognitive schematics, is\nhandled by PLN and its temporal reasoning rule base.  Decision is\ncurrently a hardwired module, heavily inspired by OpenPsi with a more\nrational sampling procedure (Thompson Sampling for better exploitation\nvs exploration tradeoff).\n\n## Status\n\nFor now learning is able to\n\n1. Discover temporal patterns based on directly observable events via\n   the pattern miner.\n2. Turn these temporal patterns into plans (cognitive schematics).\n3. Combine these plans to form new plans, possibly composed of new\n   action sequences, via temporal deduction.\n\nThe next steps are\n\n1. Add more sophisticated temporal (including dealing with longs lags\n   between cause and effect) and then spatial inference rules.\n2. Integrate ECAN, for Attention Allocation, to dynamically restrict\n   the atomspace to subsets of items to process/pay-attention-to.\n3. Record attention spreading to learn/improve Hebbian links.\n4. Carry concept creation and schematization (crystallized attention\n   allocation).\n5. Record internal processes, not just attention spreading, as\n   percepta to enable deeper forms of instrospective reasoning.\n6. Plan internal actions, not just external, to enable self-growth.\n\n## Requirements\n\nOpenCog tools\n\n- cogutil (tested with revision 555a003)\n- atomspace (tested with revision 396e1e7)\n- unify (tested with revision 1e93141)\n- ure (tested with revision 4e01b02)\n- spacetime (tested with revision 962862c)\n- pln (tested with revision 08c100f)\n- miner (tested with revision 15befc4)\n\nThird party tools\n\n- Python 3.10 (or Python 3.8 see below)\n- jupyter notebook\n- python-orderedmultidict https://pypi.org/project/orderedmultidict/\n- fastcore https://fastcore.fast.ai\n- OpenAI Gym https://gym.openai.com/\n- MineRL https://minerl.io\n- nbdev https://nbdev.fast.ai\n- black https://pypi.org/project/black/\n\n### Python 3.10 vs 3.8\n\nPython 3.10 offers a better out-of-the-box type annotation system than\nPython 3.8 and is thus the default required version.  However you may\nstill use Python 3.8 by checking out the\n[python-3.8-compatible](https://github.com/opencog/rocca/tree/python-3.8-compatible)\nbranch.  Beware that such Python 3.8 branch may not be as well\nmaintained as the master.\n\n## Install\n\nIn the root folder enter the following command (you might need to be\nroot depending on your system):\n\n```bash\npip install -e .\n```\n\nFor the tools used for development:\n```bash\npip install -r requirements-dev.txt\n```\n\n## How to use\n\nAn OpencogAgent defined under the `rocca/agents` folder is provided\nthat can used to implement agents for given environments.  See the\nexamples under the `examples` folder.\n\nThere are Jupyter notebooks provided for experimentation as well.  To\nrun them call jupyter notebook on a ipynb file, such as\n\n```bash\njupyter notebook 01_cartpole.ipynb\n```\n\n### TensorBoard support\n\nSome experiments, notably the notebooks, use TensorBoard via the\n`tensorboardX` library to store event files that show certain metrics\nover time for training / testing (for now it's just rewards).\n\nBy default, event files will be created under the\n`runs/\u003cdatetime\u003e\u003ccomment\u003e` directory. You can invoke `tensorboard\n--logdir runs` from the project root to start an instance that will\nsee all the files under that directory. Open your browser to\n`http://localhost:6006` to see its interface.\n\n## Develop\n\nIf you write code in notebooks that is exported (has the `#export`\ncomment on top of the cell), remember to invoke `nbdev_build_lib` to\nupdate the library. Remember to use `black` for formatting, you can\ninvoke `black .` from the project root to format everything.\n\nYou can also use the Makefile for your convenience, invoking `make\nrocca` will do both of the above in sequence.\n\n### Development container\n\nThe `.devcontainer` folder has configuration for [VS Code\ndevcontainer](https://code.visualstudio.com/docs/remote/containers)\nfunctionality. You can use it to setup a development environment very\nquickly and regardless of the OS you use.\n\n- The container has a JupyterLab instance running on the port 8888.\n- The container has a VNC server running on the port 5901.\n- The password for the VNC server started in the container is\n  `vncpassword`. You can use any VNC client to see the results of\n  rendering Gym environments this way.\n\n## Tests\n\n### Static type checking\n\nUsing [type annotations](https://mypy.readthedocs.io/en/stable/getting_started.html)\nis highly encouraged.  One can type check the entire Python ROCCA code\nby calling\n\n```\ntests/mypy.sh\n```\n\nfrom the root folder.\n\nTo only type check some subfolder, you may call `mypy.sh` from that\nsubfolder.  For instance to type check the `examples` subfolder\n\n```\ncd examples\n../tests/mypy.sh\n```\n\nOr directly run `mypy` on files or directories, such as\n\n```\nmypy rocca/rocca/agents/core.py\n```\n\n`tests/mypy.sh` merely calls mypy on all python files in the directory\nfrom which it is called while filtering out some error messages.\n\n### Unit tests\n\nSimply run `pytest` in the root folder.\n\n## References\n\nThere is no ROCCA paper per se yet.  In the meantime here is a list of related references\n\n+ [An Inferential Approach to Mining Surprising Patterns in Hypergraphs, Nil Geisweiller et Ben Goertzel](https://www.researchgate.net/publication/334769428_An_Inferential_Approach_to_Mining_Surprising_Patterns_in_Hypergraphs)\n+ [Partial Operator Induction with Beta Distributions, Nil Geisweiller](https://raw.githubusercontent.com/ngeiswei/papers/master/PartialBetaOperatorInduction/PartialBetaOperatorInduction.pdf)\n+ [Thompson Sampling is Asymptotically Optimal in General Environments, Jan Leike et al](http://auai.org/uai2016/proceedings/papers/20.pdf)\n+ [Draft about temporal reasoning in PLN, Nil Geisweiller](https://github.com/opencog/pln/blob/master/opencog/pln/rules/temporal/temporal-reasoning.md)\n+ [Presentation and Demo of Temporal and Procedural Reasoning with OpenCog, AGI-21, Nil Geisweiller et Hedra Yusuf](https://odysee.com/@ngeiswei:d/AGI-21-Temporal-Procedural-Reasoning-Nil-Geisweiller-Hedra-Yusuf:6)\n+ [References about OpenCog including PLN](https://wiki.opencog.org/w/Background_Publications)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencog%2Frocca","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencog%2Frocca","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencog%2Frocca/lists"}