{"id":20756530,"url":"https://github.com/dragoncomputer/cerebrum","last_synced_at":"2025-07-04T06:07:02.843Z","repository":{"id":62561390,"uuid":"50776012","full_name":"DragonComputer/Cerebrum","owner":"DragonComputer","description":"Crossmodal Supervised Learning Toolkit using High-Performance Extreme Learning Machines over the audio-visual-textual data","archived":false,"fork":false,"pushed_at":"2019-03-04T10:26:57.000Z","size":12269,"stargazers_count":44,"open_issues_count":6,"forks_count":19,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-15T09:19:14.763Z","etag":null,"topics":["audio","cerebrum","neural-network","python","supervised-learning","textual","visual"],"latest_commit_sha":null,"homepage":"https://pypi.python.org/pypi/cerebrum","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DragonComputer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-31T13:35:50.000Z","updated_at":"2025-03-30T05:36:56.000Z","dependencies_parsed_at":"2022-11-03T15:15:27.184Z","dependency_job_id":null,"html_url":"https://github.com/DragonComputer/Cerebrum","commit_stats":null,"previous_names":["mertyildiran/cerebrum"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/DragonComputer/Cerebrum","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DragonComputer%2FCerebrum","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DragonComputer%2FCerebrum/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DragonComputer%2FCerebrum/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DragonComputer%2FCerebrum/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DragonComputer","download_url":"https://codeload.github.com/DragonComputer/Cerebrum/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DragonComputer%2FCerebrum/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260890033,"owners_count":23077857,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio","cerebrum","neural-network","python","supervised-learning","textual","visual"],"created_at":"2024-11-17T09:32:59.228Z","updated_at":"2025-06-20T05:35:05.032Z","avatar_url":"https://github.com/DragonComputer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cerebrum\n\nCrossmodal Supervised Learning Toolkit using High-Performance Extreme Learning Machines over the audio-visual-textual data\n\n![Parts of The Human Brain (Psychology)](https://raw.githubusercontent.com/mertyildiran/Cerebrum/master/docs/img/areas-of-memory.jpg)\n\nParts of The Cerebrum:\n\n - Cerebrum\n\n\t- Vision\n\t\t- Amodal Perception\n\t\t- Color Perception\n\t\t- Depth Perception (in Future) (needs Stereoscopic Vision)\n\t\t- Form Perception (in Future) (needs Stereoscopic Vision)\n\t\t- Relative Velocity Perception (in Future) (needs Stereoscopic Vision)\n\t- Hearing\n\t\t- Speech Perception\n\t\t- Rhythmic Perception\n\t\t- Harmonic Perception (in Future) (WARNING: High Complexity)\n\t\t- Acceleration Perception (in Future) (needs 2 units of Triple Axis Accelerometer)\n\t- Language\n\t\t- Speech Analysis\n\t\t- Speech Synthesis\n\t- Multisensorial (Not Yet Available)\n\t\t- Touching (Not Yet Available)\n\t\t\t- Mechanic Perception (in Future) (needs lots of Pressure Sensors)\n\t\t\t- Heat \u0026 Cooling Perception (in Future) (needs lots of Temperature Sensors)\n\t\t- Tasting (Not Yet Available)\n\t\t\t- Solid/Fluid State Chemical Perception (in Future) (WARNING: Sensor Technology Not Available)\n\t\t- Smelling (Not Yet Available)\n\t\t\t- Gas State Chemical Perception (in Future) (WARNING: Sensor Technology Not Available)\n\t- Emotion (Not Yet Available)\n\t\t- Activation - Pleasant : Alerted, Excited, Elated, Happy\n\t\t- Pleasant - Deactivation : Contented, Serene, Relaxed, Calm\n\t\t- Deactivation - Unpleasant : Fatigued, Bored, Depressed, Sad\n\t\t- Unpleasant - Activation : Upset, Stressed, Nervous, Tense\n - Crossmodal: Defines time based relations between the types of perceptions\n - NeuralNet: Multiple Neural Network interconnects the parts of The Cerebrum\n\t - Current Speech Sequence to Current Visual Sequence\n\t - Current Speech Sequence to Next Speech Sequence\n\t - Current Visual Sequence to Current Speech Sequence\n\t - Current Vİsual Sequence to Next Visual Sequence\n\t - Current Speech Sequence to Text Sequence\n\n*For Multisensorial Part technological advancements are insufficient, currently. Because of this deficiency, there should be a False Reward \u0026 Punishment Mechanism for Reinforcement Learning*\n\n![Mapping Morphometry and Connectedness of the Human Brain](https://raw.githubusercontent.com/mertyildiran/Cerebrum/master/docs/img/connectedness-of-brain.png)\n\n\n\u003e Cerebrum's purpose is getting continuous data inputs from different types of perceptions in real time as\n\u003e memory sequences that triggered according to predefined threshold values and creating complex time\n\u003e based relations between those memories by Crossmodal logic and training multiple Artificial\n\u003e Neural Networks with this extracted data. Lastly creating outputs triggered by a stimuli, using\n\u003e pre-trained Artificial Neural Networks. - *Mehmet Mert Yıldıran*\n\n### Hebbian Theory\n\n![Experience-dependent spine formation and elimination.](https://raw.githubusercontent.com/mertyildiran/Cerebrum/master/docs/img/hebbian.jpg)\n\n### High-Performance Extreme Learning Machines\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/mertyildiran/Cerebrum/master/docs/img/elm.jpg\" width=\"60%\"\u003e\u003c/p\u003e\n\n### Version\n0.1.81\n\n### Installation\n\nInstall RethinkDB:\n\n```Shell\nsource /etc/lsb-release \u0026\u0026 echo \"deb http://download.rethinkdb.com/apt $DISTRIB_CODENAME main\" | sudo tee /etc/apt/sources.list.d/rethinkdb.list\nwget -qO- https://download.rethinkdb.com/apt/pubkey.gpg | sudo apt-key add -\nsudo apt-get update\nsudo apt-get install rethinkdb\n```\n\nWatch data flow (optional) - \u003chttp://localhost:8080\u003e\n\nInstall Cerebrum:\n\n```Shell\nsudo apt-get install python-pyaudio python-opencv python-scipy python-qt4 python-tk\nsudo pip install cerebrum\n```\n\n### Usage\n\nTrain with files:\n\n```Shell\ncerebrum --video PATH_TO_VIDEO_FILE --audio PATH_TO_AUDIO_FILE --captions PATH_TO_CAPTIONS_FILE\n```\n\nExample:\n\n```Shell\ncerebrum --video trainingData/Can\\ You\\ Make\\ Someone\\ Fall\\ In\\ Love\\ With\\ You-7_w_EA4u6oQ.mp4 --audio trainingData/Can\\ You\\ Make\\ Someone\\ Fall\\ In\\ Love\\ With\\ You-7_w_EA4u6oQ.wav --captions trainingData/Can\\ You\\ Make\\ Someone\\ Fall\\ In\\ Love\\ With\\ You-7_w_EA4u6oQ.en.vtt\n```\n\nTrain with your cam (Not recommended):\n\n```Shell\ncerebrum --video 0 --audio 0\n```\n\n*On the first run Cerebrum will start with a few more seconds delay.*\n\n#### Screenshot\n\n![Screenshot](https://raw.githubusercontent.com/mertyildiran/Cerebrum/master/docs/img/screenshot.png)\n\n#### RethinkDB Web Interface\n\n\u003chttp://localhost:8080\u003e\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://raw.githubusercontent.com/mertyildiran/Cerebrum/master/docs/img/rethinkdb.png\"\u003e\u003c/p\u003e\n\n### Dependencies\n\nCerebrum uses a number of open source libraries to do the job:\n\n* [Python 2.7] - a widely used general-purpose, high-level programming language.\n* [RethinkDB] - is the open-source, scalable database that makes building realtime apps dramatically easier.\n* [PyAudio] - provides Python bindings for PortAudio, the cross platform audio API.\n* [OpenCV] - (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision.\n* [wave Module] - provides a convenient interface to the WAV sound format.\n* [datetime Module] - supplies classes for manipulating dates and times in both simple and complex ways.\n* [os.path Module] - the path module suitable for the operating system Python is running on, and therefore usable for local paths.\n* [sys Module] - provides access to some variables used or maintained by the interpreter and to functions that interact strongly with the interpreter. It is always available.\n* [audioop Module] - operates on sound fragments consisting of signed integer samples 8, 16 or 32 bits wide, stored in Python strings.\n* [NumPy] - the fundamental package for scientific computing with Python.\n* [multiprocessing Module] - a package that supports spawning processes using an API similar to the threading module.\n* [imutils Module] - a series of convenience functions to make basic image processing functions such as translation, rotation, resizing, skeletonization etc.\n* [PyQtGraph] - a pure-python graphics and GUI library built on PyQt4 / PySide and numpy\n* [PyQt4] - a comprehensive set of Python bindings for Digia's Qt cross platform GUI toolkit.\n* [time Module] - provides various time-related functions.\n* [argparse Module] - makes it easy to write user-friendly command-line interfaces.\n* [os Module] - provides a portable way of using operating system dependent functionality.\n* [subprocess Module] - allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes.\n* [random Module] - pseudo-random number generators for various distributions.\n* [pysrt Module] - SubRip (.srt) subtitle parser and writer\n* [itertools Module] - implements a number of iterator building blocks inspired by constructs from APL, Haskell, and SML. Each has been recast in a form suitable for Python\n\n[Python 2.7]: \u003chttps://www.python.org/download/releases/2.7/\u003e\n[RethinkDB]: \u003chttps://www.rethinkdb.com/\u003e\n[PyAudio]: \u003chttps://people.csail.mit.edu/hubert/pyaudio/r\u003e\n[OpenCV]: \u003chttp://opencv.org/r\u003e\n[wave Module]: \u003chttps://docs.python.org/2/library/wave.html\u003e\n[datetime Module]: \u003chttps://docs.python.org/2/library/datetime.html\u003e\n[os.path Module]: \u003chttps://docs.python.org/2/library/os.path.html\u003e\n[sys Module]: \u003chttps://docs.python.org/2/library/sys.html\u003e\n[audioop Module]: \u003chttps://docs.python.org/2/library/audioop.html\u003e\n[NumPy]: \u003chttp://www.numpy.org/\u003e\n[multiprocessing Module]: \u003chttps://docs.python.org/2/library/multiprocessing.html\u003e\n[imutils Module]: \u003chttps://pypi.python.org/pypi/imutils/0.2\u003e\n[PyQtGraph]: \u003chttp://www.pyqtgraph.org/\u003e\n[PyQt4]: \u003chttps://pypi.python.org/pypi/PyQt4\u003e\n[time Module]: \u003chttps://docs.python.org/2/library/time.html\u003e\n[argparse Module]: \u003chttps://docs.python.org/2.7/library/argparse.html\u003e\n[os Module]: \u003chttps://docs.python.org/2/library/os.html\u003e\n[subprocess Module]: \u003chttps://docs.python.org/2/library/subprocess.html\u003e\n[random Module]: \u003chttps://docs.python.org/2/library/random.html\u003e\n[pysrt Module]: \u003chttps://pypi.python.org/pypi/pysrt\u003e\n[itertools Module]: \u003chttps://docs.python.org/2/library/itertools.html\u003e\n\n### License\n\nThe MIT License (MIT)\n\nCopyright (c) 2016 Mehmet Mert Yıldıran mert.yildiran@bil.omu.edu.tr\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdragoncomputer%2Fcerebrum","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdragoncomputer%2Fcerebrum","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdragoncomputer%2Fcerebrum/lists"}