{"id":23935145,"url":"https://github.com/osh/kerlym","last_synced_at":"2025-09-17T15:47:53.523Z","repository":{"id":141758976,"uuid":"58911266","full_name":"osh/kerlym","owner":"osh","description":"KEras Reinforcement Learning gYM agents","archived":false,"fork":false,"pushed_at":"2017-07-08T03:48:20.000Z","size":1013,"stargazers_count":290,"open_issues_count":5,"forks_count":60,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-04-09T23:17:28.994Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://kerlym.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/osh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-05-16T07:08:18.000Z","updated_at":"2025-03-21T16:05:57.000Z","dependencies_parsed_at":null,"dependency_job_id":"a402eeca-d28a-4ad6-a054-d064d20d59f1","html_url":"https://github.com/osh/kerlym","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osh%2Fkerlym","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osh%2Fkerlym/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osh%2Fkerlym/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/osh%2Fkerlym/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/osh","download_url":"https://codeload.github.com/osh/kerlym/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248125593,"owners_count":21051771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-06T00:44:45.893Z","updated_at":"2025-09-17T15:47:48.502Z","avatar_url":"https://github.com/osh.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# THIS REPO IS DEPRECATED!!!\n\nPlease use something which is actually kept up to date and properly debugged such as RLLAB, https://github.com/openai/rllab\n\n# KEras Reinforcement Learning gYM agents, KeRLym\n\nThis repo is intended to host a handful of reinforcement learning agents implemented using the Keras (http://keras.io/) deep learning library for Theano and Tensorflow.\nIt is intended to make it easy to run, measure, and experiment with different learning configuration and underlying value function approximation networks while running a variery of OpenAI Gym environments (https://gym.openai.com/).\n\n![Screenshot img](/examples/example.png?raw=true \"KeRLym Screenshot\")\n\n# Agents\n\n - pg: policy gradient method with Keras NN policy network\n - dqn: q-learning agent with Keras NN Q-fn approximation (/w concurrent actor-learners)\n\n# Installation\n\n```\nsudo python setup.py install\n```\n\n# Usage\n\n```\n./run_pong.sh\n```\n\nor\n\n```\nExmaple: kerlym -e Go9x9-v0 -n simple_dnn -P\n\nUsage: kerlym [options]\n\nOptions:\n  -h, --help            show this help message and exit\n  -e ENV, --env=ENV     Which GYM Environment to run [Pong-v0]\n  -n NET, --net=NET     Which NN Architecture to use for Q-Function\n                        approximation [simple_dnn]\n  -b BS, --batch_size=BS\n                        Batch size durring NN training [32]\n  -o DROPOUT, --dropout=DROPOUT\n                        Dropout rate in Q-Fn NN [0.5]\n  -p EPSILON, --epsilon=EPSILON\n                        Exploration(1.0) vs Exploitation(0.0) action\n                        probability [0.1]\n  -D EPSILON_DECAY, --epsilon_decay=EPSILON_DECAY\n                        Rate of epsilon decay: epsilon*=(1-decay) [1e-06]\n  -s EPSILON_MIN, --epsilon_min=EPSILON_MIN\n                        Min epsilon value after decay [0.05]\n  -d DISCOUNT, --discount=DISCOUNT\n                        Discount rate for future reards [0.99]\n  -t NFRAMES, --num_frames=NFRAMES\n                        Number of Sequential observations/timesteps to store\n                        in a single example [2]\n  -m MAXMEM, --max_mem=MAXMEM\n                        Max number of samples to remember [100000]\n  -P, --plots           Plot learning statistics while running [False]\n  -F PLOT_RATE, --plot_rate=PLOT_RATE\n                        Plot update rate in episodes [10]\n  -a AGENT, --agent=AGENT\n                        Which learning algorithm to use [dqn]\n  -i, --difference      Compute Difference Image for Training [False]\n  -r LEARNING_RATE, --learning_rate=LEARNING_RATE\n                        Learning Rate [0.0001]\n  -E PREPROCESSOR, --preprocessor=PREPROCESSOR\n                        Preprocessor [none]\n  -R, --render          Render game progress [False]\n  -c NTHREADS, --concurrency=NTHREADS\n                        Number of Worker Threads [1]\n```\n\nor\n\n```python\nfrom gym import envs\nenv = lambda: envs.make(\"SpaceInvaders-v0\")\n\nimport kerlym\nagent = kerlym.agents.DQN(\n                    env=env, \n                    nframes=1, \n                    epsilon=0.5, \n                    discount=0.99, \n                    modelfactory=kerlym.dqn.networks.simple_cnn,\n                    batch_size=32, \n                    dropout=0.1, \n                    enable_plots = True, \n                    epsilon_schedule=lambda episode,epsilon: max(0.1, epsilon*(1-1e-4)),                \n                    dufference_obs = True,\n                    preprocessor = kerlym.preproc.karpathy_preproc,\n                    learning_rate = 1e-4, \n                    render=True\n                    )\nagent.train()\n```\n\n# Custom Action-Value Function Network\n\n```\ndef custom_Q_nn(agent, env, dropout=0, h0_width=8, h1_width=8, **args):\n    S = Input(shape=[agent.input_dim])\n    h = Reshape([agent.nframes, agent.input_dim/agent.nframes])(S)\n    h = TimeDistributed(Dense(h0_width, activation='relu', init='he_normal'))(h)\n    h = Dropout(dropout)(h)\n    h = LSTM(h1_width, return_sequences=True)(h)\n    h = Dropout(dropout)(h)\n    h = LSTM(h1_width)(h)\n    h = Dropout(dropout)(h)\n    V = Dense(env.action_space.n, activation='linear',init='zero')(h)\n    model = Model(S,V)\n    model.compile(loss='mse', optimizer=RMSprop(lr=0.01) )\n    return model\n\nagent = keras.agents.D2QN(env, modelfactory=custom_Q_nn)\n\n```\n\n# Citation\n\nIf using this work in your research, citation of our publication introducing this platform would be greatly appreciated!\nThe arXiv paper is available at https://arxiv.org/abs/1605.09221 and a simple bibtex entry is provided below.\n\n```\n@misc{1605.09221,\nAuthor = {Timothy J. O'Shea and T. Charles Clancy},\nTitle = {Deep Reinforcement Learning Radio Control and Signal Detection with KeRLym, a Gym RL Agent},\nYear = {2016},\nEprint = {arXiv:1605.09221},\n}\n```\n\n# Acknowledgements\n\nMany thanks to the projects below for their inspiration and contributions\n - https://github.com/dandxy89/rf_helicopter\n - https://github.com/sherjilozair/dqn\n - https://gist.github.com/karpathy/a4166c7fe253700972fcbc77e4ea32c5\n - https://github.com/coreylynch/async-rl\n - Keras, Gym, TensorFlow and Theano\n\n-Tim\n\n# Remind\nInstall `pip install gym` and `pip install gym[atari]`. If gym[atari] has install error, `apt-get install cmake`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosh%2Fkerlym","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fosh%2Fkerlym","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fosh%2Fkerlym/lists"}