{"id":28494828,"url":"https://github.com/hyperopt/hyperopt-convnet","last_synced_at":"2025-07-01T18:32:55.395Z","repository":{"id":6238050,"uuid":"7469844","full_name":"hyperopt/hyperopt-convnet","owner":"hyperopt","description":"Convolutional computer vision architectures that can be tuned by hyperopt.","archived":false,"fork":false,"pushed_at":"2014-05-06T15:04:32.000Z","size":294,"stargazers_count":71,"open_issues_count":8,"forks_count":20,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-06-08T10:41:05.234Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hyperopt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-01-06T16:39:51.000Z","updated_at":"2025-05-18T12:34:46.000Z","dependencies_parsed_at":"2022-09-09T10:52:26.262Z","dependency_job_id":null,"html_url":"https://github.com/hyperopt/hyperopt-convnet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hyperopt/hyperopt-convnet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperopt%2Fhyperopt-convnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperopt%2Fhyperopt-convnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperopt%2Fhyperopt-convnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperopt%2Fhyperopt-convnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hyperopt","download_url":"https://codeload.github.com/hyperopt/hyperopt-convnet/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hyperopt%2Fhyperopt-convnet/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263017618,"owners_count":23400545,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-08T10:30:22.406Z","updated_at":"2025-07-01T18:32:55.382Z","avatar_url":"https://github.com/hyperopt.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Hyperparameter Optimization for Convolutional Vision Architectures\n==================================================================\n\nThis package provides a [Theano](http://www.deeplearning.net/software/theano)-based implementation of convolutional networks\nas described in (Bergstra, Yamins, and Cox, 2013), which exposes many\narchitectural hyperparameters for optimization by\n[hyperopt](http://jaberg.github.com/hyperopt).\n\n# Installation\n\n1. Requirements:\n\n\n   * A Python/Numpy/Scipy stack.\n     The Python package requirements are listed in\n     requirements.txt.\n\n   * Optional (but strongly recommended) is an NVIDIA GPU device at least as\n     fast as, say, a GTX280, and CUDA. See Theano's documentation for setting\n     up Theano to use a GPU device.\n\n   * Optional (but strongly recommended) is the MongoDB database software,\n     which allows hyperopt to support parallel optimization.\n\n2. Check out this project\n\n   `git clone https://github.com/jaberg/hyperopt-convnet.git`.\n\n3. Install it as a Python package. This installation makes the code files\n   importable, which is required when running asynchronous hyperparameter\n   optimization (i.e. with hyperopt-mongo-worker, as explained below).\n\n   `python setup.py install`\n\n   Consider installing this within your user account (`--user`) or within a\n   virtualenv to avoid installing this package system-wide, and to avoid\n   needing root privileges.\n\n   Installing hyperopt-convnet will install a pile of Python packages,\n   which are listed in requirements.txt.\n   On my computer, I had to explicitly install a few packages, because\n   whatever the setup.py script was doing wasn't working (I still don't\n           understand python packaging...):\n   * `pip install numpy`,\n   * `pip install scipy`,\n   * `pip install matplotlib`\n\n4. Replace sklearn \u003c 0.13 with git version (we need some new stuff in SVC).\n\n\n# Testing\n\nIf installation goes well, then you will now be able to import the `hpconvnet`\nmodule. The easiest way to test your installation is\n\n\n```bash\nTHEANO_FLAGS=device=gpu shovel lfw.random_driver  --max_n_per_class=20\n```\n\nThis command should not crash, it should (i) download LFW if necessary and\nthen (ii) loop indefinitely doing random search on a tiny subset of the LFW\ntraining data.\n\n\n\n# Running An Experiment in Parallel with MongoDB\n\nRunning hyperparameter optimization on large convolutional networks for data\nsets such as [LFW](http://vis-www.cs.umass.edu/lfw/)\nand [CIFAR10](http://www.cs.toronto.edu/~kriz/cifar.html) takes a significant amount of time:\nexpect a search of a few hundred points to take about a GPU-week.\nThis cannot be completely parallelized (Bayesian optimization works on the\nbasis of feedback about the fitness landscape after all), but in my experience\nit can easily be parallelized 5-fold to 10-fold.\nSo if you have access to a small cluster you can see significant progress in\nan hour or two, and be done in a day.\n\nWhat follows here is a sketch of the unix commands you would need to do to\nmake this happen.\nTo get more of a sense about what's going on, read through\n[hyperopt documentation on using\nmongo](https://github.com/jaberg/hyperopt/wiki/Parallelizing-search).\n\n\n1. Set up a mongodb process for inter-process communication.\n\n   `\n    mongod --dbpath . --port PORT --directoryperdb --fork --journal --logpath log.log --nohttpinterface\n   `\n\n    If this machine is visible to the internet, you should either bind mongod\n    to the local loopback address and connect to the database via an ssh\n    tunnel, or set up mongodb for password-protected access.\n\n2. Start an asynchronous search process, that connects to the mongodb and\n   polls a work queue created there.\n\n   `\n    shovel cifar10.tpe_driver localhost PORT 0.0\n   `\n\n3. Start one or more generic hyperopt worker processes to crank through the\n   trials of the experiment, pointing at the database that's written into the\n   shovel script, in this case:\n\n   `\n    ssh WORKNODE hyperopt-mongo-worker --mongo=localhost:PORT/DBNAME\n   `\n\n   The PORT should match the one used to launch mongodb.\n   The DBNAME should match the one used in shovel/cifar10.py:make_trials,\n   which is \"dbname\" by default.\n\n   If you have a cluster with a queue system (e.g. Torque, PBS, etc.) then use\n   that system to schedule a few hyperopt-mongo-worker processes. When they\n   start, they will connect to the database and reserve an experiment trial.\n   These processes will loop indefinitely by default, dequeueing/reserving trials\n   and storing the results back to the database. They will stop when the\n   search process no longer adds new trials to the database, or when several\n   (4) consecutive trials fail to complete successfully (i.e. your trial\n   evaluation code is faulty and you should either fix it or at least catch the\n   terminating exceptions).\n\n# Rough Guide to the Code\n\n* `shovel/{cifar10,lfw,mnist}.py` driver code for various data sets.\n  When you type `shovel lfw.foo` in bash, it will try to run the `foo` task in\n  the lfw.py file.\n\n* `hpconvnet/lfw.py` describes the search space and the objective function\n  that hyperopt.fmin requires to optimize LFW's view 1 data set.\n\n* `hpconvnet/cifar10.py` describes the search space and the objective function\n  that hyperopt.fmin requires to optimize CIFAR10 validation performance.\n\n* `hpconvnet/slm_visitor_esvc.py` provides a LearningAlgo (skdata-style) derived\n  from `SLM_Visitor` that does classification based on sklearn's SVC binary\n  SVM and a precomputed kernel. This is generally a good choice for data sets\n  without too many examples. The LFW experiments use this class.\n\n* `hpconvnet/slm_visitor_primal.py` has a LearningAlgo (skdata-style) derived\n  from `SLM_Visitor` that does classification based on a primal SVM solver.\n  This is generally a good choice for data sets with larger numbers of\n  examples. The MNIST and CIFAR10 experiments use this class.\n\n* `hpconvnet/slm_visitor.py` provides `SLM_Visitor`,\n  a LearningAlgo (skdata-style) base class\n  with image feature extraction code and several LearningAlgo interface\n  methods.\n\n* `hpconvnet/slm.py` - creates the \"pipeline\" part of the search space, which\n  describes the full set of possibilities for image feature extraction (the\n  full set of convolutional architectures). The `uslm_domain` function\n  returns this search space as a pyll graph.\n  Note also the `call_catching_pipeline_errors` function, which includes\n  `except` clauses for all known errors which may arise in the course of\n  evaluating that pyll graph.\n\n* `hpconvnet/pyll_slm.py` - defines many custom pyll.scope functions which\n  serve to describe the `uslm_domain` search space.\n\nThe basic idea of the code is that the driver code (e.g. in shovel/lfw.py)\ndefines a search space and an objective function for hyperopt.\n\nThe search space is relatively complex, not only in terms of its size (238\nhyperparameters) but also in its internal logic: a \"sample\" from the search\nspace is a dictionary that alongside some some simpler key-value pairs,\ncontains a \"pipeline\" key whose value is itself a pyll graph (seriously, pyll\nhas support for lambda expressions),\nwhich evaluates to a theano function, which can process images.\n\nThe objective function is implemented by e.g. lfw.slm_visitor_lfw which\nallocates a LearningAlgo (an SLM_Visitor_ESVC instance called `visitor`)\nto handle most of the work.\nThe lfw.slm_visitor_lfw routine passes a LearningAlgo\nto the LFW data set's \"protocol\" function, which\nwalks the LearningAlgo through the various steps of an LFW experiment: showing\nit the right data at the right time, asking it to compute various statistics,\nand so on.\nWhen that's all done, lfw.slm_visitor_lfw asks the LearningAlgo to make\na report (`visitor.hyperopt_rval()`) in the form of a dictionary.\nThat dictionary is augmented with what hyperopt needs to see (loss and status\nkeys) and passed back to hyperopt.\n\n\nThere are other files too in the hpconvnet folder, but these ones summarize\nthe logic and control flow.\n\n\n# References\n\n* J. Bergstra, D. Yamins, D. D. Cox (2013).\n  [Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures](http://jmlr.csail.mit.edu/proceedings/papers/v28/bergstra13.pdf),\n  in Proc. ICML2013. -- This paper describes the convolutional architectures\n  implemented in this software package, and the results you should expect from\n  hyperparameter optimization.\n\n* J. Bergstra, R. Bardenet, Y. Bengio, B. Kegl (2011).\n  [Algorithms for Hyper-parameter Optimization](http://books.nips.cc/papers/files/nips24/NIPS2011_1385.pdf)\n  In Proc. NIPS2011. -- This paper introduces the TPE hyperparameter optimization algorithm.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyperopt%2Fhyperopt-convnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhyperopt%2Fhyperopt-convnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhyperopt%2Fhyperopt-convnet/lists"}