{"id":13398337,"url":"https://github.com/HIPS/Kayak","last_synced_at":"2025-03-14T02:31:18.663Z","repository":{"id":66049863,"uuid":"21425991","full_name":"HIPS/Kayak","owner":"HIPS","description":"Kayak is a library for automatic differentiation with applications to deep neural networks.","archived":false,"fork":false,"pushed_at":"2017-08-22T13:45:16.000Z","size":228,"stargazers_count":226,"open_issues_count":9,"forks_count":32,"subscribers_count":37,"default_branch":"master","last_synced_at":"2024-07-31T06:39:05.119Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HIPS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2014-07-02T13:50:59.000Z","updated_at":"2024-07-24T05:26:52.000Z","dependencies_parsed_at":"2023-04-04T15:18:34.840Z","dependency_job_id":null,"html_url":"https://github.com/HIPS/Kayak","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HIPS%2FKayak","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HIPS%2FKayak/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HIPS%2FKayak/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HIPS%2FKayak/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HIPS","download_url":"https://codeload.github.com/HIPS/Kayak/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243511660,"owners_count":20302595,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T19:00:23.135Z","updated_at":"2025-03-14T02:31:18.245Z","avatar_url":"https://github.com/HIPS.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"Don't use this: use [Autograd](http://github.com/hips/autograd) instead!\n=======================================\n\nKayak: Library for Deep Neural Networks\n=======================================\n\nThis is a library that implements some useful modules and provides\nautomatic differentiation utilities for learning deep neural networks.\nIt is similar in spirit to tools like\n[Theano](http://deeplearning.net/software/theano/) and\n[Torch](http://torch.ch/).  The objective of Kayak is to be simple to\nuse and extend, for rapid prototyping in Python.  It is unlikely to be\nfaster than these other tools, although it is competitive and\nsometimes faster in performance when the architectures are highly\ncomplex.  It will certainly not be faster on convolutional\narchitectures for visual object detection and recognition tasks than,\ne.g., [Alex Krizhevsky's CUDA\nConvnet](https://code.google.com/p/cuda-convnet2/) or\n[Caffe](http://caffe.berkeleyvision.org/).  The point of Kayak is to\nbe able to experiment in Python with patterns that look a lot like\nwhat you're already used to with Numpy.  It makes it easy to manage\nbatches of data and compute gradients with backpropagation.\n\nThere are some examples in the 'examples' directory, but the main idea\nlooks like this:\n\n    import kayak\n    import numpy.random as npr\n\n    X = ... your feature matrix ...\n    Y = ... your label matrix ...\n\n    # Create Kayak objects for features and labels.\n    inputs  = kayak.Inputs(X)\n    targets = kayak.Targets(Y)\n\n    # Create Kayak objects first-layer weights and biases.  Initialize\n    # them with random Numpy matrices.\n    weights_1 = kayak.Parameter(npr.randn( input_dims, hidsize_1 ))\n    biases_1  = kayak.Parameter(npr.randn( 1, hidsize_1 ))\n\n    # Create Kayak objects that implement a network layer.  First,\n    # multiply the features by weights and add biases.\n    hiddens_1a = kayak.ElemAdd(kayak.MatMult( inputs, weights_1 ), biases_1)\n\n    # Then, apply a \"relu\" (rectified linear) nonlinearity.\n    # Alternatively, you can apply your own favorite nonlinearity, or\n    # add one for an idea that you want to try out.\n    hiddens_1b = kayak.HardReLU(hiddens_1a)\n\n    # Now, apply a \"dropout\" layer to prevent co-adaptation.  Got a\n    # new idea for dropout?  It's super easy to extend Kayak with it.\n    hiddens_1 = kayak.Dropout(hiddens_1b, drop_prob=0.5)\n\n    # Okay, with that layer constructed, let's make another one the\n    # same way: linear transformation + bias with ReLU and dropout.\n    # First, create the second-layer parameters.\n    weights_2 = kayak.Parameter(npr.randn(hidsize_1, hidsize_2))\n    biases_2  = kayak.Parameter(npr.randn(1, hidsize_2))\n\n    # This time, let's compose all the steps, just to show we can.\n    hiddens_2 = kayak.Dropout( kayak.HardReLU( kayak.ElemAdd( \\\n                    kayak.MatMult( hiddens_1, weights_2), biases_2)), drop_prob=0.5)\n\n    # Make the output layer linear.\n    weights_out = kayak.Parameter(npr.randn(hidsize_2, 1))\n    biases_out  = kayak.Parameter(npr.randn())\n    out         = kayak.ElemAdd( kayak.MatMult( hiddens_2, weights_out), biases_out)\n\n    # Apply a loss function.  In this case, we'll just do squared loss.\n    loss = kayak.MatSum( kayak.L2Loss( out, targets ))\n\n    # Maybe roll in an L1 norm for the first layer and an L2 norm for the others?\n    objective = kayak.ElemAdd(loss,\n                              kayak.L1Norm(weights_1, weight=100.0),\n                              kayak.L2Norm(weights_2, weight=50.0),\n                              kayak.L2Norm(weights_out, weight=3.0))\n\n    # This is the fun part and is the whole point of Kayak.  You can\n    # now get the gradient of anything in terms of anything else.\n    # Probably, if you're doing neural networks, you want the gradient\n    # of the parameters in terms of the overall objective. That way\n    # you can go off and do some kind of optimization.\n    weights_1_grad   = objective.grad(weights_1)\n    biases_1_grad    = objective.grad(biases_1)\n    weights_2_grad   = objective.grad(weights_2)\n    biases_2_grad    = objective.grad(biases_2)\n    weights_out_grad = objective.grad(weights_out)\n    biases_out-grad  = objective.grad(biases_out)\n\n    ... use the gradients for learning ...\n    ... probably this whole thing would be in a loop ...\n    ... in practice you'd probably also use minibatches ...\n\nThis is a work in progress and we welcome contributions. Some\nnosetests are implemented.  We're working on documentation.  Whatever\ndocs come into existence will end up at\n[http://hips.gihub.io/Kayak](http://hips.gihub.io/Kayak).\n\nThis project is primarily develped by the [Harvard Intelligent\nProbabilistic Systems (HIPS)](http://hips.seas.harvard.edu) group in\nthe [Harvard School of Engineering and Applied Sciences\n(SEAS)](http://www.seas.harvard.edu).  The primary developers to date\nhave been Ryan Adams, David Duvenaud, Scott Linderman, Dougal\nMaclaurin, and Jasper Snoek.\n\nKayak is Copyrighted by The President and Fellows of Harvard\nUniversity, and is distributed under an MIT license, which can be\nfound in the license.txt file but is also below:\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHIPS%2FKayak","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHIPS%2FKayak","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHIPS%2FKayak/lists"}