{"id":13880177,"url":"https://github.com/taskrabbit/waistband","last_synced_at":"2025-04-05T15:05:30.094Z","repository":{"id":9485714,"uuid":"11374557","full_name":"taskrabbit/waistband","owner":"taskrabbit","description":"Ruby tools for Elastic Search","archived":false,"fork":false,"pushed_at":"2025-03-07T18:11:32.000Z","size":229,"stargazers_count":29,"open_issues_count":6,"forks_count":6,"subscribers_count":40,"default_branch":"master","last_synced_at":"2025-03-29T14:06:27.880Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/taskrabbit.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-07-12T17:48:24.000Z","updated_at":"2025-03-07T18:10:12.000Z","dependencies_parsed_at":"2024-11-16T11:04:47.103Z","dependency_job_id":null,"html_url":"https://github.com/taskrabbit/waistband","commit_stats":{"total_commits":140,"total_committers":7,"mean_commits":20.0,"dds":"0.12142857142857144","last_synced_commit":"dd607780c3081389b51abc949b4df3170d938c6e"},"previous_names":[],"tags_count":69,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskrabbit%2Fwaistband","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskrabbit%2Fwaistband/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskrabbit%2Fwaistband/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/taskrabbit%2Fwaistband/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/taskrabbit","download_url":"https://codeload.github.com/taskrabbit/waistband/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247353731,"owners_count":20925329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-06T08:02:50.243Z","updated_at":"2025-04-05T15:05:30.076Z","avatar_url":"https://github.com/taskrabbit.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# Waistband\n\n[![Build Status](https://travis-ci.org/taskrabbit/waistband.png?branch=master)](https://travis-ci.org/taskrabbit/waistband)\n\nConfiguration and sensible defaults for ElasticSearch on Ruby.  Handles configuration, index creation, quality of life, etc, of Elastic Search in Ruby.\n\nWaistband doens't handle connections or API requests, it merely acts as a translator between commonly used patterns and the underlying elasticsearch gems.\n\n# Installation\n\nInstall ElasticSearch:\n\n```bash\nbrew install elasticsearch\n```\n\nAdd this line to your application's Gemfile:\n\n    gem 'waistband'\n\nAnd then execute:\n\n    $ bundle\n\nOr install it yourself as:\n\n$ gem install waistband\n\n## Compatibility\nWaistband relies on [elasticsearch-ruby](https://github.com/elastic/elasticsearch-ruby), which\nmirrors its versioning with Elasticsearch.\n\n   | Gem Version   |   | Elasticsearch |\n   |:-------------:|:-:| :-----------: |\n   | 0.x           | → | 1.x           |\n   | 6.x           | → | 6.x           |\n   | master        | → | 6.x           |\n\n## Configuration\n\nConfiguration is generally pretty simple.  First, create a folder where you'll store your Waistband configuration docs, usually under `#{APP_DIR}/config/waistband/`, you can also just throw it under `#{APP_DIR}/config/` if you want.  The baseline config contains something like this:\n\n```yml\n# #{APP_DIR}/config/waistband/waistband.yml\ndevelopment:\n    retries: 5\n    timeout: 2\n    reload_on_failure: true\n    servers:\n        server1:\n            protocol: http\n            host: localhost\n            port: 9200\n```\n\nYou can name the servers whatever you want.  The connection, retries, timeouts, etc are handled by the `elasticsearch-transport` gem.\n\nHere's an example with two servers:\n\n```yml\n# #{APP_DIR}/config/waistband/waistband.yml\ndevelopment:\n    retries: 5\n    timeout: 2\n    reload_on_failure: true\n    servers:\n        server1:\n            protocol: http\n            host: 173.247.192.214\n            port: 9200\n        server2:\n            protocol: http\n            host: 173.247.192.215\n            port: 9200\n```\n\nYou'll need a separate config file for each index you use, containing the index settings and mappings.  For example, for my search index, I use something akin to this:\n\n```yml\n# #{APP_DIR}/config/waistband/waistband_search.yml\ndevelopment:\n    stringify: false\n    settings:\n        index:\n            number_of_shards: 4\n    mappings:\n        event:\n            _source:\n                includes: [\"*\"]\n```\n\nNote that you can configure specific connection settings for a particular index, if for some reason or another that index in particular uses a different set of servers:\n\n```yml\ndevelopment:\n  connection:\n    timeout: 2\n    retries: 5\n    reload_on_failure: true\n    servers:\n      server1:\n        host: 192.168.31.112\n        port: 9200\n        protocol: http\n      server2:\n        host: 192.168.31.113\n        port: 9200\n        protocol: http\n  settings:\n    index:\n      number_of_shards: 1\n      number_of_replicas: 1\n  mappings:\n    event:\n      _source:\n        includes: [\"*\"]\n```\n\n## List of config settings:\n\n* `settings`: settings for the Elastic Search index.  Refer to the [\"admin indices update settings\"](http://www.elasticsearch.org/guide/reference/api/admin-indices-update-settings/) document for more info.\n* `mappings`: the index mappings.  More often than not you'll want to include all of the document attribute, so you'll do something like in the example above.  For more info, refer to the [mapping reference](\"http://www.elasticsearch.org/guide/reference/mapping/\").\n* `retries`: number of times to retry before moving on to the next server node.\n* `reload_on_failure`: should we reload the node list from the server on failure.\n* `timeout`: seconds till a timeout exception is raise when trying to connect to the node.\n* `name`: optional - name of the index.  You can (and probably should) have a different name for the index for your test environment.  If not specified, it defaults to the name of the yml file minus the `waistband_` portion, so in the above example, the index name would become `search_#{env}`, where env is your environment variable as defined in `Waistband::Configuration#setup` (determined by `RAILS_ENV` or `RACK_ENV`).\n* `stringify`: optional - determines wether whatever is stored into the index is going to be converted to a string before storage.  Usually false unless you need it to be true for specific cases, like if for some `key =\u003e value` pairs the value is of different types some times.\n* `connection`: optional - determines which server/s the index uses.  If not present, it'll use the default connection settings specified in `config/waistband.yml`.\n* `permissions`: optional - determines which permissions to allow on this index, please refer to the [permissions section](#permissions) for more information.\n\n## Initializer\n\n\nWaistband will look for config files by default in `File.join(Rails.root, 'config')`.  You can override the default location of the config folder. After getting all the YML config files in place, you'll just need to hook up an initializer to these files:\n\n```ruby\n# #{APP_DIR}/config/initializers/waistband.rb\nWaistband.configure do |c|\n    c.config_dir = \"#{APP_DIR}/spec/config/waistband\"\nend\n```\n\n## Usage\n\n### Indexes\n\n\n#### Creating and destroying the indexes\n\nFor each index you have, you'll probably want to make sure it's created on initialization, or in a Rake task, so either in the same waistband initializer or in another initializer, depending on your preferences, you'll have to create them.  For our search example:\n\n```ruby\n# #{APP_DIR}/config/initializers/waistband.rb\n# ...\nWaistband::Index.new('search').create\n```\n\nThis will create the index if it's not been created already or return nil if it already exists.  If you want to raise an exception if it already exists, use the `#create!` method.\n\nDestroying an index is equally easy:\n\n```ruby\nWaistband::Index.new('search').delete\n```\n\nWhen writing tests, it would generally be advisable to destroy and create the indexes in a `before(:each)` or `before(:all)` depending in your circumstances.  Also, remember for testing that replication and data availability is not inmediate on the indexes, so if you create an immediate expectation for data to be there, you should refresh the index before it:\n\n```ruby\nWaistband::Index.new('search').refresh\n```\n\nNote: most index methods such as `create`, `delete`, etc, have an equivalent bang method (`delete!`) that will actually throw an exception if something goes wrong.  For example, `delete` will return nil if the index doesn't exist, but will raise any other unrelated exceptions, whereas `delete!` will raise even the Index Not Found exception.\n\n#### Writing, reading and deleting from an index\n\n```ruby\nindex = Waistband::Index.new('search')\n\n# writing\nindex.save('my_data', {'important' =\u003e true, 'valuable' =\u003e {'always' =\u003e true}}) # =\u003e true\n\n# reading\nindex.find('my_data') # =\u003e {\"important\"=\u003etrue, \"valuable\"=\u003e{\"always\"=\u003etrue}}\n\n# reading with all the internal data\nindex.read('my_data') # =\u003e {'_id' =\u003e '123123', '_source' =\u003e {\"important\"=\u003etrue, \"valuable\"=\u003e{\"always\"=\u003etrue}}, ...}\n\n# partial updating\nindex.save('my_data', {'important' =\u003e true, 'valuable' =\u003e {'always' =\u003e true}}) # =\u003e true\nindex.update('my_data', {'important' =\u003e false }) # =\u003e true\nindex.read('my_data') # =\u003e {'_id' =\u003e '123123', '_source' =\u003e {\"important\"=\u003efalse, \"valuable\"=\u003e{\"always\"=\u003etrue}}, ...}\n\n# deleting\nindex.destroy('my_data') # =\u003e \"{\\\"ok\\\":true,\\\"found\\\":true,\\\"_index\\\":\\\"search\\\",\\\"_type\\\":\\\"search\\\",\\\"_id\\\":\\\"my_data\\\",\\\"_version\\\":2}\"\n\n# reading non-existent data\nindex.find('my_data') # =\u003e nil\n```\n\n### Searching\n\nFor searching, you construct a search from your index:\n\n```ruby\nindex = Waistband::Index.new('search')\nresults = index.search({\n    query: {\n        term: { hidden: false }\n    },\n    sort: { created_at: {order: 'desc' } },\n    page: 1,\n    page_size: 5\n})\n\nresults.hits # =\u003e returns a search results object\nresults.total_hits # =\u003e 28481\n```\n\nFor paginating the results, you can use the `#paginated_results` method, which will provide an array object compatible with the kaminari gem.  If you use another gem, you can just override the method, etc.\n\nFor more information and extra methods, take a peek into the class docs.\n\nAlso, for convenience, the gem provides the `Result` class, which just provides some quality-of-life methods for working with search result hashes or their inner `_source` hashes, for example:\n\n```ruby\nsearch = index.search({\n    query: {\n        term: { hidden: false }\n    }\n})\nresults = search.results\nresult = result.first\n\nresult._id # =\u003e '123123'\nresult._type # =\u003e 'search_result'\nresult._index # =\u003e 'search'\nresult.task_id # =\u003e 991122 -- note that this is a method missing interface directly either to the search result hash, or to the _source sub-hash\n```\n\nThe `Result` class is directly exposed via two methods in the `SearchResults` class: `#results` and `#paginated_results`.  You can use `#paginated_results` if you're using Kaminari for pagination and wish to use the awesomeness it provides.\n\n### Index Aliasing\n\nSometimes it can be useful to sub-divide your index into smaller indexes based on dates or other partitioning schemes.  To do this, the `Index` class exposes the `subs` option on instantiation:\n\n```ruby\nindex = Waistband::Index.new('events', subs: %w(2013 01))\nindex.create!\n```\n\nThis creates the index `events__2013_01`, which in your application logic you could design to store all event data for Jan 2013.  You'd do the same for Feb, etc., and when you no longer need one of the older ones, you could delete just that sub-index, instead of things getting more complicated.\n\nWe've also found quite a bit of usefulness in using index versioning, so you can add/remove fields to your object without much worry.  Waistband accomodates this pattern as follows:\n\n```ruby\nindex = Waistband::Index.new('events', version: 1)\nindex.create!\n```\n\n### Aliasing\n\nPart of subbing is gonna be creating the correct aliases that group up your sub-indexes.\n\n```ruby\nindex = Waistband::Index.new('events', subs: %w(2013 01))\nindex.create!\nindex.alias('my_super_events_alias') # =\u003e true\nindex.alias_exists?('my_super_events_alias') # =\u003e true\n```\n\nThe `alias` methods receives a param to define the alias name.  The same pattern can be used when using index versions.\n\n### Permissions\n\nWe've found it safer to tighten up the permissions to determine which actions can be done on each index based on environments.  For example, you might want to allow an index to be created and deleted at will on the development or staging environments, but you probably don't want to allow it to be deleted on production.  To that effect, you're able to set permissions on each index's config file:\n\n```yml\nproduction:\n    permissions:\n        create: true\n        delete_index: false\n        destroy: true\n        read: true\n        write: true\n```\n\nBy default, all permissions are true unless set otherwise.\n\nThe specific permissions are:\n\n* `create`: can Waistband create the index?\n* `delete_index`: can Waistband delete the entire index?\n* `destroy`: can Waistband destroy an object in the index?\n* `read`: can Waistband `read`, `find`, or `find_result` in the index?\n* `write`: can Waistband `save` (create or update) an object to the index?\n\n### Logging\n\nIt's very useful to log what the output from all operations is, with that in mind, you can specify a log for Waistband easily:\n\n```ruby\nWaistband.configure do |c|\n  c.logger = Rails.logger.dup\nend\n```\n\nYou can do this in an initializer, even the same initializer you used to specify the `config_dir` ealirer if you did so.\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaskrabbit%2Fwaistband","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftaskrabbit%2Fwaistband","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftaskrabbit%2Fwaistband/lists"}