{"id":26936126,"url":"https://github.com/vecnet/vecnet-dl","last_synced_at":"2026-02-12T22:01:53.201Z","repository":{"id":8063249,"uuid":"9475214","full_name":"vecnet/vecnet-dl","owner":"vecnet","description":"VecNet Digital Library","archived":false,"fork":false,"pushed_at":"2016-03-14T14:54:53.000Z","size":15095,"stargazers_count":2,"open_issues_count":1,"forks_count":2,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-02T12:38:58.922Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vecnet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-04-16T15:02:36.000Z","updated_at":"2018-11-23T10:30:37.000Z","dependencies_parsed_at":"2022-08-06T06:16:03.733Z","dependency_job_id":null,"html_url":"https://github.com/vecnet/vecnet-dl","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/vecnet/vecnet-dl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vecnet%2Fvecnet-dl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vecnet%2Fvecnet-dl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vecnet%2Fvecnet-dl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vecnet%2Fvecnet-dl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vecnet","download_url":"https://codeload.github.com/vecnet/vecnet-dl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vecnet%2Fvecnet-dl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29382864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-12T20:34:40.886Z","status":"ssl_error","status_checked_at":"2026-02-12T20:23:00.490Z","response_time":55,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-02T12:31:54.364Z","updated_at":"2026-02-12T22:01:53.183Z","avatar_url":"https://github.com/vecnet.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vecnet Metadata Catalog\n\nThis application provides the [Vecnet Metadata Catalog](http://dl.vecnet.org).\nIt handles the curation and indexing of the data generated by the Vecnet [cyberinfrastructure](https://vecnet.org/).\n\n## Dependencies\n\n * [Fedora Commons](http://fedora-commons.org/) 3.6\n * Solr 4.3\n * Redis (version?)\n * Postgresql or other SQL database\n * nginx\n * [chruby](https://github.com/postmodern/chruby) Ruby version manager\n\nThe [SETUP](SETUP) file has detailed steps on installing the platform on a bare RHEL machine.\n\n## Deployment\n\nFirst, your public ssh key needs to be put on the server. Ask Don to do this.\nTo deploy to QA:\n\n    cap qa deploy\n\nTo deploy to Production:\n\n    cap production deploy\n\nTo deploy from branch\n\n    cap \u003cenvironment\u003e deploy -S branch=\u003cbranch name\u003e\n\nTo deploy new nginx config. This will reload nginx.\n\n    cap \u003cenvironment\u003e vecnet:update_nginx_config\n\n## Other server admin tasks\n\nTo rebuild the Fedora object store:\n\n    sudo service tomcat6 stop\n    cd /opt/fedora/server/bin\n    sudo FEDORA_HOME=/opt/fedora CATALINA_HOME=/usr/share/tomcat6 ./fedora-rebuild.sh\n    # choose option 1 to rebuild the resource index\n    sudo FEDORA_HOME=/opt/fedora CATALINA_HOME=/usr/share/tomcat6 ./fedora-rebuild.sh\n    # choose option 2 to rebuild the SQL database\n    sudo service tomcat6 start\n\nTo resolarize everything...it will take a LONG time to complete.\n\n    chruby 2.0.0-p353\n    RAILS_ENV=qa bundle exec rake solrizer:fedora:solrize_objects\n\nTo load and build the MeSH trees run. This will run for a while (~0.5--1 hours)\n\n    chruby 2.0.0-p353\n    RAILS_ENV=qa bundle exec rake vecnet:import:mesh_subjects vecnet:import:eval_mesh_trees\n\nTo resolrize with mesh synonyms...it will take a LONG time to complete.\n\n    chruby 2.0.0-p353\n    # This builds the synonyms.txt file if needed.\n    # you could skip this if synonyms did not change\n    RAILS_ENV=qa bundle exec rake vecnet:solrize_synonym:get_synonyms FILE=solr_conf/conf/synonyms.txt\n    #copy this file to solr core\n    sudo  cp solr_conf/conf/synonyms.txt /opt/solr-4.3.0/vecnet/conf/synonyms.txt\n    #copy schema and solrconfig\n    sudo  cp solr_conf/conf/schema.xml /opt/solr-4.3.0/vecnet/conf/schema.xml\n    sudo  cp solr_conf/conf/solrconfig.xml /opt/solr-4.3.0/vecnet/conf/solrconfig.xml\n    #change owner to be tomcat\n    sudo chown tomcat:tomcat -R /opt/solr-4.3.0\n    #restart solr\n    sudo service tomcat6 restart\n    #resolrize all objects\n    RAILS_ENV=qa bundle exec rake solrizer:fedora:solrize_objects\n\nTo ingest Citation to qa/Production\n     #Copy endnote file to file to /opt/endnote  and make sure everyone can read\n     sudo cp /from/path/to/endnote/file /opt/endnote\n     sudo chmod -r 755 /opt/endnote\n     #Copy pdf to /opt/citation_file/\u003ccreatefolder_with_endnote_file_name\u003e and make sure everyone can read\n     sudo cp -r /from/path/to/endnote/pdf/* /opt/citation_file/\u003cendnotefilename\u003e\n     sudo chmod -r 755 /opt/citation_file/\u003cendnotefilename\u003e\n     #Execute citation task as app user\n     sudo su app\n     cd /home/app/vecnet/current\n     chruby 2.0.0-p353\n     RAILS_ENV=production bundle exec rake vecnet:import:endnote_conversion ENDNOTE_FILE=/opt/endnote/\u003cendnotefilename\u003e ENDNOTE_PDF_PATH=/opt/citation_files:/opt/citation_files/\u003cendnotefilename\u003e\n\n\nInitializing new production environment\n\n 1. Do system setup as in `SETUP` file\n 2. Get capistrano deploy working to new site\n 3. on production machine:\n  * setup ruby: `chruby 2.0.0-p353`\n  * Setup mesh terms: `RAILS_ENV=production bundle exec rake vecnet:import:mesh_subjects vecnet:import:eval_mesh_trees`\n  * Migrate user table: See below\n  * Resolrize: `RAILS_ENV=production bundle exec rake solrizer:fedora:solrize_objects`\n  * Migrate fedora objects: `RAILS_ENV=production bundle exec rake vecnet:migrate:batch_to_collection`\n 7. Done!\n\n## NCBI Terminalogy\n\nWork in progress. After running `rake db:migrate` the following task will download the NCBI taxonomy from\nthe following location\n\n    ftp://ftp.ncbi.nih.gov/pub/taxonomy/taxdump.tar.gz\n\nand ingest the terms into the database.\n\n    rake vecnet:import:ncbi_taxonomy\n\nThere are about 1,091,096 terms (November 2013).\n\n## Gather repository contents for statistics\n\n    OUTFILE=~/repo-stats-20130916.csv RAILS_ENV=production bundle exec rake vecnet:dump_statistics\n\n## Pubtkt Authentication\n\nThe site uses the pubtkt authentication scheme, which uses a signed cookie for every request.\nFor development, a dummy login to create a pubtkt is provided (class `DevelopmentSessions`).\nBut, first, a public/private key pair needs to be generated and installed.\n\n    rake pubtkt:generate_keys\n    mv pubtkt.pem config/pubtkt-development.pem\n    mv pubtkt-private.pem config/pubtkt-private-development.pem\n\nAnd that should be enough for development.\nThere are also utility rake tasks for creating and verifying tickets:\n\n1. To create a ticket on the comand line:\n\n    $ P_KEY=pubtkt-private.pem P_UID=dbrower P_VALIDUNTIL=3456789012 P_TOKENS='dl_librarian,dl_write' rake pubtkt:create\n    uid=dbrower;validuntil=3456789012;tokens=dl_librarian,dl_write;sig=MCwCFHiaErA+7lHoHxbSUIZaSnmTovIPAhRf4RxtrmArBMD8CBnZaUM/yWI+Cw==\n\nThe valid until date above has the date July 16, 2079 in the Unix epoch, so the ticket should not expire while you are using it.\n2. To validate tickets from the command line:\n\n    $ P_KEY=pubtkt-private.pem P_TICKET='uid=dbrower;validuntil=3456789012;tokens=dl_librarian,dl_write;sig=MCwCFF1/aaSbtrxN9PLrZE1XvLH5SIWQAhRXN8AHevzPMFbMuIIlOwuCLTZDPw==' rake pubtkt:verify\n    Ticket text: uid=dbrower;validuntil=3456789012;tokens=dl_librarian,dl_write\n    Ticket sig : MCwCFF1/aaSbtrxN9PLrZE1XvLH5SIWQAhRXN8AHevzPMFbMuIIlOwuCLTZDPw==\n    Sig Valid? : true\n    Expired?   : true\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvecnet%2Fvecnet-dl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvecnet%2Fvecnet-dl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvecnet%2Fvecnet-dl/lists"}