https://github.com/paradisec-archive/nabu
nabu is a digital media item management system that provides a catalog of audio and video items, metadata for these items, and information about the workflow status of the items.
https://github.com/paradisec-archive/nabu
Last synced: about 1 month ago
JSON representation
nabu is a digital media item management system that provides a catalog of audio and video items, metadata for these items, and information about the workflow status of the items.
- Host: GitHub
- URL: https://github.com/paradisec-archive/nabu
- Owner: paradisec-archive
- License: gpl-3.0
- Created: 2011-07-09T06:16:42.000Z (almost 14 years ago)
- Default Branch: main
- Last Pushed: 2024-07-22T11:37:36.000Z (11 months ago)
- Last Synced: 2024-08-03T04:07:16.029Z (11 months ago)
- Language: Ruby
- Homepage:
- Size: 11.1 MB
- Stars: 17
- Watchers: 14
- Forks: 8
- Open Issues: 34
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- low-resource-languages - nabu - nabu is a digital media item management system that provides a catalog of audio and video items, metadata for these items, and information about the workflow status of the items. www.paradisec.org.au (Software / Utilities)
README
# Nabu Catalog
## Setting up your dev environment
Use direnv to add bin to your path
```bash
PATH_add bin
```
Bring up the environment
```bash
# Build the base image
docker compose build
# Install the gems
nabu_run bundle
# Bring up all the containers
docker compose up
```
This brings up the following containers
* app - the rails app
* search - Solr instance for search (dev + test)
* db - mysql data base (dev + test)
* s3 - s3 mock
You can then easily run all the standard commands by prefixing with ***nabu***
``` bash
nabu_run bundle install
nabu_run bundle exec rake db:prepare
nabu_run bundle exec guard # Test runner
```
## Production
The application is designed to be deployed with containers into an AWS account using CDK
To bootstrap a new account
```bash
# Setup an AWS account and credentials as per your preferred method and set the environment to use it
AWS_PROFILE=nabu
REGION=ap-southeast-2
ACCOUNT=$(aws sts get-caller-identity | jq -r .Account)
cdk bootstrap aws://$ACCOUNT/$REGION
```
If ECR complains about access
```bash
ACCOUNT=$(AWS_PROFILE=nabu-stage aws sts get-caller-identity | jq -r .Account)
AWS_PROFILE=nabu-stage aws ecr get-login-password --region ap-southeast-2 | docker login --username AWS --password-stdin $ACCOUNT.dkr.ecr.ap-southeast-2.amazonaws.com
```
## Deployment
Use CDK to deploy new code via docker as well as any infrastructure changes
``` bash
bin/release stage
bin/release prod
```
If necessary:
``` bash
bin/aws/ecs_rake app deploy:migrate
bin/aws/ecs_rake app searchkick:reindex
```
## Importing a production database into your development environment
``` bash
AWS_PROFILE=nabu-prod bin/aws/db_backup
mysql -h 127.0.0.1 -u root nabu_devel < ../schema.sql
pv ../data.sql | mysql -h 127.0.0.1 -u root nabu_devel
nabu_run bin/rails db:environment:set RAILS_ENV=development
nabu_run bin/rake db:migrate
nabu_run bin/rake searchkick:reindex:all
```
# New Ethnologue data
We use the following source locations
*
*
Run the following rake task to import everything
``` bash
bundle exec rake import:ethnologue
```
This will
* Add new countries and update names
* Update country names
* Add new languages and update names (Only Living languages)
* Add mappings of language to countries
* All existing languages that are retired are marked as such, incl name change.
* Where name changes occurred items in CollectionLanguage, ItemContentLanguage, ItemSubjectLanguage are updated with the replacement language code.
* Where splits happened, a message is printed.
# OAI-PMH
OLAC available at:
*
The feeds that OLAC harvests:
*
* (Archive identification)
*
*
Individual item:
*
RIF-CS available at:
*
use resulting server on an OAI repository explorer:
* (OLAC)
*
* (OAI-PMH validator)
* (OAI-PMH validator)
* (test protocol)
URLs to test:
* [http://localhost:3000/oai/collection?verb=Identify
* [http://localhost:3000/oai/collection?verb=ListMetadataFormats
* [http://localhost:3000/oai/collection?verb=ListSets
* [http://localhost:3000/oai/collection?verb=ListIdentifiers
*
The feed that ANDS harvests:
*
Test at ANDS:
*
Feed for a single collection:
*
To validate our XML output as per OLAC
* Download
* Extract it
```bash
java -cp xercesImpl.jar:xercesSamples.jar sax.Counter -n -np -v -s -f item.xml
```
## Setup Secrets
```bash
aws secretsmanager list-secrets
aws secretsmanager put-secret-value --secret-id ARN --secret-string "{\"site_key\":\"***\", \"secret_key\":\"***\"}"
```
## Upgrades
We should regularly make sure we are running the latest versions of third-party packages
```bash
# Ruby gems
nabu_run bundle outdated
nabu_run bundle update
# node modules
nabu_run yarn upgrade-interactive
# New rails version
rails new nabu --database=mysql --javascript=esbuild --css=sass --skip-action-cable --skip-kamal
```