{"id":21762762,"url":"https://github.com/nulib/meadow","last_synced_at":"2025-04-13T13:41:27.323Z","repository":{"id":35012519,"uuid":"191423461","full_name":"nulib/meadow","owner":"nulib","description":"A Northwestern Libraries repository application","archived":false,"fork":false,"pushed_at":"2025-04-09T18:28:06.000Z","size":64771,"stargazers_count":17,"open_issues_count":9,"forks_count":5,"subscribers_count":8,"default_branch":"deploy/staging","last_synced_at":"2025-04-09T19:36:34.964Z","etag":null,"topics":["elasticsearch","elixir","graphql","react"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nulib.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-06-11T17:56:36.000Z","updated_at":"2025-04-09T18:26:31.000Z","dependencies_parsed_at":"2024-01-04T19:29:15.112Z","dependency_job_id":"ab63adeb-5016-43c1-92c2-37bbe4dd0c8f","html_url":"https://github.com/nulib/meadow","commit_stats":null,"previous_names":[],"tags_count":107,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nulib%2Fmeadow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nulib%2Fmeadow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nulib%2Fmeadow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nulib%2Fmeadow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nulib","download_url":"https://codeload.github.com/nulib/meadow/tar.gz/refs/heads/deploy/staging","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248724307,"owners_count":21151557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","elixir","graphql","react"],"created_at":"2024-11-26T12:13:05.990Z","updated_at":"2025-04-13T13:41:27.316Z","avatar_url":"https://github.com/nulib.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Meadow\n\n[![Build](https://github.com/nulib/meadow/actions/workflows/test.yml/badge.svg)](https://github.com/nulib/meadow/actions/workflows/test.yml)\n[![Coverage Status](https://coveralls.io/repos/github/nulib/meadow/badge.svg)](https://coveralls.io/github/nulib/meadow)\n\u003c!-- [![Dependabot Status](https://api.dependabot.com/badges/status?host=github\u0026repo=nulib/meadow)](https://dependabot.com) --\u003e\n\n## Prerequisites\n\n- [NUL's AWS Cloud Developer Environment](https://github.com/nulib/aws-developer-environment) setup\n\n## Initial startup:\n\n- From the `meadow` project root, `cd app`.\n- Install Elixir dependencies with `mix deps.get`\n- Run `mix meadow.setup`. This creates the Sequins pipeline, S3 buckets, and database.\n- Install Node.js dependencies with `mix assets.install`\n  - `assets.install` looks for all `package-lock.json` files project-wide and runs `npm install` in each directory found, so you don't need to run `npm install` in individual directories.\n- run `sg open all 3001`\n- Start the Phoenix server with `mix phx.server` (or `iex -S mix phx.server` if you want to an interactive shell).\n\nNow you can visit [`https://[YOURENV].dev.rdc.library.northwestern.edu:3001/`](https://[YOURENV].dev.rdc.library.northwestern.edu:3001/) from your browser.\n\n## Stopping the application\n\nYou can stop the Phoenix server with `Ctrl + C` twice\n\n## Clearing and resetting data\n\nIf you need to clear your data and reset the entire development environment, from `meadow/app` run:\n\n```\nmix ecto.reset\nmix meadow.search.clear\nmix meadow.pipeline.purge\nclean-s3 dev -y\n\n...then\nmix deps.get\nmix meadow.setup\nmix phx.server\n```\n\n### Dependencies\n\nYou may need to run `mix deps.get` again if new dependencies have been added\n\nYou may need to run `mix assets.install` if new `node` packages have been added\n\n### Database\n\nIf you need to reset the database you can run `mix ecto.reset` which will drop + create + migrate the database\n\nIf you just want to run the migrations but leave the data intact, you can just do `mix ecto.migrate`\n\nIf you would like to interact directly with the database\n\n### Run the Elixir test suite\n\n#### Start Test Services\n\nIn one terminal:\n```\ncd infrastructure/localstack\ndocker compose up\n```\n\n#### Provision Test Environment\n\nWatch the logs until the services seem to stabilize. Then, in another terminal:\n```\ncd infrastructure/localstack\nterraform init\nterraform apply -auto-approve \\\n  -var-file test.tfvars \\\n  -var localstack_endpoint=https://localhost.localstack.cloud:4566\n```\n\nYou will probably see `Warning: AWS account ID not found for provider`, but this can be safely ignored.\n\n#### Run Tests\n\n```\ncd app\nmix test [test args...]\n```\n\n**Note:** `mix test` can be run repeatedly without re-provisioning as long as the Docker services are running. If you stop the services, you will need to run Terraform again.\n\n### GraphQL API\n\nYou can visit the GraphiQL interface at: [`https://[YOURENV].dev.rdc.library.northwestern.edu:3001/api/graphiql`](https:/[YOURENV].dev.rdc.library.northwestern.edu:3001/api/graphiql)\n\n### Livebook Integration\n\nTo start meadow with superuser Livebook integration, run: `MEADOW_ROOT/bin/meadow-livebook [iex arguments]`\n\nFor example, from Meadow's root directory: `./bin/meadow-livebook phx.server`\n\n### Opensearch Dashboard\n\n- To start: `es-proxy start`\n- To stop: `es-proxy stop`\n- See the console output for the url to the dashboard\n\n### Digital Collections API\n\nIn order to see data and thumbnails from your current environment, you'll need to run the DC API alongside Meadow. Follow the instructions for [Running the API locally](https://github.com/nulib/dc-api-v2#running-the-api-locally) and [Running the API locally via our AWS dev domain](Running the API locally via our AWS dev domain) to get it running.\n\n### Reindexing data\n\nTo force an Elasticsearch re-index, and not wait for the 2-minute cycle to kick in when updating a Meadow item:\n\nRun the interactive shell in a terminal tab\n\n```\niex -S mix\n```\n\nAnd force a re-index:\n\n```\nMeadow.Data.Indexer.reindex_all()\n```\n\n### Doing development on the Meadow Pipeline lambdas\n\nIn the AWS developer environment, the lambdas associated with the pipeline are shared amongst developers. In order to do development and see whether it's working you can override the configuration to use your local files instead of the deployed lambdas. Example below (you don't have to override them all. Just the ones you need).\n\nEdit `config/dev.local.exs` to get the lambdas to use the local copy through the port:`\n\n```elixir\n  config :meadow, :lambda,\n    digester: {:local, {Path.expand(\"../lambdas/digester/index.js\"), \"handler\"}},\n    exif: {:local, {Path.expand(\"../lambdas/exif/index.js\"), \"handler\"}},\n    frame_extractor: {:local, {Path.expand(\"../lambdas/frame-extractor/index.js\"), \"handler\"}},\n    mediainfo: {:local, {Path.expand(\"../lambdas/mediainfo/index.js\"), \"handler\"}},\n    mime_type: {:local, {Path.expand(\"../lambdas/mime-type/index.js\"), \"handler\"}},\n    tiff: {:local, {Path.expand(\"../lambdas/pyramid-tiff/index.js\"), \"handler\"}}\n```\n\n### TypeScript/GraphQL Types\n\nMeadow now supports TypeScript and GraphQL types in the React app. To generate types, run the following commands:\n\n```bash\n# Generate a local JSON version of GraphQL schema\nmix graphql.schema.export -o priv/graphql/schema.json\n\n# Generate TypeScript types for the UI\ncd assets\nnpm run generate-types\n```\n\nTypes will be generated in `meadow/app/assets/js/__generated__`. You can import them into React components like so:\n\n```tsx\nimport type { FileSet, Work as WorkType } from \"@js/__generated__/graphql\";\n\nconst SomeComponent = ({ work }: { work: WorkType }) =\u003e {\n  // ...\n};\n```\n\n### Terraform\n\nMeadow's Terraform code is stored in this repo. To run Terraform commands, you'll need to do the [configuration setup](https://github.com/nulib/repodev_planning_and_docs/blob/a36472895ae5c851f4f36b6f598dc5f666cea672/docs/2._Developer_Guides/Meadow/Terraform-Setup-on-Meadow.md)\n\n### UI Customization\n\nMeadow runs in Development, Staging and Production environments. To help distinguish environments (and avoid potential errors), Staging and Development environments support alternate, background colors.\n\n#### Production\n\n- A wrapper CSS class of `is-production-environment` wraps the `main` HTML element (in case anyone wants to target a selector for any reason).\n\n#### Staging\n\n- Supports a toggle background color switch in the site header\n- Customize your own dev background color by updating the hex value for `localStorage` property `devBg`\n- A wrapper CSS class of `is-staging-environment` wraps the `main` HTML element.\n\n#### Development\n\n- Supports a toggle background color switch in the site header\n- Customize your own dev background color by updating the hex value for `localStorage` property `devBg`\n- A wrapper CSS class of `is-development-environment` wraps the `main` HTML element.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnulib%2Fmeadow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnulib%2Fmeadow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnulib%2Fmeadow/lists"}