{"id":20714715,"url":"https://github.com/justingosses/nasa-repos-visual-explorer","last_synced_at":"2025-07-13T17:32:41.901Z","repository":{"id":91734476,"uuid":"432060129","full_name":"JustinGOSSES/nasa-repos-visual-explorer","owner":"JustinGOSSES","description":"Exploration of NASA code  across multiple orgs on github.com using the awesome-list-visual-explorer-template","archived":false,"fork":false,"pushed_at":"2022-01-23T04:51:05.000Z","size":5421,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-11T06:50:21.404Z","etag":null,"topics":["github-api","metadata","metadata-extraction","nasa","nasa-code","software-portal","visual-explorer"],"latest_commit_sha":null,"homepage":"https://justingosses.github.io/nasa-repos-visual-explorer/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JustinGOSSES.png","metadata":{"files":{"readme":"README.md","changelog":"changes_needed.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-26T05:06:11.000Z","updated_at":"2025-03-02T05:43:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"706fe70c-9297-4dca-9555-719f56d1cf97","html_url":"https://github.com/JustinGOSSES/nasa-repos-visual-explorer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"JustinGOSSES/awesome-list-visual-explorer-template","purl":"pkg:github/JustinGOSSES/nasa-repos-visual-explorer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustinGOSSES%2Fnasa-repos-visual-explorer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustinGOSSES%2Fnasa-repos-visual-explorer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustinGOSSES%2Fnasa-repos-visual-explorer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustinGOSSES%2Fnasa-repos-visual-explorer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JustinGOSSES","download_url":"https://codeload.github.com/JustinGOSSES/nasa-repos-visual-explorer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JustinGOSSES%2Fnasa-repos-visual-explorer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265178705,"owners_count":23723311,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["github-api","metadata","metadata-extraction","nasa","nasa-code","software-portal","visual-explorer"],"created_at":"2024-11-17T02:33:46.560Z","updated_at":"2025-07-13T17:32:41.822Z","avatar_url":"https://github.com/JustinGOSSES.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# awsome-list-visual-explorer-template\n\n## Is this the awsome-list-visual-explorer-template TEMPLATE or a repository made from the template?\n\u003ci\u003eNOTE: You may be looking at a repository made with the template https://github.com/JustinGOSSES/awsome-list-visual-explorer-template or you may be looking at the actual code repository https://github.com/JustinGOSSES/awsome-list-visual-explorer-template. If the name of the code repository is not \u003cb\u003eawsome-list-visual-explorer-template\u003c/b\u003e, you are looking at a code repository made with the template.\u003c/i\u003e\n\n## Description of this code repository if not the template:\n- Repository Name: nasa-repos-visual-explorer\n- Repository Authors: Justin Gosses\n- Purpose: Visualize trends and patterns in NASA code.\n- Link To Awesome List Parsed In This Code Project: code.nasa.gov/code.json (not an awesome list but a JSON following code.gov schema.\n- Link to Live Website: \u003cb\u003e\u003ca href=\"https://justingosses.github.io/nasa-repos-visual-explorer/\"\u003ehttps://justingosses.github.io/nasa-repos-visual-explorer/\u003c/a\u003e\u003c/b\u003e\n- Last Ran in Full On: YYYY-MM-DD\n\n### Contributing to this repository if not the template\n------------ INFORMATION GOES HERE ------------\n\n\u003ci\u003e----- REST OF INFORMATION ABOUT THIS SPECIFIC CODE REPOSITORY IF NOT THE TEMPLATE ITSELF GOES HERE ------\u003c/i\u003e\n\n## What Does the awsome-list-visual-explorer-template Template Build?\n\n\u003cb\u003e\u003ci\u003eThe goal of this project is to surface the characteristics, trends, connections, relationships, etc. that can describe the implicit community of developers and repositories in an Awesome List and make them available in a visual form to the same people who get value from an Awesome List in list form.\u003c/i\u003e\u003c/b\u003e\n\nThe template builds a website with a variety of visualizations, usually deployed as a GitHub pages page.\n\nThe idea behind its creation is by being able to quickly understand visually the trends and relationships in a community of related code repositories, it will nudge developers who are a part of that implicit community how think about those projects, where they might contribute, and who is working on the things they are also interested in. All this information has technically long been available, but it is usually slow and difficult to surface so most people don't bother. \n\n## Key Pages That The Template Builds\n##### Front Page\nThis is a catalog of all the code repositories. The organizational structure is based on topics tags on the github repositories themselves and categories that organize those tags set in `category/category_info.json`. \n##### About Page\nThis is an about page for the template itself. \n##### Explore Page\nA series of visualizations that give a high level overview of how the community of Awesome List GitHub code repositories has changed over time, including:\n- groupings of organizations that contribute many repos\n- contributions over time\n- stars over time\n- ratios of open and closed issues and pull requests\n- repository topic word map\n- repository license breakdown\n\n##### Dependencies Page\nAn interactive graph network of dependencies and organizational connections between code repositories.\n\n##### Popular Repositories Page\nVisualization of the most popular repositories including:\n- organizations as bubbles scaled by the number of repositories they've created\n- line chart of count of repositories created over time\n- number of stars over time of the top 10 most popular repositories\n- activity over time of the top 10 most popular repositories\n- licenses of the most popular repositories\n\n## Examples Webpages Exploring An Awesome List Built With This Template\n\n- https://JustinGOSSES.github.io/awsome-list-visual-explorer-template/\n- https://softwareunderground.github.io/open_geosciene_code_projects_viz/\n- https://github.com/JustinGOSSES/Awesome-Earth-Artificial-Intelligence-visual-explorer\n\n### Prior Work\n\u003ci\u003eWhat is it based off of?\u003c/i\u003e\nThis site was created by taking a fork of the \u003ca href=\"https://github.com/LLNL/llnl.github.io\"\u003eLawrence Livermore National Laboratory's open source software catalog\u003c/a\u003e and changing \u003ca href=\"https://github.com/softwareunderground/open_geosciene_code_projects_viz\"\u003ea bunch of stuff\u003c/a\u003e to make it useful for visualizing Software Underground's Awesome-Open-Geoscience list.\n\nIt was initially created as part of the Transform 21' hackathon put on by The Software Underground or SWUNG.\n\nOriginal Project Plan: https://github.com/softwareunderground/transform-2021-hackathon/discussions/14\n\nThe project plan has now been moved to issues: https://github.com/softwareunderground/open_geosciene_code_projects_viz/projects/1\n\nThe code is still largely that of the original  \u003ca href=\"https://github.com/LLNL/llnl.github.io\"\u003eLawrence Livermore National Laboratory's open source software catalog\u003c/a\u003e.\n\n### Completed Changes From Original Project \u0026 Possible Future Changes Roadmap \nhttps://github.com/softwareunderground/open_geosciene_code_projects_viz/blob/main/changes_needed.md\n\n### Presentations on What's trying to be accomplished with this repository\n\nSLIDES PRESENTED DURING HACKATHON: https://observablehq.com/@justingosses/more-visible-connections-between-projects-can-nudge-devel \n\nMEDIUM STORY: https://justingosses.medium.com/beyond-awesome-lists-3ccb074f7859\n\n\n## Overview of How awsome-list-visual-explorer-template Template Works\nAt a very high level, there a variety of bash and Python scripts that grab github repository URLs from an Awesome List README location you supply as configuration, get details about those repositories from the GitHub API, and then rebuild the webpages with that information and information from a configuration file. \n\n#### A step-by-step high level overview of how the template is used:\n- Developer clicks on green \"Use This Template\" button on the \u003ca href=\"https://github.com/JustinGOSSES/awsome-list-visual-explorer-template\"\u003eawsome-list-visual-explorer-template repository page\u003c/a\u003e. This builds them a clone repository, not a fork!. \n- They change information in the _config.yml file that sits at the top of the repository directory to reflect their name of the repositry, the location of the Awesome-list they want to build from, and other details. \n- They install ruby and jekyl following the full \"installation\" instructions below. \n- They change directory into `_explore/scripts` and follow the README there to install the python dependencies in an virtualenv. \n- They run the bash script in the `_explore/scripts` directory called `grabNewRepos.sh`. This grabs github URLs from the README whose address was added to the `_config.yml` file and puts them in the `_explore/input_lists.json` file. \n- They run the bash script in the `_explore/scripts` directory called `BUILD.sh`. This is the main build script for the repository and runs a bunch of python files in the scripts folder and also calls the GitHub API to get information like stars and contributors from each GitHub code repository listed in `input_lists.json`. It also replaces the name of the repository used in the template with the name of the new repository listed in `_config.yml`.\n- Lastly, they will run `bundle exec jekyll serve` to start up a server that will show a local version of the webpage at  http://127.0.0.1:4000/nameOfYourRepositoryThatWasSetInConfigYamlFile.\n\n\n## Prerequisites\n\nBefore you begin, make sure you have working installs of Git, Ruby, and Bundler \u003chttps://bundler.io/\u003e You will need these tools for development.\n\n## Getting Started\n\nFirst, use the template repository as your template. Got to https://github.com/JustinGOSSES/awsome-list-visual-explorer-template/ and click the big green button that say \"Use This Template\". \n\nYou'll probably want to give it a name that somewhat aligns with the Awesome List repository that you'll use for your data.\n\nNext, to work locally, clone your repository:\n\n```\ngit clone https://github.com/JustinGOSSES/awsome-list-visual-explorer-template/   \u003c--- or your name of the repository!\n```\n\nMake sure you are in the directory you just created by running `cd nameOfRepositoryHere` Then you can use `bundler` to install the Ruby dependencies (see the [Jekyll installation docs](https://jekyllrb.com/docs/installation/) for step-by-step guides to setting this up):\n\n```\nbundle install\n```\n\nRunning this will install everything in your Gemfile (including Jekyll). \n\nAfter all the data is updated and the pages are built fresh, you'll want to the development web server with:\n\n```\nbundle exec jekyll serve\n```\n\nFollowed by opening \u003chttp://127.0.0.1:4000/open_geosciene_code_projects_viz/\u003e in a web browser.\n\nYou probably want to visualize details of an Awesome list specific to you though, so we'll get to that now. \n\nGo to the `_config.yml` file and change some of the details. The important ones to change are:\n- name\n- authour\n- title\n- description\n- shortname\n- baseurl \u003c=== This should be your repository name for GitHhub Pages deployment to work!!!\n- raw_link_to_awesome_list_readme_to_parse\n- filename_to_save_awesome_list_readme\n- twitter.username\n- repository \u003c=== This should be your username/repositoryName!!!\n- replaced_all_instances_of_string_above_in_config_with \u003c== This should be the name of your repository again. Where awsome-list-visual-explorer-template is found across HTML, JavaScript, and CSS files, it will be replaed with this string.\n\nThe python scripts call the GitHub API, which means the environment you run your code in requires a GitHub API Token as an environmental variable. The token only needs READ access to public repositories. You can read about how to get a GitHub API token \u003ca href=\"https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token\"\u003ehere\u003c/a\u003e and \u003ca href=\"https://docs.github.com/en/rest/guides/getting-started-with-the-rest-api\"\u003ehere\u003c/a\u003e. If running locally in a terminal, you probably want to get it into your environment by running something like `export GITHUB_API_TOKEN digitsOfYourTokenGoHere`. If running in the GitHubActions, the `.github/workflows/update.yml` file already has you covered. \n\nNext, we'll work with the scripts that pull in data and rebuild the pages. \n\nChange directory to `_explore/scripts` and read the README there for instructions on how to start a virtual environment and install the dependencies in requirements.txt. \n\nOnce you've done that, you can proceed to run the first bash script to gather GitHub code repository URLs from the Awesome List URL you added to `_config.yml`. This done by running `bash grabNewRepos.sh`.\n\nNext step is to run `bash BUILD.sh`. This is the main build script for the repository and runs a bunch of python files in the scripts folder and also calls the GitHub API to get information like stars and contributors from each GitHub code repository listed in `input_lists.json`. It also replaces the name of the repository used in the template with the name of the new repository listed in `_config.yml`.\n\n\u003ci\u003eNOTE: the build.sh script does many calls to the GitHub API, which means it can sometimes take 15-30 minutes to complete its full run!\u003c/i\u003e\n\nLastly, change directory back to the root of the directory by running `cd ../../` and then run `bundle exec jekyll serve` to start up a server that will show a local version of the webpage at  http://127.0.0.1:4000/nameOfYourRepositoryThatWasSetInConfigYamlFile.\n\n#### GithubActions \nThe steps to gather data from the Awesome List defined in the _config.yml file and rebuild the page is also built into a GitHubActions configuration file at `.github/workflows/update.yml`. \n\nCurrently, this is set to only run on manual trigger or what's called workflow dispatch. Once changes are stabled, the GitHubActions scripts could be used to pull in new data from the Awesome list on some predetermined timing or upon pull request to keep the page visualizations up to date with the Awesome List it pulls from. \n\n#### Tips\n\nThe gems in your sourcefile get updated frequently. It is a good idea to occasionally run `bundle update` from within your project's root directory to make sure the software on your computer is up to date.\n\nSometimes there can be dependency conflicts if your local version of Ruby is different from this repo or github pages deployment settings. You can find the version number of each of GitHub Page's current dependency's [here](https://pages.github.com/versions/). You can often avoid dependency issues if you use the same versions, including for Ruby. \n\nAs an example, the default version of Ruby used to deploy GitHub Pages on github.com as of 2021-04-08 was Ruby\t2.7.1. If you tried running Ruby version 3.0.0 locally on macOS, you'll need to do some extra steps to correctly install the dependencies for this repository. You'd need to run `bundle add webrick` as it is no longer a prepackaged dependency with Ruby in 3.0.0. You may also need to run `gem install eventmachine -- --with-openssl-dir=/usr/local/opt/openssl@1.1` as MacOS \u003e10.14 doesn't use openssl from the same path as is still assumed to be in by eventmachine.\n\n#### How to get additional code repositories tracked not in the awesome list in the _config.yml file?\n------------ IN PROGESS ------------\n\n\n## Contributing to the Template\n------------ IN PROGESS ------------\n\n\n## How to get changes from the template after your repository is already built?\n------------ IN PROGESS ------------\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustingosses%2Fnasa-repos-visual-explorer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjustingosses%2Fnasa-repos-visual-explorer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjustingosses%2Fnasa-repos-visual-explorer/lists"}