{"id":16172012,"url":"https://github.com/magicmatatjahu/milv","last_synced_at":"2025-07-29T00:05:41.436Z","repository":{"id":57516860,"uuid":"140680012","full_name":"magicmatatjahu/milv","owner":"magicmatatjahu","description":"Markdown internal \u0026 external links validation library and CLI","archived":false,"fork":false,"pushed_at":"2019-07-16T16:58:18.000Z","size":492,"stargazers_count":11,"open_issues_count":8,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-02T20:38:29.859Z","etag":null,"topics":["command-line","golang","library","markdown","markdown-parser"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/magicmatatjahu.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null}},"created_at":"2018-07-12T07:48:48.000Z","updated_at":"2022-10-15T23:00:37.000Z","dependencies_parsed_at":"2022-09-15T21:31:44.362Z","dependency_job_id":null,"html_url":"https://github.com/magicmatatjahu/milv","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/magicmatatjahu/milv","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicmatatjahu%2Fmilv","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicmatatjahu%2Fmilv/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicmatatjahu%2Fmilv/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicmatatjahu%2Fmilv/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/magicmatatjahu","download_url":"https://codeload.github.com/magicmatatjahu/milv/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/magicmatatjahu%2Fmilv/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267605947,"owners_count":24114619,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line","golang","library","markdown","markdown-parser"],"created_at":"2024-10-10T03:44:37.460Z","updated_at":"2025-07-29T00:05:41.417Z","avatar_url":"https://github.com/magicmatatjahu.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Markdown internal \u0026 external links validation\n\n`MILV` is a bot that parses, checks and validates internal \u0026 external URLs links in markdown files. It can be used for [verification pull requests](#validate-pull-requests) and also as standalone library.\n\n## Installation\n\n```bash\n$ go get -u -v github.com/magicmatatjahu/milv\n```\n\nFor the above command to work you must have [GoLang](https://golang.org/doc/install) installed\n\nAfter installation, run the program with `milv` from anywhere in the file system.\n\n### Run the code from source\n\nIf you want run the code without installation, run the following commands to get the source code, resolve external dependencies and build the project. \n\nFor this operations you must have also installed package manager [Dep](https://github.com/golang/dep).\n\n```bash\ngit clone https://github.com/magicmatatjahu/milv.git\ncd milv\ndep ensure\ngo build\n```\n\n## Usage\n\n### Command line parameters\n\nYou can use the following parameters while using `milv` binary:\n\n| Name        | Description    | Default Value       |\n|-------------|----------------|---------------------|\n| `-base-path` | root directory of repository | `\"\"`\n| `-config-file` | configuration file for bot. See [more](#config-file) | `milv.config.yaml`\n| `-white-list-ext` | comma separate external links which will not be checked | `[]`\n| `-white-list-int` | comma separate internal links which will not be checked  | `[]`\n| `-black-list` | comma separate files which will not be checked | `[]`\n| `-allow-redirect` | redirects will be allowed | `false`\n| `-request-repeats` | number of request repeats | `1`\n| `-allow-code-blocks` | checking links in code blocks | `false`\n| `-timeout` | connection timeout (in seconds) | `30`\n| `-ignore-external` | ignore external links | `false`\n| `-ignore-internal` | ignore internal links | `false`\n| `-v` | enable verbose logging | `false`\n| `-help` or `-h` | Show available parameters | n/a\n\nFiles to be checked are given as free parameters.\n\n### Examples\n\n* Checks all links, without matching `github.com` in external links, in `.md` files in current directory+subdirectories without files matching `vendor` in path:\n\n```bash\nmilv -black-list=\"vendor\" -white-lis-ext=\"github.com\"\n```\n\n* Checks links only in `./README.md` and `./foo/bar.md` files:\n\n```bash\nmilv ./README.md ./foo/bar.md\n```\n\n### Docker image\n\nIf you do not want to install `milv` and it's dependencies you can simply use Docker and Docker image:\n\n```bash\ndocker run --rm -v $PWD:/milv:ro magicmatatjahu/milv:stability -base-path=/milv\n```\n\n## Config file\n\nThe configuration file allows for quick parameterization of the `milv` works. Config file must be a `.yaml` file.\n\nParameterization is very similar to using parameters in the `CLI`. However, you can configure files, located in subdirectories relative to the configuration file, separately with different config.\n\n### Examples\n\nIf your tree of your project look like this:\n\n```\n├── README.md\n├── LICENSE\n├── main.go\n├── milv.config.yaml\n└── src\n    ├── file.go\n    ├── file_test.go\n    ├── foo.md\n    └── some_dir\n        └── bar.md\n```\n\nyour config file can look like this:\n\n```yaml\nwhite-list-external: [\"localhost\", \"abc.com\"]\nwhite-list-internal: [\"LICENSE\"]\nblack-list: [\"./README.md\"]\nfiles:\n  - path: \"./src/foo.md\"\n    config:\n      white-list-external: [\"github.com\"]\n      white-list-internal: [\"#contributing\"]\n```\n\nBefore run validation, `milv` remove from files list `./README.md` file to check and connect global `white-list-external` with file `white-list-external` and `white-list-external` for `./src/foo.md` file will look that:\n\n```yaml\nwhite-list-external: [\"localhost\", \"abc.com\", \"github.com\"]\n```\n\nSimilarly will be with `white-list-internal`.\n\nIf you have a config file and you use a `CLI`, then `milv` will automatically combine the parameters from file and consol.\n\n#### Advanced configuration\n\n\u003e **NOTE**: For this example tree of project is the same as above.\n\nConfig file can look like this:\n\n```yaml\nwhite-list-external: [\"localhost\", \"abc.com\"]\nwhite-list-internal: [\"LICENSE\"]\nblack-list: [\"./README.md\"]\nrequest-repeats: 5\ntimeout: 45\nallow-redirect: false\nallow-code-blocks: true\nfiles:\n  - path: \"./src/foo.md\"\n    config:\n      white-list-external: [\"google.com\"]\n      white-list-internal: [\"#contributing\"]\n      request-repeats: 3\n      timeout: 30\n      allow-code-blocks: false\n    links:\n      - path: \"https://github.com/magicmatatjahu/milv\"\n        config:\n          timeout: 15\n          allow-redirect: true\n```\n\nIn this example we can see that `milv` will globally check external links with 45 seconds timeout, also won't allow redirect and will allow checking links in code snippets and default times of request repeats is set 5.\n\n`Milv` also allows to separately configurate files. Timeout in `./src/foo.md` file will be set to 30 seconds, links will be checking 3 times (if they will return error) and the links in code blocks won't be checked. However, a single link `https://github.com/magicmatatjahu/milv` will be checking with 15 seconds timeout with the possibility of redirection.\n\n## Troubleshooting links\n\nThe below table describes the types of errors during checking links and examples of how to solve them:\n\n| Error        | Solution example   |\n|-------------|----------------|\n| `404 Not Found` | Page doesn't exist - you have to change the external link to the correct one |\n| Error with formatting link | Correct link or if link has a variables or it is a example, add this link to the `white-list-external` or `white-list-internal` |\n| `The specified file doesn't exist` | Change the relative path to the file to the correct one or use a absolute path (second solution is not recommended) |\n| `The specified header doesn't exist in file` | Change the anchor link in `.md` file to the correct one. Sometimes `milv` give a hint (`Did you mean about \u003csimilar header\u003e?`) of which header (existing in the file) is very similar to the given. |\n| `The specified anchor doesn't exist...` or `The specified anchor doesn't exist in website...` | Check which anchors are on the external website and correct the specified anchor or remove the redirection to the given anchor. Sometimes `milv` give a hint (`Did you mean about \u003csimilar anchor\u003e?`) of which anchor (existing in the website) is very similar to the given. |\n| `Get \u003cexternal link\u003e: net/http: request canceled (Client.Timeout exceeded while awaiting headers)` | Increase net timeout to the all files, specific file or specific link or increase times of request repeats ([Here's](#advanced-configuration) how to do it) |\n| `Get \u003cexternal link\u003e: EOF ` | Same as above or change the link to the other one (probably website doesn't exist) |\n| Other types of errors and errors with contains `no such host` or `timeout` words | Most likely, the website doesn't exist or you do not have access to it. Possible solutions: change the link to another, correct one, remove it or add it to the `white-list-external` or `white-list-internal` |\n\nIt is a good practice to add local or internal (in the local network) links to the global white list of external or internal links, such as `http://localhost`.\n\n## Validate Pull Requests\n\n`milv` can help you validate links in all `.md` files in whole repository when a pull request is created (or a commit is pushed).\n\n### Jenkins\n\nTo use `milv` with Jenkins, connect your repo and create a [`Jenkinsfile`](https://jenkins.io/doc/book/pipeline/jenkinsfile/#creating-a-jenkinsfile) and add stage:\n\n```groovy\nstage(\"validate internal \u0026 external links\") {\n    workDir = pwd()\n    sh \"docker run --rm --dns=8.8.8.8 --dns=8.8.4.4 -v $workDir:/milv:ro magicmatatjahu/milv:0.0.6 -base-path=/milv\"\n}\n```\n\n## Other validators\n\nIn opensource community is available other links validation libraries written in JS, Ruby and others languages. Here are a few of note:\n\n* [awesome_bot](https://github.com/dkhamsing/awesome_bot): validator written in Ruby. Allows for validation external and internal links in `.md` files.\n* [remark-validate-links](https://github.com/remarkjs/remark-validate-links): validator written in JS. Allows for validation internal links in `.md` files.\n\n## Contact\n\n- [github.com/magicmatatjahu](https://github.com/magicmatatjahu)\n\n## Contributing\n\nIf you want contribute this project, firstly read [CONTRIBUTING.md](CONTRIBUTING.md) file for details of submitting pull requests.\n\n## License\n\nThis project is available under the MIT license. See the [LICENSE](LICENSE) file for more info.\n\n## ToDo\n\n* [ ] error handling \n* [ ] refactor (new architecture)\n* [ ] documentations\n* [ ] possibility to validation remote repositories hosted on **GitHub**\n* [ ] parse other type of files\n* [x] add more commands like a: timeout for http.Get(), allow redirects or SSL\n* [ ] landing page for project","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagicmatatjahu%2Fmilv","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmagicmatatjahu%2Fmilv","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmagicmatatjahu%2Fmilv/lists"}