{"id":20333274,"url":"https://github.com/zomato/vinifera","last_synced_at":"2025-07-10T10:38:23.269Z","repository":{"id":40012261,"uuid":"337703812","full_name":"Zomato/vinifera","owner":"Zomato","description":"A GitHub recon/monitoring tool for finding internal leaks belonging to your organisation.","archived":false,"fork":false,"pushed_at":"2024-01-07T09:48:28.000Z","size":488,"stargazers_count":90,"open_issues_count":9,"forks_count":15,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-25T17:46:51.785Z","etag":null,"topics":["github","recon","security"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Zomato.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-10T11:31:14.000Z","updated_at":"2025-03-19T13:05:37.000Z","dependencies_parsed_at":"2023-12-29T10:26:09.400Z","dependency_job_id":"edacd54d-a654-4843-87c3-946ea01f1859","html_url":"https://github.com/Zomato/vinifera","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zomato%2Fvinifera","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zomato%2Fvinifera/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zomato%2Fvinifera/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Zomato%2Fvinifera/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Zomato","download_url":"https://codeload.github.com/Zomato/vinifera/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248483426,"owners_count":21111446,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["github","recon","security"],"created_at":"2024-11-14T20:30:06.842Z","updated_at":"2025-04-11T21:33:12.550Z","avatar_url":"https://github.com/Zomato.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vinifera - Monitor Internal Leaks on Github\n\n#### Github Monitoring Tool :robot:\n\n![Vinifera Logo](docs/img/vinifera.png \"Vinifera Logo\")\n\n\u003chr\u003e\n\nWe have been using Vinifera in production since Dec 2019 and has helped us prevent security incidents.\nVinifera started out as an internal project to ensure Security hygiene of our public contributions and monitor potential leaks on Github.\nWe believe this will help other companies to strengthen their security hygiene when it comes to public sources like Github.\n\n![Stats](docs/img/stats.png \"Production Stats\")\n\n## What is Vinifera?\nVinifera allows Companies/Organizations to monitor public assets to find references to internal code leaks and potential breaches.\n\nSometimes developers might leak internal code and credentials by accident. Vinifera aims to help companies detect those breaches in due time and respond to the incident.\n\n## How does it work?\n\nVinifera monitors developers belonging to the organization, monitors and scans public contributions to look for potential violations and breach of internal/secret/proprietary code by looking for references defined.\n\n* Vinifera works by syncing org users.\n* For each user, all the public assets are registered (if not already tracked).\n* Each asset (Repo, Gist) is then scanned for any differences.\n* Each difference is then scanned, stored, and reported (if contains any leaks)\n\n![Vinifera Workflow](docs/img/workflow.png \"Vinifera Workflow\")\n\n## What does the name mean?\n\nDuring the development and inception of the tool, the Security team consumed lots of grapes, so we named it after the fruit we love :)\n\nVinifera is inspired from the [Bionomial name of Grapes](https://en.wikipedia.org/wiki/Vitis_vinifera)\n\n## Setup and Usage.\n\n### Requirements\nVinifera requires the installation of the following tools:\n1. PostgreSQL\n2. Redis\n3. Docker\n4. Ruby (Install via rbenv/rvm )\n\n### Setup\n\n##### Github Access Token\nTo scan your organization members, Vinifera requires a token with the ability to read Organization members.\n\n\n\nGenerate a new token [https://github.com/settings/tokens/new](https://github.com/settings/tokens/new) with no special scope\n\nYou would want to use the token of an admin user (with no special scope), since the admin can list all users of an organization.\n\n[https://docs.github.com/en/rest/reference/orgs#list-organization-members](https://docs.github.com/en/rest/reference/orgs#list-organization-members)\n\n\u003e If the authenticated user is also a member of this organization then both concealed and public members will be returned.\n\n\n#### Docker\n\n* Rename `.docker_env.example` to `.docker_env`\n\n```bash\nmv .docker_env.example .docker_env\n```\n\n* Update `.docker_env` with needed variables\n\n```bash\nGITHUB_ACCESS_TOKEN=\u003cREDACTED\u003e\nVINIFERA_ORG_NAME=\u003cYour_org_name\u003e\n\nRAILS_MAX_THREADS=60 # This also controls DB pool\n\nRAILS_MASTER_KEY=\u003cADD_YOUR_MASTER_KEY_HERE\u003e\n\n# set the environment\nRAILS_ENV=\u003cdevelopment|production\u003e\n\n# set the following if RAILS_ENV is set to production\nSECRET_KEY_BASE=\u003cYOUR_SECRET_KEY\u003e # min 32 bits key is allowed\n\n# add slack incoming webhook URLs for respective slack channels\nSLACK_UPDATES_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_TARGETS_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_USER_TRACKING_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_VINIFERA_VIOLATION_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_ERROR_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\n\n# By default fork and big fork scanning is disabled\nVINIFERA_ENABLE_FORK_SCANNING=false\nVINIFERA_ENABLE_BIG_FORK_SCANNING=false\n```\n\n* Build\n\n```bash\ndocker-compose build\n```\n* Run\n\n```bash\ndocker-compose up\n```\n\n* Re-building after any changes\n\n```bash\ndocker-compose up --build\n```\n\n* Sync Github Org Users\n\n```bash\ndocker exec -it vinifera_sidekiq bundle exec rake periodic_syncs:sync_github_users\n```\n\n\n\u003chr\u003e\n\n[Docker Compose Commands for Reference](https://docs.docker.com/compose/reference/)\n\u003chr\u003e\n\n#### Manual\n\n* Install required dependencies\n```bash\ncd \u003clocation_of_cloned_repo\u003e\nbundle install\n```\n\n* Setup DB and migrations\n```bash\nbundle exec rails db:create\nbundle exec rails db:migrate\n```\n\n* Setup Environment Variables and Slack WebHook\nSample env file is available at `.example_env`\n```bash\nGITHUB_ACCESS_TOKEN=\u003cREDACTED\u003e\nVINIFERA_ORG_NAME=\u003cYour_org_name\u003e # Name of org for which token was generated.\n\nVINIFERA_DATABASE_HOST=\u003clocalhost\u003e\nVINIFERA_DATABASE_PASSWORD=\u003cYOUR_PASSWORD\u003e\n\nRAILS_MAX_THREADS=60 # This also controls DB pool\n\nRAILS_MASTER_KEY=\u003cREDACTED\u003e\n\n\nSLACK_UPDATES_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_TARGETS_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_USER_TRACKING_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_VINIFERA_VIOLATION_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\nSLACK_ERROR_GROUP_URL=https://hooks.slack.com/services/\u003cYOUR_CONFIG_HERE\u003e\n\n\n\n# Add this only if you are using Docker over TLS, recommended way, if, on the same host as vinifera, you may skip it\n\nDOCKER_CLIENT_CERT_PATH=/home/deployer/.docker\nDOCKER_HOST=tcp://\u003cIP\u003e:\u003cPORT\u003e\n\n# By default fork and big fork scanning is disabled\nVINIFERA_ENABLE_FORK_SCANNING=false\nVINIFERA_ENABLE_BIG_FORK_SCANNING=false\n\nREDIS_URL=redis://\u003cREDIS_URI\u003e\nSIDEKIQ_REDIS_URL=redis://\u003cREDIS_URI\u003e\n\n# For PagerDuty Integration\nDEFAULT_PD_INTEGRATION_KEY=\u003cxxxxxxxx\u003e\nENABLE_PAGER_DUTY_TRIGGER=true\n```\n\n* Setup Cron Jobs\n```bash\nbundle exec whenever --update-crontab\n```\n* Setup your custom toml rules for Gitleaks. [For more info](https://github.com/zricethezav/gitleaks#rules-summary)\n```toml\n[[rules]]\n# Insert your rules here\n# description = \"Internal References\"\n# regex = '''(?i)((.*)\u003cCOMPANY_INTERNAL_REFERENCES\u003e(.*))'''\n# tags = [\"internal\", \"company\",\"references\"]\n```\n\n* Start Sidekiq\n```bash\nbundle exec sidekiq\n```\n\n##### Datadog\n\nAdditionally, to get the metrics on Datadog like in the above screenshot, you can use the DataDog agent - [https://docs.datadoghq.com/agent/](https://docs.datadoghq.com/agent/)\n\n\n##### PagerDuty\n\nTo ensure the team never misses any violation, PagerDuty integration option is there:\n\n![PagerDuty Incident](docs/img/pager_duty_incident.png \"PagerDuty Incident\")\n\nFor PagerDuty integration, following environment variable needs to be set to `true`\n\n```bash\nENABLE_PAGER_DUTY_TRIGGER=true\n```\n\nThen create a new service and a integration key as described in following doc - https://support.pagerduty.com/docs/services-and-integrations#create-a-new-service\n\n![PagerDuty Service](docs/img/pager_duty_service.png \"PagerDuty Service\")\n\n```bash\nDEFAULT_PD_INTEGRATION_KEY=\u003cxxxxxxxx\u003e\n```\n\n## Contributing\n\nWe are open to contributions/bug fixes/performance improvements to our project :)\n\n## Donations\n\nIf you appreciate the tool we have built, feel free to contribute/donate to the projects on the top of which Vinifera was built :)\n\nVinifera is built on top of other open-source software:\n1. [Rails](https://github.com/rails/rails) (Our Favourite Web Framework)\n2. [Sidekiq](https://github.com/mperham/sidekiq) (Handles Job LifeCycle, Scheduling, and Retries)\n3. [Gitleaks](https://github.com/zricethezav/gitleaks) (Gitleaks powers the code scanning via Docker Images)\n4. [Docker](https://github.com/docker) (For running Scans in an isolated environment)\n5. [Docker-api](https://github.com/swipely/docker-api) (Ruby Client to interact with Docker Remote API)\n6. [Sidekiq Throttled ](https://github.com/sensortower/sidekiq-throttled) (For Throttling workers)\n7. [Octokit](https://github.com/octokit/octokit.rb) (Ruby Toolkit for Github API)\n8. [Whenever](https://github.com/javan/whenever) (Cron Jobs in Ruby)\n\n.... (list will go on ..., you get the gist ;) )\n\nYou can also donate to [Feeding India](https://www.feedingindia.org/)\n\n## Disclaimer\n\nNeither Zomato nor the developers of this tool are responsible for any damage caused by this tool or usage of the same.\nUse responsibly. Refer to [LICENSE](LICENSE) for more details.\n\n\n## License\n\nVinifera is licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full license text.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzomato%2Fvinifera","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzomato%2Fvinifera","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzomato%2Fvinifera/lists"}