https://github.com/ail-project/ail-feeder-github-repo
AIL feeder for GitHub Repository
https://github.com/ail-project/ail-feeder-github-repo
Last synced: about 2 months ago
JSON representation
AIL feeder for GitHub Repository
- Host: GitHub
- URL: https://github.com/ail-project/ail-feeder-github-repo
- Owner: ail-project
- License: agpl-3.0
- Created: 2021-12-01T10:10:19.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-11-08T04:38:36.000Z (over 2 years ago)
- Last Synced: 2025-02-28T21:29:04.015Z (over 1 year ago)
- Language: Python
- Size: 29.3 KB
- Stars: 2
- Watchers: 5
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# AIL - feeder from Github Repository
This AIL feeder is a generic piece of software to extract information from Github Repositories to collect and feed AIL via its ReST API.
# Usage
~~~shell
dacru@dacru:~/git/ail-feeder-github-repo/bin$ python3 github_repo.py --help
usage: github_repo.py [-h] -l LIST_REPO [--nocache] [-v] [-d]
optional arguments:
-h, --help show this help message and exit
-l LIST_REPO, --list_repo LIST_REPO
list of repo to analyse
--nocache disable store of repository
-v, --verbose verbose, more display
-d, --debug debug mode
~~~
# JSON output format to AIL
- `source` is the name of the AIL feeder module
- `source-uuid` is the UUID of the feeder (unique per feeder)
- `data` is data in file
- `meta` is the generic field where feeder can add the metadata collected
Using the AIL API, `data` will be compressed in gzip format and encoded in base64. Then a new field will be created, `data-sha256` which will be the result of sha256 on data after treatment.
# (main) Requirements
- [PyAIL](https://github.com/ail-project/PyAIL)
- [magic](https://github.com/ahupp/python-magic)
- For magic, according to your OS, some additional dependecies need to be installed.
- For Debian/Ubuntu:
```
sudo apt-get install libmagic1
```
- For Windows
```
pip install python-magic-bin
```
- For MacOS
```
brew install libmagic
```
- [redis](https://github.com/redis/redis-py)
## ail_feeder_github_repo
~~~json
{
"data": "[general]\nuuid = 183f2812-db38-4935-b5da-ad03f94f118f\n\n[github]\napi_token = \n\n[cache]\nexpire = 86400\n\n[ail]\nurl = https://127.0.0.1:7020/api/v1/import/json/item\napikey = \n\n[redis]\nhost = 127.0.0.1\nport = 6379\ndb = 0\n\n[repo]\npathRepo = ",
"meta": {
"github_repo:path_file": "ail-feeder-github-repo-main/etc/config.cfg",
"github_repo:file_size": "260",
"github_repo:file_extention": "cfg",
"github_repo:id": "433775806",
"github_repo:node_id": "R_kgDOGdrkvg",
"github_repo:name": "ail-project/ail-feeder-github-repo",
"github_repo:owner_login": "ail-project",
"github_repo:owner_id": "62389074",
"github_repo:owner_node_id": "MDEyOk9yZ2FuaXphdGlvbjYyMzg5MDc0",
"github_repo:datestamp": "2021-12-01",
"github_repo:timestamp": "10:10:19",
"github_repo:timezone": "UTC"
}
}
~~~
## Format list to process repository
~~~json
[
{
"user": "ail-project",
"repo_name": "ail-feeder-github-repo",
"commit": "",
"branch": ""
},
{
"user": "ahupp",
"repo_name": "python-magic",
"commit": "",
"branch": "libmagic-compat"
}
]
~~~
## Download all repository from an organization
If no `repo_name` is given, then the user is consider as an organization and all of is repository will be download
~~~
[
{
"user": "ail-project",
"repo_name": "",
"commit": "",
"branch": ""
}
]
~~~
## License
This software is licensed under [GNU Affero General Public License version 3](http://www.gnu.org/licenses/agpl-3.0.html)
Copyright (C) 2021-2023 CIRCL - Computer Incident Response Center Luxembourg
Copyright (C) 2021-2023 David Cruciani