{"id":22180577,"url":"https://github.com/dcso/tiffy","last_synced_at":"2025-07-26T17:32:19.097Z","repository":{"id":48392467,"uuid":"203974995","full_name":"DCSO/tiffy","owner":"DCSO","description":"TIE Feed Generator for MISP (replaces tie2misp)","archived":false,"fork":false,"pushed_at":"2021-07-28T12:29:35.000Z","size":124,"stargazers_count":5,"open_issues_count":1,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2023-03-08T18:48:28.952Z","etag":null,"topics":["misp"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DCSO.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-23T10:11:50.000Z","updated_at":"2021-09-15T15:45:24.000Z","dependencies_parsed_at":"2022-09-21T20:01:10.794Z","dependency_job_id":null,"html_url":"https://github.com/DCSO/tiffy","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCSO%2Ftiffy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCSO%2Ftiffy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCSO%2Ftiffy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DCSO%2Ftiffy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DCSO","download_url":"https://codeload.github.com/DCSO/tiffy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227700405,"owners_count":17806365,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["misp"],"created_at":"2024-12-02T09:18:45.808Z","updated_at":"2024-12-02T09:18:46.600Z","avatar_url":"https://github.com/DCSO.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tiffy\n\n## About\nGenerate Feeds from TIE Content.\n\n## Requirements\n### Base\n- Python 3.7\n- TIE API Key http://tie.dcso.de  \n\n### Packages\n- PyTest https://pytest.org\n- pytest-testdox https://github.com/renanivo/pytest-testdox\n- Requests http://python-requests.org\n- PyYAML http://pyyaml.org\n- Click http://click.pocoo.org/\n- PyMISP https://github.com/MISP/PyMISP\n- python-dateutil https://dateutil.readthedocs.io\n\n## Install\n```bash\n$ git clone https://github.com/DCSO/tiffy.git\n$ pip3 install -r requirements.txt\n```\n\n## Configuration\nThe command line client expects a configuration file in the `tiffy/settings` directory where you have to\ndefine the required API key and URL. To create the config file, just copy the `config.sample.yml` file to `config.yml`\nand edit it. You also need to copy `tags.sample.yml` and set up any desired tags to be set. \n\n```bash\n$ cp settings/config.sample.yml settings/config.yml\n\n$ vim settings/config.yml\n```\n\n## HowTo\nTo start the generator just run:\n```bash\n$ ./tiffy.py\n```\nThe generator will now process all IOCs as attributes beginning from the current system date.\n\nIf no first seen date is set, the generator will always use the current system date as default.\n\nTo process attributes from or until a specific date you can use the `--first-seen YYYY-MM-DD` or \n`--last-seen YYYY-MM-DD` option. You can also combine both parameters.\n\n```bash\n$ ./tiffy.py --first-seen 2019-03-13\n\n$ ./tiffy.py --last-seen 2019-07-13\n\n$ ./tiffy.py --first-seen 2019-03-13 --last-seen 2019-07-13\n```\n\n### Using the Source, Actor, Category or Family parameter\nSometimes it's necessary to get all IOC's from a specific attacker group or tool family. In most cases these are known \nunder more than one name. Because of this, tiffy offers the capability to search for these values.\n\nYou can pass one or multiple values either as a single string or a comma delimited list of strings\n\nQuery family based IOC's\n```bash\n$ ./tiffy.py --family example\n\n$ ./tiffy.py --family example1,example2\n```\n\nQuery actor based IOC's\n```bash\n$ ./tiffy.py --actor example\n\n$ ./tiffy.py --actor example1,example2\n```\n\nQuery category based IOC's\n```bash\n$ ./tiffy.py --category example\n\n$ ./tiffy.py --category example1,example2\n```\n\nQuery source based IOC's\n```bash\n$ ./tiffy.py --source example\n\n$ ./tiffy.py --source example1,example2\n```\n\n## Using Severity and Confidence Parameters\n\ntiffy is also able to filter ioc's based on min or max severity/confidence. If you pass only a min-value, tiffy\nwill search all values at or above the value. If you pass only a max-value, ioc's at or below the value will be returned.\nWhen passing min and max-value, all ioc's between these values are used.\n\nQuery severity based IOC's\n```bash\n$ ./tiffy.py --min-severity 2                          #gets all ioc's from and including severity 2    \n\n$ ./tiffy.py --max-severity 4                          #gets all ioc's up to and including severity 4\n\n$ ./tiffy.py --min-severity 2 --max-severity 4         #gets all ioc's from severity 2 to severity 4\n```\n\nQuery confidence based IOC's\n```bash\n$ ./tiffy.py --min-confidence 2                        #gets all ioc's from and including confidence 2    \n\n$ ./tiffy.py --max-confidence 4                        #gets all ioc's up to and including confidence 4\n\n$ ./tiffy.py --min-confidence 2 --max-confidence 4     #gets all ioc's from confidence 2 to confidence 4\n```\n\n## Query specific data types\n\nYou can specify the data type to be queried by TIE. This is useful, for example, if you use the txt output parameter to\ncreate specific files that contain only domains, URL, and so on.\n\n```bash\n$ ./tiffy.py --data-type urlverbatim\n\n$ ./tiffy.py --data-type urlverbatim,domainname\n\n$ ./tiffy.py --data-type urlverbatim  --output-format txt\n```\n\n## Setting Default Tags for the MISP Event\n\nYou can pass tags for the newly created event. Tags are passed as MISP-compatible JSON Strings and will be added \nto the base event. Double quotes need to be escaped. If no tags are passed `TLP:AMBER` will be used as default.\n\n```bash\n$ ./tiffy.py --event-tags {\\\"name\\\":\\\"tlp:amber\\\"}\n```\n\n## Setting Tags for Attributes\n\nYou can assign Tags to attributes using the `settings/tags.yml` file. Currently only attributes derived from TIE\n`c2-server` and `malware` observations can be tagged. Add your desired Tag to the list of tags as seen in the sample\nfile. The format for a new entry is:\n\n```\n- name: tag:name:here\n```\n\n## Setting the Output Format\n\nYou can choose the output format of the feed. Currently MISP-JSON and text files are supported but more formats will\nfollow.\n\n```bash\n$ ./tiffy.py --output-format MISP\n\n$ ./tiffy.py --output-format txt\n```\n\n## Disable the Default Filter\n\ntiffy will use the default TIE filter. You can disable this behaviour by passing the `--no-filter` parameter.\n\n```bash\n$ ./tiffy.py --no-filter\n```\n\n## Additional Parameters\n\ntiffy offers some additional parameters:\n- `--loglvl` sets the log level. Values are 0 - NOTSET / 10 - DEBUG / 20 - INFO / 30 - WARNING / 40 - ERROR / 50 - CRITICAL\n- `--disable_console_log` disables log output to the console\n- `--disable_file_log` disables logging to file\n- `--log_file_path` sets the path where the file log should be saved\n\n```bash\n$ ./tiffy.py --loglvl 10\n\n$ ./tiffy.py --disable_console_log\n\n$ ./tiffy.py --disable_file_log\n\n$ ./tiffy.py --log_file_path \"path/to/log/destination\"\n```\n\n## Using a Proxy\ntiffy offers various ways for the use of a proxy. First, if the system variable `HTTP_PROXY` or `HTTPS_PROXY` is \nset, tiffy will automatically use the given information's.\n\nIf no system variable is used, tiffy will check if the parameter `--proxy_http` or `--proxy_https` is set. If so, tiffy will use the parameter for pulling informations. \n\nYou can use only `--proxy_http` or `--proxy_https` or both\n```bash\n$ ./tiffy.py --proxy_http \"http://10.8.0.1:8000\"\n$ ./tiffy.py --proxy_http \"http://10.8.0.1:8000 --proxy_https \"http://10.8.0.1:8443\"\n```\nWith HTTP Basic Auth\n```bash\n$ ./tiffy.py --proxy_http \"http://user:pass@10.8.0.1:8000\"\n```\n\n### Disable Certificate Verification\n\nIf your Proxy is using SSL-Interception, it might be necessary to disable\nthe certificate verification for requests. Use the `--disable_cert_verify` flag\nin this case.\n\n```bash\n$ ./tiffy.py --proxy_http \"http://10.8.0.1:8000\" --disable_cert_verify\n```\n\n### Environment Variables\n\nAll config file settings can also be set using the environment variables documented in the tables below.\nYou can run tiffy without a config file only using environment variables, but in this case you MUST provide\nat least the required variables.\n\n#### Required Variables\n\n| Variable                          | Default | Example                                | Description                    |\n| --------------------------------- | ------- | -------------------------------------- | ------------------------------ |\n| TIFFY_CONF_TIE_APIURL             |         |  https://tie.dcso.de/v1/api            | URL to TIE.                    |\n| TIFFY_CONF_TIE_APIKEY             |         |  12345683127481209123789               | API token for TIE access       |\n| TIFFY_CONF_MISP_ORGANISATION_NAME |         |  ACME                                  | Name of your MISP organization |\n| TIFFY_CONF_MISP_ORGANISATION_UUID |         |  5804adw2-12fe-1234-34av-07lk82aw012a  | UUID of your MISP organization |\n\n#### Optional Variables\n\n| Variable                                 | Default              | Example                  | Description                                                       |\n| ---------------------------------------- | -------------------- | ------------------------ | ----------------------------------------------------------------- |\n| TIFFY_CONF_MISP_EVENTS_BASE_THREAT_LEVEL | 3                    |                          | IoC will get this threat level if it is added                     |\n| TIFFY_CONF_MISP_EVENTS_BASE_CONFIDENCE   | 80                   |                          | IoC will get this confidence if it is added                       |\n| TIFFY_CONF_MISP_EVENTS_BASE_SEVERITY     | 2                    |                          | IoC will get this severity if it is added                         |\n| TIFFY_CONF_MISP_EVENTS_PUBLISHED         | false                |                          | IoC will get published in MISP                                    |\n| TIFFY_CONF_MISP_ATTRIBUTES_TO_IDS        | false                |                          | Set IDS flag for this IoC                                         |\n| TIFFY_CONF_MISP_ATTRIBUTES_TO_IDS_THRESHOLD| 90                 |  90                      | Confidence Threshold at or above which attributes should get IDS flag                                        |\n| TIFFY_PARAM_TIE_SEEN_FIRST               |                      | YYYY-MM-DD               | Download only IoC which are first seen at ... and newer           |\n| TIFFY_PARAM_TIE_SEEN_LAST                |                      | YYYY-MM-DD               | Download only IoC which are last seen at ... and older            |\n| TIFFY_PARAM_TIE_ACTOR                    |                      | example1,example2        | Download only IoC with this actor                                 |\n| TIFFY_PARAM_TIE_CATEGORY                 |                      | example1,example2        | Download only IoC with this category                              |\n| TIFFY_PARAM_TIE_FAMILY                   |                      |   example1,example2      | Download only IoC with this family                                |\n| TIFFY_PARAM_TIE_SOURCE                   |                      | example1,example2        | Download only IoC from this source                                |\n| TIFFY_PARAM_TIE_SEVERITY_MIN             |                      | 2                        | Download only IoC with this minimum severity                      |\n| TIFFY_PARAM_TIE_SEVERITY_MAX             |                      | 4                        | Download only IoC with this maximum severity                      |\n| TIFFY_PARAM_TIE_CONFIDENCE_MIN           |                      | 2                        | Download only IoC with this minimum confidence                    |\n| TIFFY_PARAM_TIE_CONFIDENCE_MAX           |                      | 4                        | Download only IoC with this maximum confidence                    |\n| TIFFY_PARAM_TIE_MISP_EVENT_TAGS          | {\\\\\"name\\\\\":\\\\\"tlp:amber\\\\\"}| {\\\\\"name\\\\\":\\\\\"tlp:amber\\\\\"} | Tag Event with the defined tags                                   |\n| TIFFY_PARAM_OUTPUT_FORMAT                | MISP                 |                          | You can choose the output format of the feed.                     |\n| TIFFY_PARAM_TIE_DISABLE_DEFAULT_FILTER   | false                | true / false             | To disable the default TIE filter.                                |\n| TIFFY_PARAM_LOG_LEVEL                    | INFO                 |                          | Define one of these log levels: DEBUG, INFO, WARNING, ERROR, CRITICAL |\n| TIFFY_PARAM_LOG_DISABLE_CONSOLE          | false                | true / false             | Disables log output to stdout                                     |\n| TIFFY_PARAM_LOG_DISABLE_FILE             |   false              | true / false             | Disables log output to file                                       |\n| TIFFY_PARAM_LOG_FILE                     | \"~/tiffy.log\"        |                          | Define the log path                                               |\n\n#### Proxy Variables\n\n| Variable    | Default | Example                               | Description                              |\n| ----------- | ------- | ------------------------------------- | ---------------------------------------- |\n| HTTP_PROXY  |         |  http://10.8.0.1:8000                 | Set an Proxy server for HTTP connections |\n| HTTPS_PROXY |         |  https://\u003cuser\u003e:\u003cpass\u003e@10.8.0.1:8000  | Set Proxy server for HTTPS connections   |\n\n## Setting up Feed\nif tiffy ran successfully at least once, the directory `tiffy/feed` will be present. In this directory are all files needed for a MISP Feed. You need to upload these files onto a file server like nginx or apache.\n\nWe used nginx as an example. Upload the files into a directory on the nginx server and add a server configuration in the nginx.conf to make the directory accessible.\n\n```\nserver {\n\t\tlisten 8001;  #port the nginx should listen on and provide the feed\n\t    root /path/to/feed/directory;\n\t    \tautoindex on;        # tells nginx to automatically index the files so that they can be accessed\n\t    location / {\n\t    \tautoindex on;        # tells nginx to automatically index the files so that they can be accessed\n\t    }\n\t}\n```\n\nAfter setting up the file server, the feed can be added to MISP. From the main menu, go to `Sync Actions`-\u003e `List Feeds`. In the menu on the left, select `Add Feed`. \n\nSet the feed to enabled and activate lookup and caching. Name the feed and the provider and set input source to Network. In the url field, add the url to the feed directory on your file server.\n\n![alt text](https://raw.githubusercontent.com/DCSO/tiffy/master/images/add_feed.png \"Add Feed\")\n\nAfter setting up the feed and enabling it, the events will be imported into MISP. To manually start an import, click the download icon on the TIE feed in the feed list. To see all events and import single events contained in the feed, click on the magnifying glass icon.\n\n![alt text](https://raw.githubusercontent.com/DCSO/tiffy/master/images/options.png \"Options\")\n\n## License\n\nThis software is released under a BSD 3-Clause license.\nPlease have a look at the LICENSE file included in the repository.\n\nCopyright (c) 2019, DCSO Deutsche Cyber-Sicherheitsorganisation GmbH\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcso%2Ftiffy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdcso%2Ftiffy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcso%2Ftiffy/lists"}