{"id":13539661,"url":"https://github.com/lmco/laikaboss","last_synced_at":"2025-04-02T06:31:18.438Z","repository":{"id":33674856,"uuid":"37327577","full_name":"lmco/laikaboss","owner":"lmco","description":"Laika BOSS: Object Scanning System","archived":false,"fork":false,"pushed_at":"2023-02-11T03:48:48.000Z","size":1847,"stargazers_count":720,"open_issues_count":26,"forks_count":155,"subscribers_count":132,"default_branch":"master","last_synced_at":"2024-04-16T07:46:00.510Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lmco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2015-06-12T14:49:03.000Z","updated_at":"2024-04-08T17:08:33.000Z","dependencies_parsed_at":"2022-07-31T11:08:00.462Z","dependency_job_id":"3b92da86-60fe-4c08-bb14-03dabb281c01","html_url":"https://github.com/lmco/laikaboss","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmco%2Flaikaboss","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmco%2Flaikaboss/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmco%2Flaikaboss/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lmco%2Flaikaboss/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lmco","download_url":"https://codeload.github.com/lmco/laikaboss/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246767864,"owners_count":20830569,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T09:01:30.132Z","updated_at":"2025-04-02T06:31:13.430Z","avatar_url":"https://github.com/lmco.png","language":"Python","funding_links":[],"categories":["Network","\u003ca id=\"8f92ead9997a4b68d06a9acf9b01ef63\"\u003e\u003c/a\u003e扫描器\u0026\u0026安全扫描\u0026\u0026App扫描\u0026\u0026漏洞扫描","Challenges","Tools","\u003ca id=\"132036452bfacf61471e3ea0b7bf7a55\"\u003e\u003c/a\u003e工具"],"sub_categories":["Other Resources","\u003ca id=\"de63a029bda6a7e429af272f291bb769\"\u003e\u003c/a\u003e未分类-Scanner","Frameworks"],"readme":"# Laika BOSS: Object Scanning System\n\nLaika is an object scanner and intrusion detection system that strives to achieve the following goals:\n\n+ **Scalable**\n\t+ Work across multiple systems\n\t+ High volume of input from many sources\n\n+ **Flexible**\n\t+ Modular architecture\n\t+ Highly configurable dispatching and dispositioning logic\n\t+ Tactical code insertion (without needing restart)\n\n+ **Verbose**\n\t+ Generate more metadata than you know what to do with\n\nEach scan does three main actions on each object:\n\n+ **Extract child objects** Some objects are archives, some are wrappers, and others are obfuscators. Whatever the case may be, find children objects that should be scanned recursively by extracting them out.\n\n+ **Mark flags** Flags provide a means for dispositioning objects and for pivoting on future analysis.\n\n+ **Add metadata** Discover as much information describing the object for future analysis.\n\n**Feel free to read the [whitepaper](https://github.com/lmco/laikaboss/raw/master/LaikaBOSS_Whitepaper.pdf)!**\n\n## Components\n\nLaika is composed of the following pieces:\n\n+ **Framework** (`laika.py`) This is the core of Laika BOSS. It includes the object model and the dispatching logic.\n\n+ **laikad** This piece contains the code for running Laika as a deamonized, networked service using the ZeroMQ broker.\n\n+ **cloudscan** A command-line client for sending a local system file to a running service instance of Laika (laikad).\n\n+ **modules** The scan itself is composed of the running of modules. Each module is its own program that focuses on a particular sub-component of the overall file analysis.\n\n\n## Getting Started\n\nLaika BOSS has been tested on the latest versions of CentOS and Ubuntu LTS\n\n### Installing on Ubuntu\n\n1. Install framework dependencies:\n\n\t```shell\n\tapt-get install yara python-yara python-progressbar python-pip\n\tpip install interruptingcow\n\t```\n\n2. Install network client and server dependencies:\n\n\t```shell\n\tapt-get install libzmq3 python-zmq python-gevent python-pexpect\n\t```\n\n3. Install module dependencies:\n\n\t```shell\n\tapt-get install python-ipy python-m2crypto python-pyclamd liblzma5 libimage-exiftool-perl python-msgpack libfuzzy-dev python-cffi python-dev unrar\n\tpip install fluent-logger olefile ssdeep py-unrar2 pylzma javatools\n\twget https://github.com/smarnach/pyexiftool/archive/master.zip\n\tunzip master.zip\n\tcd pyexiftool-master\n\tpython setup.py build\n\tpython setup.py install\n\twget https://github.com/erocarrera/pefile/archive/pefile-1.2.10-139.tar.gz\n\ttar vxzf pefile-1.2.10-139.tar.gz\n\tcd pefile-1.2.10-139\n\tpython setup.py build\n\tpython setup.py install\n\t```\n\n### Installing on CentOS\n\n1. Install framework dependencies\n\n\t```shell\n\tsudo yum install -y epel-release\n\tsudo yum install -y autoconf automake libtool libffi-devel python-devel python-pip python-zmq ssdeep-devel swig\n\t```\n\n2. Install Python modules\n\n\t```shell\n\tpip install IPy cffi interruptingcow fluent-logger javatools m2crypto olefile pylzma pyclamd py-unrar2\n\tpip install six --upgrade --force-reinstall\n\tpip install ssdeep\n\t```\n\n3. Install Yara\n\n\tThere is no Yara package for CentOS, so we have to build it from source. You can't use a checkout from Github as it won't contain the Python code; you must download one of the [release versions](https://github.com/virustotal/yara/releases). The following uses Yara version 3.5.0\n\n\t```shell\n\twget https://github.com/VirusTotal/yara/archive/v3.5.0.zip\n\tunzip yara-3.5.0.zip\n\tcd yara-3.5.0\n\tchmod +x ./build.sh\n\t./build.sh\n\tsudo make install\n\tcd yara-python\n\tpython setup.py build\n\tsudo python setup.py install\n\t```\n\n4. Install pyexif\n\n\t```shell\n\twget https://github.com/smarnach/pyexiftool/archive/master.zip\n\tunzip master.zip\n\tpython setup.py build\n\tsudo python setup.py install\n\t```\n\n5. Install pefile\n\n\t```shell\n\twget https://github.com/erocarrera/pefile/archive/pefile-1.2.10-139.tar.gz\n\ttar vxzf pefile-1.2.10-139.tar.gz\n\tcd pefile-1.2.10-139\n\tpython setup.py build\n\tpython setup.py install --user\n\t```\n\nYou may need to set the `LD_LIBRARY_PATH` variable to include `/usr/local/lib` when running Laika.\n\n### Installing Laika BOSS (optional)\n\nYou may use the provided setup script to install the Laika BOSS framework, client library, modules and associated scripts (`laika.py`, `laikad.py`, `cloudscan.py`).\n\n```shell\npython setup.py install\n```\n\n#### Standalone instance\n\nFrom the directory containing the framework code, you may run the standalone scanner, `laika.py` against any file you choose. If you move this file from this directory you'll have to specify various config locations. By default it uses the configurations in the `./etc` directory.\n\nWe recommend using installing [jq](http://stedolan.github.io/jq/) to parse Laika output.\n\n```javascript\n$ ./laika.py ~/test_files/testfile.cws.swf | jq '.scan_result[] | { \"file type\" : .fileType, \"flags\" : .flags, \"md5\" : .objectHash }'\n100%[############################################] Processed: 1/1 total files (Elapsed Time: 0:00:00) Time: 0:00:00\n{\n  \"md5\": \"dffcc2464911077d8ecd352f3d611ecc\",\n  \"flags\": [],\n  \"file type\": [\n    \"cws\",\n    \"swf\"\n  ]\n}\n{\n  \"md5\": \"587c8ac651011bc23ecefecd4c253cd4\",\n  \"flags\": [],\n  \"file type\": [\n    \"fws\",\n    \"swf\"\n  ]\n}\n```\n\n#### Networked instance\n\n```javascript\n$ ./laikad.py\n\n$ ./cloudscan.py ~/test_files/testfile.cws.swf | jq '.scan_result[] | { \"file type\" : .fileType, \"flags\" : .flags, \"md5\" : .objectHash }'\n{\n  \"md5\": \"dffcc2464911077d8ecd352f3d611ecc\",\n  \"flags\": [],\n  \"file type\": [\n    \"cws\",\n    \"swf\"\n  ]\n}\n{\n  \"md5\": \"587c8ac651011bc23ecefecd4c253cd4\",\n  \"flags\": [],\n  \"file type\": [\n    \"fws\",\n    \"swf\"\n  ]\n}\n```\n\n#### Milter\n\nThe Laika BOSS milter server allows you to integrate Laika BOSS with mail transfer agents such as Sendmail or Postfix. This enables better visibility (passive visibility can be hampered by TLS) and provides a means to block email according to Laika BOSS disposition.\n\n```\n+----------------+             +---------------+             +----------------+\n|                |    email    |               |   email     |                |\n|    sendmail    +-------------\u003e  laikamilter  +-------------\u003e     laikad     |\n|                | accept/deny |               | scan result |                |\n|                \u003c-------------+               \u003c-------------+                |\n+----------------+             +---------------+             +----------------+\n```\n\nThe Laika BOSS milter server requires the [python-milter](https://pythonhosted.org/milter) module and the Laika BOSS client library. Check out the comments in the source code for more details.\n\n#### Suricata Integration Prototype\n\nWe have released a proof of concept feature for Suricata that allows it to store extracted files and their associated metadata in a Redis database. You will find this code under a [new branch](https://github.com/lmco/suricata/tree/file_extract_redis_prototype_v1) in our Suricata fork. We hope to refine the implementation and eventually have it accepted by the project.\n\nOnce you've enabled file extraction and the optional Redis integration in Suricata, you can extract these files from Redis and submit them to Laika BOSS for scanning by using the middleware script `laika_redis_client.py` as shown below. Note that it requires the `python-redis` module.\n\nFirst, start `laikad.py` in async mode:\n\n```shell\n./laikad.py -a\n```\n\nThen launch the middleware script and give it the address of the `laikad` broker and Redis database (defaults shown below):\n\n```shell\n./laika_redis_client.py -b tcp://localhost:5558 -r localhost -p 6379\n```\n\nNote that you will need to use a logging module such as `LOG_FLUENT` to export the full scan result of the these file scans from `laikad`.\n\n## Licensing\n\nThe Laika framework and associated modules are released under the terms of the Apache 2.0 license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmco%2Flaikaboss","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flmco%2Flaikaboss","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flmco%2Flaikaboss/lists"}