{"id":22748037,"url":"https://github.com/githubfoam/moloch-sandbox","last_synced_at":"2025-03-30T05:42:34.120Z","repository":{"id":113753727,"uuid":"225482481","full_name":"githubfoam/moloch-sandbox","owner":"githubfoam","description":"network security monitoring visibility , ELK, CTI, DFIR","archived":false,"fork":false,"pushed_at":"2020-08-05T21:53:03.000Z","size":142,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-05T07:49:40.912Z","etag":null,"topics":["elasticsearch","elk","kibana","logstash","moloch","network-forensics","network-monitoring","pcap"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/githubfoam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-02T22:38:11.000Z","updated_at":"2020-08-05T14:25:29.000Z","dependencies_parsed_at":"2023-05-15T04:30:21.857Z","dependency_job_id":null,"html_url":"https://github.com/githubfoam/moloch-sandbox","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/githubfoam%2Fmoloch-sandbox","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/githubfoam%2Fmoloch-sandbox/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/githubfoam%2Fmoloch-sandbox/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/githubfoam%2Fmoloch-sandbox/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/githubfoam","download_url":"https://codeload.github.com/githubfoam/moloch-sandbox/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246281216,"owners_count":20752207,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","elk","kibana","logstash","moloch","network-forensics","network-monitoring","pcap"],"created_at":"2024-12-11T03:19:14.151Z","updated_at":"2025-03-30T05:42:34.103Z","avatar_url":"https://github.com/githubfoam.png","language":"Shell","readme":"# moloch-sandbox\n\n[![Build Status](https://travis-ci.com/githubfoam/moloch-sandbox.svg?branch=master)](https://travis-ci.com/githubfoam/moloch-sandbox)  \n\nInstall moloch\n~~~~\n\n\u003evagrant up \"vg-moloch-01\"\n\u003evagrant ssh \"vg-moloch-01\"\n\n$ whoami\nvagrant\n$ id vagrant\nuid=1000(vagrant) gid=1000(vagrant) groups=1000(vagrant),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),110(lxd),115(lpadmin),116(sambashare)\n\n\n$ sudo /data/moloch/bin/Configure\nFound interfaces: eth0;eth1;lo\nSemicolon ';' seperated list of interfaces to monitor [eth1]\nInstall Elasticsearch server locally for demo, must have at least 3G of memory, NOT recommended for production use (yes or no) [no] yes\n/usr/bin/java\nPassword to encrypt S2S and other things [no-default] no\nMoloch - Creating configuration files\nInstalling systemd start files, use systemctl\nMoloch - Downloading and installing demo OSS version of Elasticsearch\n--2020-08-05 12:05:17--  https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.8.2.deb\nResolving artifacts.elastic.co (artifacts.elastic.co)... 151.101.114.222, 2a04:4e42:3::734\nConnecting to artifacts.elastic.co (artifacts.elastic.co)|151.101.114.222|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 68347988 (65M) [application/octet-stream]\nSaving to: ‘elasticsearch-oss-6.8.2.deb’\n\nelasticsearch-oss-6.8.2.deb             100%[=============================================================================\u003e]  65.18M  5.76MB/s    in 12s\n\n2020-08-05 12:05:30 (5.59 MB/s) - ‘elasticsearch-oss-6.8.2.deb’ saved [68347988/68347988]\n\nSelecting previously unselected package elasticsearch-oss.\ndpkg: regarding elasticsearch-oss-6.8.2.deb containing elasticsearch-oss:\n elasticsearch-oss conflicts with elasticsearch\n  elasticsearch (version 6.8.5) is present and installed.\n\ndpkg: error processing archive elasticsearch-oss-6.8.2.deb (--install):\n conflicting packages - not installing elasticsearch-oss\nErrors were encountered while processing:\n elasticsearch-oss-6.8.2.deb\nMoloch - Installing /etc/logrotate.d/moloch to rotate files after 7 days\nMoloch - Installing /etc/security/limits.d/99-moloch.conf to make core and memlock unlimited\nDownload GEO files? (yes or no) [yes] yes\nMoloch - Downloading GEO files\n2020-08-05 12:05:39 URL:https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.csv [23322/23322] -\u003e \"ipv4-address-space.csv\" [1]\nWARNING: timestamping does nothing in combination with -O. See the manual\nfor details.\n\nUsername/Password Authentication Failed.\nWARNING: timestamping does nothing in combination with -O. See the manual\nfor details.\n\nUsername/Password Authentication Failed.\n2020-08-05 12:05:41 URL:https://raw.githubusercontent.com/wireshark/wireshark/master/manuf [1733449/1733449] -\u003e \"oui.txt\" [1]\n\nMoloch - Configured - Now continue with step 4 in /data/moloch/README.txt\n\n      /sbin/start elasticsearch # for upstart/Centos 6/Ubuntu 14.04\n      systemctl start elasticsearch.service # for systemd/Centos 7/Ubuntu 16.04\n 5) Initialize/Upgrade Elasticsearch Moloch configuration\n  a) If this is the first install, or want to delete all data\n      /data/moloch/db/db.pl http://ESHOST:9200 init\n  b) If this is an update to moloch package\n      /data/moloch/db/db.pl http://ESHOST:9200 upgrade\n 6) Add an admin user if a new install or after an init\n      /data/moloch/bin/moloch_add_user.sh admin \"Admin User\" THEPASSWORD --admin\n 7) Start everything\n   a) If using upstart (Centos 6 or sometimes Ubuntu 14.04):\n      /sbin/start molochcapture\n      /sbin/start molochviewer\n   b) If using systemd (Centos 7 or Ubuntu 16.04 or sometimes Ubuntu 14.04)\n      systemctl start molochcapture.service\n      systemctl start molochviewer.service\n 8) Look at log files for errors\n      /data/moloch/logs/viewer.log\n      /data/moloch/logs/capture.log\n 9) Visit http://MOLOCHHOST:8005 with your favorite browser.\n      user: admin\n      password: THEPASSWORD from step #6\n\nAny configuration changes can be made to /data/moloch/etc/config.ini\nSee https://molo.ch/faq#moloch-is-not-working for issues\n\nAdditional information can be found at:\n  * https://molo.ch/faq\n  * https://molo.ch/settings\n\n\n\n$ sudo systemctl start elasticsearch.service\n$ sudo systemctl status elasticsearch.service\n● elasticsearch.service - Elasticsearch\n   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)\n   Active: active (running) since Wed 2020-08-05 12:03:11 UTC; 3min 19s ago\n     Docs: http://www.elastic.co\n Main PID: 24675 (java)\n    Tasks: 42\n   Memory: 1.2G\n      CPU: 38.482s\n   CGroup: /system.slice/elasticsearch.service\n           ├─24675 /usr/bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkad\n           └─24911 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller\n\nAug 05 12:03:11 vg-moloch-01 systemd[1]: Started Elasticsearch.\nAug 05 12:03:12 vg-moloch-01 elasticsearch[24675]: warning: Falling back to java on path. This behavior is deprecated. Specify JAVA_HOME\nAug 05 12:06:26 vg-moloch-01 systemd[1]: Started Elasticsearch.\n\n\n~~~~\n\n~~~~\n$ sudo /data/moloch/bin/Configure\nFound interfaces: eth0;eth1;lo\nSemicolon ';' seperated list of interfaces to monitor [eth1] eth1\nInstall Elasticsearch server locally for demo, must have at least 3G of memory, NOT recommended for production use (yes or no) [no] no\nElasticsearch server URL [http://localhost:9200]\nPassword to encrypt S2S and other things [no-default] no\nMoloch - Creating configuration files\nInstalling systemd start files, use systemctl\nMoloch - Installing /etc/logrotate.d/moloch to rotate files after 7 days\nMoloch - Installing /etc/security/limits.d/99-moloch.conf to make core and memlock unlimited\nDownload GEO files? (yes or no) [yes] yes\nMoloch - Downloading GEO files\n~~~~\n\n~~~~\na) If this is the first install, or want to delete all data\n$ sudo /data/moloch/db/db.pl http://localhost:9200 init\nIt is STRONGLY recommended that you stop ALL moloch captures and viewers before proceeding.  Use 'db.pl http://localhost:9200 backup' to backup db first.\nThere is 1 elastic search data node, if you expect more please fix first before proceeding.\nThis is a fresh Moloch install\nErasing\nCreating\nFinished\n\n\n$ sudo npm update\n\nAdd an admin user if a new install or after an init\n$ sudo /data/moloch/bin/moloch_add_user.sh admin admin admin123 --admin\nAdded\n\n$ sudo systemctl start molochcapture.service\n$ sudo systemctl status molochcapture.service\n● molochcapture.service - Moloch Capture\n   Loaded: loaded (/etc/systemd/system/molochcapture.service; disabled; vendor preset: enabled)\n   Active: failed (Result: start-limit-hit) since Wed 2020-08-05 12:09:37 UTC; 19s ago\n  Process: 25607 ExecStart=/bin/sh -c /data/moloch/bin/moloch-capture -c /data/moloch/etc/config.ini ${OPTIONS} \u003e\u003e /data/moloch/logs/capture.log 2\u003e\u00261 (code=ex\n  Process: 25595 ExecStartPre=/data/moloch/bin/moloch_config_interfaces.sh (code=exited, status=0/SUCCESS)\n Main PID: 25607 (code=exited, status=1/FAILURE)\n\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Main process exited, code=exited, status=1/FAILURE\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Unit entered failed state.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Failed with result 'exit-code'.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Service hold-off time over, scheduling restart.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: Stopped Moloch Capture.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Start request repeated too quickly.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: Failed to start Moloch Capture.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Unit entered failed state.\nAug 05 12:09:37 vg-moloch-01 systemd[1]: molochcapture.service: Failed with result 'start-limit-hit'.\n\n\n$ sudo systemctl start molochviewer.service\n$ sudo systemctl status molochviewer.service\n● molochviewer.service - Moloch Viewer\n   Loaded: loaded (/etc/systemd/system/molochviewer.service; disabled; vendor preset: enabled)\n   Active: active (running) since Wed 2020-08-05 12:09:42 UTC; 1min 0s ago\n Main PID: 25616 (sh)\n    Tasks: 12\n   Memory: 35.8M\n      CPU: 1.360s\n   CGroup: /system.slice/molochviewer.service\n           ├─25616 /bin/sh -c /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini  \u003e\u003e /data/moloch/logs/viewer.log 2\u003e\u00261\n           └─25618 /data/moloch/bin/node viewer.js -c /data/moloch/etc/config.ini\n\nAug 05 12:09:42 vg-moloch-01 systemd[1]: Started Moloch Viewer.\n\n\n$ sudo tail -n 20 /data/moloch/logs/viewer.log\nExpress server listening on port 8005 in development mode\nWed, 05 Aug 2020 12:09:50 GMT - GET /eshealth.json 200 455 bytes 29.140 ms\nWed, 05 Aug 2020 12:10:00 GMT - GET /eshealth.json 200 455 bytes 12.440 ms\nWed, 05 Aug 2020 12:10:10 GMT - GET /eshealth.json 200 455 bytes 11.167 ms\nWed, 05 Aug 2020 12:10:20 GMT - GET /eshealth.json 200 455 bytes 9.283 ms\nWed, 05 Aug 2020 12:10:30 GMT - GET /eshealth.json 200 455 bytes 30.133 ms\nWed, 05 Aug 2020 12:10:40 GMT - GET /eshealth.json 200 455 bytes 0.756 ms\nWed, 05 Aug 2020 12:10:50 GMT - GET /eshealth.json 200 455 bytes 6.628 ms\nWed, 05 Aug 2020 12:11:00 GMT - GET /eshealth.json 200 455 bytes 7.309 ms\nWed, 05 Aug 2020 12:11:11 GMT - GET /eshealth.json 200 455 bytes 11.053 ms\nWed, 05 Aug 2020 12:11:21 GMT - GET /eshealth.json 200 455 bytes 6.484 ms\nWed, 05 Aug 2020 12:11:31 GMT - GET /eshealth.json 200 455 bytes 18.125 ms\nWed, 05 Aug 2020 12:11:41 GMT - GET /eshealth.json 200 455 bytes 0.519 ms\nWed, 05 Aug 2020 12:11:51 GMT - GET /eshealth.json 200 455 bytes 11.925 ms\nWed, 05 Aug 2020 12:12:01 GMT - GET /eshealth.json 200 455 bytes 11.021 ms\nWed, 05 Aug 2020 12:12:11 GMT - GET /eshealth.json 200 455 bytes 0.596 ms\nWed, 05 Aug 2020 12:12:21 GMT - GET /eshealth.json 200 455 bytes 20.048 ms\n\n$ sudo tail -n 10 /data/moloch/logs/capture.log\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 404 http://localhost:9200/stats/stat/vg-moloch-01 0/71 0ms 1ms\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 200 http://localhost:9200/fields/field/_search?size=3000 0/13200 0ms 6ms\nAug  5 12:09:36 config.c:819 moloch_config_monitor_file(): Couldn't stat country file file /data/moloch/etc/GeoLite2-Country.mmdb error No such file or directory\nAug  5 12:09:36 main.c:202 parse_args(): WARNING: gethostname doesn't return a fully qualified name and getdomainname failed, this may cause issues when viewing pcaps, use the --host option - vg-moloch-01\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 200 http://localhost:9200/_template/sessions2_template?filter_path=**._meta\u0026include_type_name=true 0/80 4ms 10ms\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 200 http://localhost:9200/sequence/sequence/fn-vg-moloch-01 0/123 0ms 28ms\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 200 http://localhost:9200/sequence/sequence/fn-vg-moloch-01 2/177 0ms 8ms\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 404 http://localhost:9200/stats/stat/vg-moloch-01 0/71 0ms 1ms\nAug  5 12:09:36 http.c:305 moloch_http_send_sync(): 1/1 SYNC 200 http://localhost:9200/fields/field/_search?size=3000 0/13200 0ms 5ms\nAug  5 12:09:36 config.c:819 moloch_config_monitor_file(): Couldn't stat country file file /data/moloch/etc/GeoLite2-Country.mmdb error No such file or directory\n\n\n$ sudo lsof -i | grep 8005\nnode     25618        nobody   20u  IPv6  61680      0t0  TCP *:8005 (LISTEN)\nnode     25618        nobody   30u  IPv6  62609      0t0  TCP vg-moloch-01.local:8005-\u003e192.168.16.1:23377 (ESTABLISHED)\n\n$ sudo lsof -i | grep 9200\njava     24675 elasticsearch  216u  IPv6  61679      0t0  TCP localhost:9200-\u003elocalhost:59754 (ESTABLISHED)\njava     24675 elasticsearch  217u  IPv6  60036      0t0  TCP localhost:9200 (LISTEN)\njava     24675 elasticsearch  218u  IPv6  57865      0t0  TCP localhost:9200 (LISTEN)\njava     24675 elasticsearch  242u  IPv6  61130      0t0  TCP localhost:9200-\u003elocalhost:59756 (ESTABLISHED)\njava     24675 elasticsearch  243u  IPv6  61131      0t0  TCP localhost:9200-\u003elocalhost:59758 (ESTABLISHED)\njava     24675 elasticsearch  254u  IPv6  61687      0t0  TCP localhost:9200-\u003elocalhost:59760 (ESTABLISHED)\njava     24675 elasticsearch  255u  IPv6  61688      0t0  TCP localhost:9200-\u003elocalhost:59762 (ESTABLISHED)\njava     24675 elasticsearch  256u  IPv6  61689      0t0  TCP localhost:9200-\u003elocalhost:59764 (ESTABLISHED)\njava     24675 elasticsearch  257u  IPv6  61690      0t0  TCP localhost:9200-\u003elocalhost:59766 (ESTABLISHED)\njava     24675 elasticsearch  258u  IPv6  61691      0t0  TCP localhost:9200-\u003elocalhost:59768 (ESTABLISHED)\njava     24675 elasticsearch  259u  IPv6  61692      0t0  TCP localhost:9200-\u003elocalhost:59770 (ESTABLISHED)\njava     24675 elasticsearch  260u  IPv6  61693      0t0  TCP localhost:9200-\u003elocalhost:59772 (ESTABLISHED)\nnode     25618        nobody   19u  IPv4  61678      0t0  TCP localhost:59754-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   21u  IPv4  61121      0t0  TCP localhost:59756-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   22u  IPv4  61122      0t0  TCP localhost:59758-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   23u  IPv4  61123      0t0  TCP localhost:59760-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   24u  IPv4  61124      0t0  TCP localhost:59762-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   25u  IPv4  61125      0t0  TCP localhost:59764-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   26u  IPv4  61126      0t0  TCP localhost:59766-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   27u  IPv4  61127      0t0  TCP localhost:59768-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   28u  IPv4  61128      0t0  TCP localhost:59770-\u003elocalhost:9200 (ESTABLISHED)\nnode     25618        nobody   29u  IPv4  61129      0t0  TCP localhost:59772-\u003elocalhost:9200 (ESTABLISHED)\n\nbrowse \nhttp://192.168.16.9:8005\nhttp://192.168.16.9:9200/\n\n~~~~\nsmoketest moloch\n~~~~\nlive capture\n$ sudo /data/moloch/bin/./moloch-capture -r ~capture.pcap\nindex a directory\n$ sudo /data/moloch/bin/./moloch-capture -R ~/\n~~~~\n~~~~\nMemory:  4GB\nNetwork: 2 network cards, one for management, one for capture\n~~~~\nElasticsearch reqs\n~~~~\nmoloch_version: 2.1.0-1\nelasticsearch_version: 6.8.5\n\nThis is a fresh Moloch install\nCurrently using Elasticsearch version  5.6.16  which isn't supported\n * \u003c 6.7.0 is not supported\n * 7.0.x is not supported\n~~~~\nCommon bugs that may need to be addressed:\n~~~~\n\n\nThe memlock must be set to unlimited or the elasticsearch will close the moloch capture. Prep the elasticsearch machines by increasing max file descriptors and allowing memory locking. On CentOS and others this is done by adding the following to bottom of: /etc/security/limits.conf: - nofile 128000 - memlock unlimited\n\nIf this is a dedicated machine, disable swap by commenting out the swap lines in /etc/fstab and either reboot or use the swapoff command.\n\nMake sure the network card is configured correctly by increasing the ring buf to max size and turning off most of the card's features.since we want to capture what is on the network instead of what the local OS sees.\nSet ring buf size, see max with\nethtool -g eth0\n\n$ ethtool -G eth1 rx 4096 tx 4096\n$ ethtool -g eth1\nRing parameters for eth1:\nPre-set maximums:\nRX:             4096\nRX Mini:        0\nRX Jumbo:       0\nTX:             4096\nCurrent hardware settings:\nRX:             4096\nRX Mini:        0\nRX Jumbo:       0\nTX:             4096\n\n\nsee available features\n$ ethtool -k eth1\n$ sudo ethtool -K eth1 rx off tx off tso off gso off\n\n\u003chttps://github.com/sofwerx/pcas/wiki/Moloch-User-Guide\u003e\n~~~~\n\n~~~~\nPublicly available PCAP files\n\u003chttps://www.netresec.com/?page=PcapFiles\u003e\n\nSharkFest EUROPE Retrospective\n\u003chttps://sharkfesteurope.wireshark.org/retrospective\u003e\n\nhttps://molo.ch/\nhttps://github.com/aol/moloch\n~~~~\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgithubfoam%2Fmoloch-sandbox","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgithubfoam%2Fmoloch-sandbox","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgithubfoam%2Fmoloch-sandbox/lists"}