{"id":13701618,"url":"https://github.com/endgameinc/binarypig","last_synced_at":"2025-05-04T21:31:10.233Z","repository":{"id":9811596,"uuid":"11793502","full_name":"endgameinc/binarypig","owner":"endgameinc","description":"Scalable Binary Data Extraction in Hadoop","archived":false,"fork":false,"pushed_at":"2014-07-14T14:41:48.000Z","size":962,"stargazers_count":143,"open_issues_count":9,"forks_count":45,"subscribers_count":51,"default_branch":"master","last_synced_at":"2024-11-13T07:36:16.386Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/endgameinc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-07-31T14:59:27.000Z","updated_at":"2024-09-16T13:35:35.000Z","dependencies_parsed_at":"2022-09-05T05:41:12.004Z","dependency_job_id":null,"html_url":"https://github.com/endgameinc/binarypig","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fbinarypig","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fbinarypig/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fbinarypig/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/endgameinc%2Fbinarypig/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/endgameinc","download_url":"https://codeload.github.com/endgameinc/binarypig/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252403919,"owners_count":21742463,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T20:01:52.084Z","updated_at":"2025-05-04T21:31:08.368Z","avatar_url":"https://github.com/endgameinc.png","language":"JavaScript","readme":"BinaryPig\n========\n\nMalware Processing and Analytics over Pig, Exploration through Django, Twitter Bootstrap, and Elasticsearch\n\n## Pig Libaries and Scripts\n\n### Building BinaryPig JAR\n\n    ./build.sh\n\n### Installing dependencies for binarypig modules\n\nSee installation docs in \"docs\" directory.\n\n### Using binarypig\n\nSee pig scripts in \"examples\" directory.\n\n## Webapp\n\n### Installing the binarypig webapp\n\n    cd webapp\n\n    # create a local settings file\n    cp project/local_settings.py.example project/local_settings.py\n    vi project/local_settings.py\n\n    # create your mysql DB\n    mysql\n    mysql\u003e create database binarypig;\n\n    virtualenv env\n    . env/bin/activate\n    pip install -r requirements.txt\n\n    # initialize the database\n    ./manage.py syncdb\n    ./manage.py migrate\n\n### Running the binarypig webapp\n\n    ./manage.py runserver 0.0.0.0:8000\n\n## Issues\n\nSome issues we encountered when running python based binarypig pig jobs\n\nOn Centos, if you have SELinux running and you disable it, you must restart arbtd or many/all of\nthe python processes will hang and provide no output.\n\n[Reference](http://stackoverflow.com/questions/13790475/python-wont-exit-when-called-with-absolute-path-from-cron-or-subshell)\n\n## Getting up and Running with Vagrant\n\nThis is a mini howto on getting Binary Pig up and running on an Ubuntu 14.04 VM running over Vagrant.\n\nVersions\n - Ubuntu 14.04\n - Hadoop 1.2.1\n - Pig 0.12.1\n - Elasticsearch 1.2.1\n\nSteps:\n\n    # From workstation\n    $ git clone \u003crepo\u003e\n    $ cd binarypig\n    $ vagrant up\n    $ vagrant ssh\n\n    # now logged into VM\n    $ sudo su - \n    $ cd /vagrant/\n    $ ./install-ubuntu-vagrant.sh\n\n    # press \u003cENTER\u003e when asked\n\n    # agree to Oracle Java license when asked\n    \n    # type \"yes\" when asked if you want to ssh into localhost\n\n    # enter password for root user of mysql when asked (MySQL installation)\n\n    # enter password for root user of mysql when asked (Django app install)\n\n    # walk through the django admin user creation:\n    #   You just installed Django's auth system, which means you don't have any superusers defined.\n    #   Would you like to create one now? (yes/no): yes\n    #   Username (leave blank to use 'root'): \n    #   E-mail address: YOU@gmail.com\n    #   Password: \n    #   Password (again): \n    #   Superuser created successfully.\n\nNow run the examples (Note: this is still a **Work in Progress**).  This will launch a series of pig jobs that\nexecute various BinaryPig scripts from the examples directory.\n\n    # still from VM\n    $ cd /vagrant/\n    $ ./run_examples.sh\n\n\n## License\n\n![Endgame, Inc.](http://www.endgame.com/images/navlogo.png)\n\nLicensed under the Apache 2.0 license\n\nCopyright 2013 [Endgame, Inc.](http://www.endgame.com/)\n\n## Contributors\n\n - [Jason Trost](https://github.com/jt6211/)\n - [Telvis Calhoun](https://github.com/telvis07/)\n - [Zach Hanif](https://github.com/zhanif3/)\n","funding_links":[],"categories":["Big Data","JavaScript"],"sub_categories":["Development"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fendgameinc%2Fbinarypig","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fendgameinc%2Fbinarypig","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fendgameinc%2Fbinarypig/lists"}