{"id":32972281,"url":"https://github.com/Netflix/inviso","last_synced_at":"2025-11-16T03:00:47.284Z","repository":{"id":19160947,"uuid":"22392595","full_name":"Netflix/inviso","owner":"Netflix","description":null,"archived":false,"fork":false,"pushed_at":"2023-05-23T06:54:02.000Z","size":213,"stargazers_count":204,"open_issues_count":14,"forks_count":64,"subscribers_count":458,"default_branch":"master","last_synced_at":"2025-07-01T23:36:16.515Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Netflix.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2014-07-29T19:28:14.000Z","updated_at":"2025-03-07T01:30:35.000Z","dependencies_parsed_at":"2024-01-06T12:24:31.820Z","dependency_job_id":"e3386133-2e5b-4bc9-ab27-5dd328de2f2d","html_url":"https://github.com/Netflix/inviso","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Netflix/inviso","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Finviso","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Finviso/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Finviso/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Finviso/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Netflix","download_url":"https://codeload.github.com/Netflix/inviso/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Netflix%2Finviso/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":284654194,"owners_count":27041729,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-11-16T02:00:05.974Z","response_time":65,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-11-13T05:00:34.993Z","updated_at":"2025-11-16T03:00:47.272Z","avatar_url":"https://github.com/Netflix.png","language":"JavaScript","readme":"![Inviso Logo](inviso-lg.png)\n\n# Overview\nInviso is a lightweight tool that provides the ability to search for Hadoop jobs, visualize the performance, and view cluster utilization.  \n\n# Design and Components\n\n**REST API for Job History:** REST endpoint to load an entire job history file as a json object.\n\n**ElasticSearch:** Search over jobs and correlate Hadoop jobs for Pig and Hive scripts.\n\n**Python Scripts:** Scripts to index job configurations into ElasticSearch for\nquerying.  These scripts can accommodate a pub/sub model for use with SQS or some\nother queuing service to better distribute the load or allow other systems to know\nabout job events.\n\n**Web UI:** Provides an interface to serach and visualize jobs and cluster data.\n\n\n# Requirements\n\n* JDK 1.7+\n* [Apache Tomcat (7+)](http://tomcat.apache.org/download-70.cgi)\n* [ElasticSearch (1.0+)](http://www.elasticsearch.org/overview/elkdownloads/)\n* Hadoop 2 Cluster\n  * Log aggregation must be enabled for task log linking to work\n  * Specific version of Hadoop may need to set in the gradle build file\n  * Some functionality is available for Hadoop 1, but requires more configuration\n\n# QuickStart\n\nInviso is easy to setup given a Hadoop cluster.  To get a quick preview, it is easiest to\nconfigure Inviso on the NameNode/ResourceManager host.\n\n1. Pull down required resources and stage them\n\n  ```bash\n  \u003e wget http://\u003cmirror\u003e/.../apache-tomcat-7.0.55.tar.gz\n  \u003e tar -xzf apache-tomcat-7.0.55.tar.gz\n  \u003e rm -r apache-tomcat-7.0.55/webapps/*\n  \u003e wget http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.tar.gz\n  \u003e tar -xzf elasticsearch-1.3.2.tar.gz\n  ```\n2. Clone the Inviso repository and build the java project\n\n  ```bash\n  \u003e git clone https://github.com/Netflix/inviso.git\n  \u003e cd inviso\n  \u003e ./gradlew assemble\n  \u003e cd ..\n  ```\n\n3. Copy WAR files and link Static Web Pages\n\n  ```bash\n  \u003e cp inviso/trace-mr2/build/libs/inviso#mr2#v0.war apache-tomcat-7.0.55/webapps/\n  \u003e ln -s `pwd`/inviso/web-ui/public apache-tomcat-7.0.55/webapps/ROOT\n  ```\n\n4. Start ElasticSearch and create Indexes\n\n  ```bash\n  \u003e ./elasticsearch-1.3.2/bin/elasticsearch -d\n  \u003e curl -XPUT http://localhost:9200/inviso -d @inviso/elasticsearch/mappings/config-settings.json\n    {\"acknowledged\":true}\n  \u003e curl -XPUT http://localhost:9200/inviso-cluster -d @inviso/elasticsearch/mappings/cluster-settings.json\n    {\"acknowledged\":true}\n  ```\n\n5. Start Tomcat\n\n  ```bash\n  \u003e ./apache-tomcat-7.0.55/bin/startup.sh\n  ```\n\n6. Build virtual environment and index some jobs\n\n  ```bash\n  \u003e virtualenv venv\n  \u003e source venv/bin/activate\n  \u003e pip install -r inviso/jes/requirements.txt\n  \u003e cd inviso/jes/\n  \u003e cp settings_default.py settings.py\n  \u003e python jes.py\n  \u003e python index_cluster_stats.py\n\n  #Run in a cron or loop\n  \u003e while true; do sleep 60s; python jes.py; done\u0026\n  \u003e while true; do sleep 60s; python index_cluster_stats.py; done\u0026\n  ```\n\n7. Navigate to http://hostname:8080/\n\n# QuickStart - Docker Version\n\nAn alternate way of starting the inviso project would be via docker.  If you already have docker installed, you can run the following command:\n\n```\ndocker run -d -p 8080:8080 savaki/inviso\n```\n\nThis will launch inviso in your container running on port 8080.\n\nEnjoy!\n\n","funding_links":[],"categories":["DevOps Tools","Tools","Packaging, Provisioning and Monitoring","大数据"],"sub_categories":["Winetricks","Objective-C Tools, Libraries, and Frameworks"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNetflix%2Finviso","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FNetflix%2Finviso","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FNetflix%2Finviso/lists"}