{"id":18035846,"url":"https://github.com/echo-devim/bugzilladatamining","last_synced_at":"2025-04-04T23:25:38.660Z","repository":{"id":82335310,"uuid":"59603731","full_name":"echo-devim/BugzillaDataMining","owner":"echo-devim","description":"This repository contains some datasets extracted from bugzilla, software to analyze them and some results.","archived":false,"fork":false,"pushed_at":"2016-07-22T17:32:21.000Z","size":41508,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-02-10T08:12:26.973Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/echo-devim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-05-24T19:50:21.000Z","updated_at":"2022-03-18T02:34:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"93dc893d-a484-4823-9810-4fc11ca72773","html_url":"https://github.com/echo-devim/BugzillaDataMining","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/echo-devim%2FBugzillaDataMining","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/echo-devim%2FBugzillaDataMining/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/echo-devim%2FBugzillaDataMining/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/echo-devim%2FBugzillaDataMining/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/echo-devim","download_url":"https://codeload.github.com/echo-devim/BugzillaDataMining/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247263983,"owners_count":20910505,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-30T12:10:28.781Z","updated_at":"2025-04-04T23:25:38.646Z","avatar_url":"https://github.com/echo-devim.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BugzillaAnalyzer\n\nThis is a project developed to analyze bugs reported with Bugzilla. I used Spark and HBase.\n\n## Architecture\n\nThe system is formed by two parts.\n\n#### Analyzer\n\nThis component takes a dataset of bugs as input and makes some analysis saving the results into HBase. Each row in the database is associated with a specific dataset and each column contains a result (e.g. number of total bugs, number of security bugs, etc.).\n\n#### Reporter\n\nThis component takes all the partial results from HBase (i.e. from all the datasets) and creates a report with global analysis. The report will be saved on the HDFS in html format.\n\n------------\n\nIf you have few really big datasets you can use the following architecture.\n![arch1](https://raw.githubusercontent.com/echo-devim/BugzillaDataMining/master/arch1.jpg)\n\nOtherwise if you have a lot of small datasets you can use this variant.\n![arch2](https://raw.githubusercontent.com/echo-devim/BugzillaDataMining/master/arch2.jpg)\n\n\n## Datasets\n\n* gnome_all.csv.bz2 (uncompressed: 83.4 MB, bugs from 2002 to 2016, 555050 bugs )\n* kde_all.csv.bz2 (uncompressed: 47.7 MB, bugs from 2002 to 2016, 339822 bugs )\n* xfce_all.csv.bz2 (uncompressed: 1.3 MB, bugs from 2004 to 2016, 10198 bugs )\n* names.txt (list of most common Italian's names).\n\nI got all the information using:\n\n\u003e https://SITE.bugzilla.org/buglist.cgi?bug_status=UNCONFIRMED\u0026bug_status=CONFIRMED\u0026bug_status=ASSIGNED\u0026bug_status=REOPENED\u0026bug_status=RESOLVED\u0026bug_status=NEEDSINFO\u0026bug_status=VERIFIED\u0026bug_status=CLOSED\u0026chfieldfrom=2003-06-29\u0026chfieldto=2004-05-31\u0026limit=20000\u0026query_format=advanced\u0026ctype=csv\u0026human=1\u0026order=Last+Changed\n\nSlicing the time window you can download all the bugs, the maximum number of bugs retrieved by a single query is 10000 (also if you set limit with a value greater than it).\n\nCSV format:\n\n*\"Bug ID\",\"Product\",\"Component\",\"Assignee\",\"Status\",\"Resolution\",\"Summary\",\"Changed\"*\n\n## Requirements\n\nYou need the following library to check the language used in the reports.\n\n[language-detection library](https://github.com/echo-devim/language-detection)\n\nYou can use HDFS, but you can run the project also as standalone on your local machine. In this last case, use the original language detection library.\n\n## Execution\n\n1. Start HBase (e.g. `$ hbase-1.2.1/bin/start-hbase.sh`)\n2. Run the Analyzer with `$ spark-1.6.1-bin-hadoop2.4/bin/spark-submit --class \"main.Analyzer\" --master local[*] --driver-class-path \"$(/opt/shared/hbase-1.2.1/bin/hbase classpath):/opt/shared/libs/langdetect-hdfs.jar:/opt/shared/libs/jsonic-1.2.7.jar\" /opt/shared/project/buganalyzer.jar file:///opt/shared/project/datasets/xfce_all.csv /opt/shared/project/languages/profiles`\n3. Run the Reporter with `$ spark-1.6.1-bin-hadoop2.4/bin/spark-submit --class \"main.Reporter\" --master local[*] --driver-class-path \"$(/opt/shared/hbase-1.2.1/bin/hbase classpath):/opt/shared/libs/langdetect-hdfs.jar:/opt/shared/libs/jsonic-1.2.7.jar\" /opt/shared/project/buganalyzer.jar hdfs://172.97.0.9:9000/user/root/output`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fecho-devim%2Fbugzilladatamining","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fecho-devim%2Fbugzilladatamining","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fecho-devim%2Fbugzilladatamining/lists"}