{"id":13697287,"url":"https://github.com/uwdata/termite-data-server","last_synced_at":"2025-04-08T01:33:37.656Z","repository":{"id":12924358,"uuid":"15601934","full_name":"uwdata/termite-data-server","owner":"uwdata","description":"Data Server for Topic Models","archived":false,"fork":false,"pushed_at":"2023-04-06T15:00:02.000Z","size":26814,"stargazers_count":121,"open_issues_count":15,"forks_count":46,"subscribers_count":31,"default_branch":"master","last_synced_at":"2025-03-23T04:11:27.728Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uwdata.png","metadata":{"files":{"readme":"README-ScatterPlot1.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-01-03T05:21:34.000Z","updated_at":"2024-08-19T15:36:34.000Z","dependencies_parsed_at":"2024-11-06T07:39:13.953Z","dependency_job_id":null,"html_url":"https://github.com/uwdata/termite-data-server","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Ftermite-data-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Ftermite-data-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Ftermite-data-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uwdata%2Ftermite-data-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uwdata","download_url":"https://codeload.github.com/uwdata/termite-data-server/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247760846,"owners_count":20991531,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T18:00:55.141Z","updated_at":"2025-04-08T01:33:35.962Z","avatar_url":"https://github.com/uwdata.png","language":"Python","funding_links":[],"categories":["Visualizations"],"sub_categories":["Embedding based Topic Models"],"readme":"Termite Explorer\n============================\n\nTermite Explorer is a visual tool built in an MVC framework that allows users to view results of their topic models and identify the individual documents.  \n\nThe tool is comprised of two components:\n\n* Interactive scatterplot: This chart plots a point for each document, where the x-axis is a selected covariate based on the document's metadata, and the y-axis is the model's estimated proportion of a selected topic.  It also provides the option of coloring each point.\n* Document Viewer: A selected document's corpus is shown in the document viewer on the right.\n\nThese two charts work in tandem in that any document can be clicked to be viewed in the document viewer.  Each portion of the visualization is an independent View that has a data Model and uses a charting function to render the visual components.\n\n\nScatterplot\n--------------\n\nThe scatterplot is an instantiation of the view [ScatterView](https://github.com/mkfreeman/termite-dev/blob/master/js/ScatterView.js) and has a data model [ScatterModel](https://github.com/mkfreeman/termite-dev/blob/master/js/ScatterModel.js).  The chart layout and representation is powered by the [Scatter](https://github.com/mkfreeman/termite-dev/blob/master/js/scatter.js) function.\n\nThe ScatterView formats the data from the ScatterModel and passes it to the Scatter function.  The Scatter function expects is powered by an array of objects (each corresponding to a single data point), each with the following data attributes:\n\n| Variable      | Type          | Functionality |\n| ------------- |:-------------:| :-----|\n| id    \t\t\t\t| integer\t\t| The value is used to identify the document which is clicked or hovered over|\n| x\t\t\t   \t\t\t| numeric    \t| This dictates the position along the x-axis |\n| y \t\t\t\t\t| numeric (0 \u003c y \u003c 1)| This dictates the position along the y-axis, and is bound by 0 and 1.|\n| color (optional)\t\t| string (color)| This assigns the color value to the point, used when color scale is categorical |\n| colorValue (optional) | numeric       | This is used to calculate the color if the color range is continuous.|\n\n\nThe ScatterModel (the data for ScatterView) prepares data for both the ScatterView and ControlsView.  The data is currently accessed from static files, and assigned to static variables (line 84):\n```javascript\n// Documents is defined in data as response from http://termite.jcchuang.org/poliblogs_stm/corpus/Documents?format=json\u0026docLimit=5000\n// proportions is defined in data as response from http://termite.jcchuang.org/poliblogs_stm/lda/DocTopicMatrix?format=json\u0026docLimit=5000\n\nScatterModel.prototype.loadData = function(callback) {\t\n\tthis.rawData = Documents.Documents\n\tthis.rawProportions = proportions.DocTopicMatrix\n\tthis.prep(callback)\n}\n```\n\nTopic proportions are then filtered down for the specific topic (line 38, ScatterModel.js):\n\n```javascript\n// Prep proportions\n\t\tthis.topicProportions = {}\n\t\tthis.rawProportions.filter(function(d){return d.topic == that.get('topic')}).map(function(d) {that.topicProportions[d.docID] = d.value})\n\n```\n\nThese values are then prepped into a data object for the view (line 54, ScatterModel.js):\n```javascript\n// Prep Data for View\nthis.data = this.rawData.map(function(d,i) {\n\tvar prop = that.topicProportions[d.DocID] == undefined? 0 : that.topicProportions[d.DocID]\n\tvar content = d.DocContent\n\tvar xVar = that.get('xVariable')\n\tvar colorVar = that.get('colorVariable')\n\tvar colorValue = colorVar == 'none' ? 'none' : d[colorVar]\n\tvar groupVar = that.get('group1')\n\tvar xLabel = d[xVar]\n\tvar xValue = that.get('xVariableType') !='integer' ? that.xLabels.indexOf(xLabel) : xLabel\n\tvar groupValue =d[groupVar]\n\treturn {id:i, \n\t\tproportion:prop,\n\t\tcontent:content, \n\t\txVar:xVar, \n\t\txValue:xValue,\n\t\tcolorVar:colorVar, \n\t\tcolorValue:colorValue,\n\t\txLabel:xLabel,\n\t}\n})\n```\n\nThere is only minor object manipulation within the ScatterView before passing to the Scatter function (line 59, ScatterView.js):\n```javascript\nprepData:function() {\n\tvar that = this\n\tthis.chartData = []\n\tthis.model.data.map(function(d,i) {\n\t\tvar color =  'blue'\n\t\tthat.chartData.push({\n\t\t\tid:d.id, \n\t\t\tx:isFinite(d.xValue) ? Number(d.xValue) : d.xValue, \n\t\t\tcolorValue:d.colorValue,\n\t\t\ty:Number(d.proportion), \n\t\t\tcolor:color\n\t\t})\n\t})\t\n},\n```\n\nTextBox\n--------------\nThe document body is an instantiation of the [TextBoxView](https://github.com/mkfreeman/termite-dev/blob/master/js/TextBoxView.js) which uses the [TextBox](https://github.com/mkfreeman/termite-dev/blob/master/js/TextBox.js) function to render the corpus.\n\nThe TextBoxView formats the following variables to pass to the TextBox function\n\n| Variable      | Type          | Functionality |\n| ------------- |:-------------:| :-----|\n| id    \t\t\t\t| integer\t\t| The value is used to identify the document which is currently selected.|\n| text\t\t\t   \t\t| string    \t| This is the document body that renders as the text on the screen. |\n| width\t\t\t\t\t| numeric \t    | This is the pixel width of the TextBox|\n| height\t\t\t\t| numeric \t\t| This is the pixel height of the TextBox |\n| position\t\t\t\t| numeric       | Object with attributes \"left\" and \"top\" for positioning of the TextBox|\n| container\t\t\t\t| string        | Wrapper div on the page that the textbox will be appended to.|\n\n\nThe formatting is done here (line 27, TextBoxView.js):\n```javascript\nTextBoxView.prototype.loadSettings = function() {\n\treturn {\n\t\ttext:this.viewText,\n\t\tcontainer:this.el.id, \n\t\tid:this.textid,\n\t\twidth:this.options.width, \n\t\theight:this.options.height, \n\t\tposition:this.options.position\n\n\t}\n}\n\n```\nThe TextBoxView is also powered by the ScatterModel as the data, and isolates the corpus by referencing the current docNum attribute of the ScatterModel (line 11, TextBoxView.js)\n```javascript\nvar textId = this.model.get('docNum')\nthis.viewText = this.model.data[textId].content\n```\n\nControls\n--------------\nThe Controls (Topic, X Variable, and Color) options are powered by the ControlsView and use the Controls file to render the menus.  This data for the Controls is also the ScatterModel object.\n\n**Topic**\n\nThe data for the topic menu are prepared in the ScatterModel object (line 27, ScatterModel.js):\n\n```javascript\n// topTerms is defined as response from: http://termite.jcchuang.org/poliblogs_stm/lda/TopTerms?format=json\n// Format terms\ntopTerms.TopTerms.map(function(d) {\n\tthat.terms[d.topic] = that.terms[d.topic] == undefined ? [] : that.terms[d.topic]\n\tthat.terms[d.topic].push({term:d.term, value:d.value})\n})\n\n// Sort terms\nd3.keys(that.terms).map(function(d) {\n\tthat.terms[d].sort(function(a,b) {return a.value - b.value})\n})\n\n```\n\n**X Variable and Color options**\n\nThe X Variable and Color menu data (options) is currently set in the ScatterModel (line 21, ScatterModel.js):\n```javascript\n// Documents is defined in data as response from http://termite.jcchuang.org/poliblogs_stm/corpus/Documents?format=json\u0026docLimit=5000\nthat.variables = Documents.Metadata.filter(function(d) {return d.name.indexOf(\"Doc\")==-1})\n```\n\nAll views are instantiated in the index.html once the data for the model has loaded (line 68, index.html)\n```javascript\nsm.loadData(function() {\n\tcv = new ControlsView()\n\tsv = new ScatterView({model:sm, el:'#main', chartid:'scatter-' + d, textid:'text-' + d})\n\ttv = new TextBoxView({model:sm, el:'#main', textid:'text-' + d})\n})\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuwdata%2Ftermite-data-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuwdata%2Ftermite-data-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuwdata%2Ftermite-data-server/lists"}