{"id":16742333,"url":"https://github.com/clemsos/mitras","last_synced_at":"2025-03-16T00:25:32.026Z","repository":{"id":140975607,"uuid":"11944161","full_name":"clemsos/mitras","owner":"clemsos","description":"Investigate social dynamics of memes based on Sina Weibo","archived":false,"fork":false,"pushed_at":"2014-05-04T14:24:08.000Z","size":14447,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-22T13:25:30.821Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clemsos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-08-07T07:34:03.000Z","updated_at":"2020-02-28T13:14:07.000Z","dependencies_parsed_at":"2023-03-13T10:39:36.652Z","dependency_job_id":null,"html_url":"https://github.com/clemsos/mitras","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clemsos%2Fmitras","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clemsos%2Fmitras/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clemsos%2Fmitras/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clemsos%2Fmitras/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clemsos","download_url":"https://codeload.github.com/clemsos/mitras/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243809515,"owners_count":20351379,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T01:23:24.409Z","updated_at":"2025-03-16T00:25:31.992Z","avatar_url":"https://github.com/clemsos.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mitras : Mining memes\n\nMitras is a set of scripts used to detect, analyse and visualize memes the Chinese microblog Sina Weibo. You will need the Weiboscope dataset prepared by HKU\nIt has been developed by Clément Renaud for his phD research.\n\n\n## Mining workflow : \nThe different workflows used in this research are documented in iPython notebook in the \n```/doc``` folder.\n\n* ```es_*``` : plain-text search and mining with ElasticSearch and Kibana\n* ```hashtags_*``` : build and analyze a corpus of all hashtags in the datasets\n* ```pm_*``` : meme detection clustering algorithm using protomemes [(Ferrara, 2013)](http://www.emilio.ferrara.name/2013/08/01/clustering-memes-in-social-media/)\n\nFor visualisation, we use Matplotlib, d3js, Networkx and Gephi.\n\n## Data : the Weiboscope  corpus \nTo create this project we will use the data provided by the project Weiboscope from HKU University, JMSC - [link](http://147.8.142.179/datazip/). The dataset contains sample data from 52 weeks of 2012 from more than 350,000 Chinese microbloggers who have more than 1,000 followers (Fu, Chan, Chau, 2013 ; Fu, Chau, 2013).\n\nNote : this data has been anonymized\n\n**Data Set Statistics:**\n\n    * Number of weibo messages: 226841122\n    * Number of deleted messages: 10865955\n    * Number of censored ('Permission Denied') messages: 86083\n    * Number of unique weibo users: 14387628\n    * 57 files, 18G\n\n**Download \u0026 Prepare Data**\n\n    # to download the data\n    bash bin/get_raw_data.sh\n\n    #  Downloaded: 57 files, 18G in 6h 42m 3s (803 KB/s)\n\n    # move the files to the data folder\n    mv 147.8.142.179/datazip data/datazip\n    rm -R 147.8.142.179\n\n    # remove zip files\n    ls data/datazip/*zip | xargs -i rm {} \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclemsos%2Fmitras","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclemsos%2Fmitras","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclemsos%2Fmitras/lists"}