{"id":19681740,"url":"https://github.com/collab-uniba/pysoreputation","last_synced_at":"2026-05-16T03:02:32.551Z","repository":{"id":37747311,"uuid":"200202001","full_name":"collab-uniba/pySOreputation","owner":"collab-uniba","description":"A script / RESTful web service to get an estimate of user reputation on Stack Overflow at a given point in time.","archived":false,"fork":false,"pushed_at":"2024-09-04T00:05:21.000Z","size":2517,"stargazers_count":0,"open_issues_count":5,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-10T05:40:57.120Z","etag":null,"topics":["mining-software-repositories","reputation","restful-webservices","stackoverflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/collab-uniba.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-02T08:58:28.000Z","updated_at":"2024-06-21T15:18:19.000Z","dependencies_parsed_at":"2023-01-25T09:15:06.541Z","dependency_job_id":"dc055d42-5621-47cf-95b3-20f596f7cebd","html_url":"https://github.com/collab-uniba/pySOreputation","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/collab-uniba%2FpySOreputation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/collab-uniba%2FpySOreputation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/collab-uniba%2FpySOreputation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/collab-uniba%2FpySOreputation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/collab-uniba","download_url":"https://codeload.github.com/collab-uniba/pySOreputation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240990810,"owners_count":19889951,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mining-software-repositories","reputation","restful-webservices","stackoverflow"],"created_at":"2024-11-11T18:08:40.868Z","updated_at":"2026-05-16T03:02:32.476Z","avatar_url":"https://github.com/collab-uniba.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pySOreputation [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.3687265.svg)](https://doi.org/10.5281/zenodo.3687265)\nA service to compute an approximation of the reputation gained by a [Stack Overflow](https://stackoverflow.com/) user up to a given date.\n\n## License\nThis software is released under the MIT License.\n\n## DISCLAIMER\n\nThe extracted reputation is only an estimate (with an avg error \u003c10%). Also, note that the scripts are already compliant with the [change](https://stackoverflow.blog/2019/11/13/were-rewarding-the-question-askers) in the reputation system that doubles the reputation points earned from getting an upvote to a question.\n\nStill, the following rules are not considered:\n- Suggested edit is accepted: +2 (up to +1000 total per user)\n- Bounty awarded to your answer: + full bounty amount\n- One of your answers is awarded a bounty automatically: + half of the bounty amount (see more details about how bounties work)\n- Example you contributed to is voted up: +5\n- Proposed change is approved: +2\n- First time an answer that cites documentation you contributed to is upvoted: +5\n- You place a bounty on a question: - full bounty amount\n- One of your posts receives 6 spam or offensive flags: -100\n\n## Fair Use Policy\nPlease, cite the following works if you intend to use our tool for your own research:\n\n\u003eF. Calefato, F. Lanubile, N. Novielli. “[Moving to Stack Overflow: Best-Answer Prediction in Legacy Developer Forums.](http://collab.di.uniba.it/fabio/wp-content/uploads/sites/5/2014/05/a13-calefato.pdf)” In *Proc. 10th Int’l Symposium on Empirical Softw. Eng. and Measurement (ESEM’16)*, Ciudad Real, Spain, Sept. 8-9, 2016, DOI:10.1145/2961111.2962585.\n\n```\n@inproceedings{Calefato_esem2016,\n author = {Calefato, Fabio and Lanubile, Filippo and Novielli, Nicole},\n title = {Moving to Stack Overflow: Best-Answer Prediction in Legacy Developer Forums},\n booktitle = {Proc. 10th ACM/IEEE Int'l Symposium on Empirical Software Engineering and Measurement}, \n series = {ESEM '16},\n year = {2016},\n isbn = {978-1-4503-4427-2},\n location = {Ciudad Real, Spain},\n pages = {13:1--13:10},\n articleno = {13},\n numpages = {10},\n url = {http://doi.acm.org/10.1145/2961111.2962585},\n doi = {10.1145/2961111.2962585},\n publisher = {ACM},\n} \n```\n\u003eF. Calefato, F. Lanubile, M.C. Marasciulo, N. Novielli. MSR Challenge: “[Mining Successful Answers in Stack Overflow.](http://collab.di.uniba.it/fabio/wp-content/uploads/sites/5/2014/05/MSR_2015_calefato_et_al.pdf)” In *Proc. 12th IEEE Working Conf. on Mining Software Repositories (MSR 2015)*, Florence, Italy, May 16-17, 2015.\n\n```\n@inproceedings{Calefato_msr2015,\n author = {Calefato, Fabio and Lanubile, Filippo and Marasciulo, Maria Concetta and Novielli, Nicole},\n title = {Mining Successful Answers in Stack Overflow},\n booktitle = {Proc. 12th Working Conf. on Mining Software Repositories},\n series = {MSR '15},\n year = {2015},\n isbn = {978-0-7695-5594-2},\n location = {Florence, Italy},\n pages = {430--433},\n numpages = {4},\n url = {http://dl.acm.org/citation.cfm?id=2820518.2820579},\n publisher = {IEEE Press},\n}\n```\n\n## Installation\nYou will need to install [Git LFS](https://git-lfs.github.com/) extension to check out this project. Once installed and initialized, simply run:\n\n```bash\n$ git lfs clone https://github.com/collab-uniba/pySOreputation.git\n$ cd pySOreputation\n$ conda create --name .venv37 python=3.7 \n$ conda activate .venv37\n$ pip install -r requirements.txt\n```\n\n## Database Setup\nIf you want to run your script locally or deploy the web service on your own server, you need to setup a MySQL database. From this point forward, we assume that you have already imported the [SO dump](https://archive.org/download/stackexchange) to a local MySQL database (there are several scripts that you can easily adapt to your purpose; see [here](https://gist.github.com/megansquire/877e028504c92e94192d) for example).\n\nCd to `./db-setup/` folder and run the sql script `setup_sequential.sh`:\n\n```$ bash setup_sequential.sh```\n\n#### NOTE:\nBefore running, edit the first lines of the file to change the following information:\n```\n#!/bin/bash\nMYSQL_USER=root\nMYSQL_PASS=secret\nMYSQL_SO_DB=stackoverflow\n...\n```\nThis script will create several table/views and indexes to speed up the querying process.\n\n## How to Run\nThe scripts referenced in this section must be edited prior to execution, in order to customize the following variables:\n- MySQL username\n- MySQL password\n- SO database name\n\n### Script: Sequential version\n#### Requirements\n- Python 3.7+\n- PyMySQL version 0.7.9\n\n#### Usage\nFrom command line run:\n\n```bash\n$ python reputation.py (--uid ID | --file /path/to/file.txt) --date `YYYY-MM-DD`\n```\nwhere:\n* `-u | --uid` (*mutually exclusive*) is the numeric id associate with a Stack Overflow user account (e.g., `1315221`)\n* `-f | --file` (*mutually exclusive*) is the full path to a text file containing a list (one per line) of user ids\n* `-d | --date` (*mandatory*) is the date at which estimate the reputation, formatted as `YYYY-MM-DD`\n\nPlease, note that `--uid` and `--file` options are mutually exclusive, if provided together the `--uid` supersede the other.\n\n### Script: Parallel version\nUnlike the sequential version, the parallel one does not use a MySQL database. Instead, it relies on several CSV files that are created *ad hoc* by running the script `setup_parallel.sh`, which will retrieve the content from the database and pre-process it for increasing the speed.\n\n#### Requirements\n- Python 3.7+\n- Pandas \n- concurrent.futures\n\n#### Usage\nFrom the command line run:\n```bash\n$ python main.py (--uid ID | --file /path/to/file.txt) --date `YYYY-MM-DD`\n```\nwhere:\n* `-u | --uid` (*mutually exclusive*) is the numeric id associate with a Stack Overflow user account (e.g., `1315221`)\n* `-f | --file` (*mutually exclusive*) is the full path to a text file containing a list (one per line) of user ids\n* `-d | --date` (*mandatory*) is the date at which estimate the reputation, formatted as `YYYY-MM-DD`\n\nPlease, note that `--uid` and `--file` options are mutually exclusive, if provided together the `--uid` supersede the other.\n\n## Web Service \nThe web service is built upon the parallel version, so make sure that you have generated the CSV files by running the script `setup_parallel.sh` before starting it.\n\n#### Requirements\n- Python 3.4+\n- Flask \n- uWSGI \n\n#### How to run\nBefore first run, execute once the setup file:\n```bash\nsh setup_ws.sh\n```\n\n#### Server\nFrom the command line, run the script `start_ws.sh`. Then, wait for the application to start up (depending on your system specs, it will take about 10 min).\n\n##### Installation as a service\nIn order for the web service to start up with the system, follow these steps.\n\n1. Copy `pyso-ws.conf` to `/etc/init`\n2. Edit line 13 of copied file to point to the actual path of installation\n`cd /path/to/pySOreputation/SOWebService/StackOverflowServer`\n3. The configuration assumes you are running a conda environment called `.venv37`. Hence, edit `start_ws.sh` and uncomment line 4 `#source /anaconda3/etc/profile.d/conda.sh` if you find the service not to boot and the script to raise the error \n`CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.`\n4. Check if the syntax is correct\n```bash\n$ init-checkconf /etc/init/pyso-ws.conf\n```\n5. Start the service manually:\n```bash\n$ sudo service pyso-ws start \n```\n6. The service will start in about 10 minutes. To check the status:\n```bash\n$ sudo service pyso-ws status\n```\n7. To check the execution log:\n```bash\n$ sudo tail -f /var/log/pyso-ws.log\n```\n\n#### Client\nTo launch the client, execute the `start_client.sh` script. The client will be locally accessible at this address [http://127.0.0.1:18000](http://127.0.0.1:18000).\n\nYou can test a demo version of the service at this address: [http://172.30.8.16:18000](http://172.30.8.16:18000). \n*Note that the demo uses the official dump of 2019-08-31, so do not query anything after that date*.\n\nTo invoke the service from the command line, execute the following:\n\n```bash\ncurl \\\n  --header \"Content-type: application/json\" \\\n  --request POST \\\n  --data '{\"user_id\": \"1315221\", \"date\": \"2019-08-31\"}' \\\n  http://hostaddress:19000/estimate\n```\n\nThe returned json is formatted as follows:\n\n```json\n{\n   \"1315221\": {\n      \"estimated\": 35,\n      \"name\": \"bateman\",\n      \"registered\": 38\n   }\n}\n```\n\nwhere `name` is the name chosen by the user, `estimated` is the estimated reputation whereas `registered` \nis the reputation stored in the database as of the dump creation. \n\n##### Installation as a service\nIn order for the web client to start up with the system, follow the steps below.\nAlso, note that the client assumes that the server is executed on the same host on port 19000. If not, edit file `config.py` and change the following variables as needed:\n ```python\nWS_HOST = \"localhost\"\nWS_PORT = 19000\n``` \n\n1. Copy `pyso-client.conf` to `/etc/init`\n2. Edit line 13 of copied file to point to the actual path of installation\n`cd /path/to/pySOreputation/SOWebService/StackOverflowServer`\n3. The configuration assumes you are running a conda environment called `.venv37`. Hence, edit `start_client.sh` and uncomment line 4 `#source /anaconda3/etc/profile.d/conda.sh` if you find the service not to boot and the script to raise the error `CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.`\n4. Check if the syntax is correct\n```bash\n$ init-checkconf /etc/init/pyso-client.conf\n```\n5. Start the service manually:\n```bash\n$ sudo service pyso-client start \n```\n6. The service will start immediately. To check the status:\n```bash\n$ sudo service pyso-client status\n```\n7. To check the execution log:\n```bash\n$ sudo tail -f /var/log/pyso-client.log\n```\n\n#### Screenshots\n\n![An example of input](images/input.png)\n\n*Fig. 1: An example of input*\n\n![The result](images/result.png)\n\n*Fig. 2: And the returned result*\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcollab-uniba%2Fpysoreputation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcollab-uniba%2Fpysoreputation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcollab-uniba%2Fpysoreputation/lists"}