{"id":20180364,"url":"https://github.com/ppcamp/sigaa-tcc-scraping-analysis","last_synced_at":"2025-05-07T02:30:38.103Z","repository":{"id":40955624,"uuid":"187748948","full_name":"ppcamp/sigaa-tcc-scraping-analysis","owner":"ppcamp","description":"Sigaa's system scrapping and analysis. This code belongs to an article published in the IEEE transactions on education","archived":true,"fork":false,"pushed_at":"2022-08-23T18:32:35.000Z","size":16836,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"article","last_synced_at":"2025-03-03T05:43:41.438Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://ieeexplore.ieee.org/document/9792441","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ppcamp.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null}},"created_at":"2019-05-21T02:41:36.000Z","updated_at":"2024-05-19T17:46:07.000Z","dependencies_parsed_at":"2022-09-10T17:00:17.614Z","dependency_job_id":null,"html_url":"https://github.com/ppcamp/sigaa-tcc-scraping-analysis","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppcamp%2Fsigaa-tcc-scraping-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppcamp%2Fsigaa-tcc-scraping-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppcamp%2Fsigaa-tcc-scraping-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppcamp%2Fsigaa-tcc-scraping-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ppcamp","download_url":"https://codeload.github.com/ppcamp/sigaa-tcc-scraping-analysis/tar.gz/refs/heads/article","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252801260,"owners_count":21806283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T02:30:17.787Z","updated_at":"2025-05-07T02:30:37.436Z","avatar_url":"https://github.com/ppcamp.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Student's competency analisis: An graph approach\r\n\r\nThis repository contains some codes used in the article that will be published later.\r\n\r\n---\r\n\r\n- [Student's competency analisis: An graph approach](#students-competency-analisis-an-graph-approach)\r\n  - [How it works?](#how-it-works)\r\n  - [How to run](#how-to-run)\r\n  - [Documentation](#documentation)\r\n    - [How to document](#how-to-document)\r\n    - [Sphinx themes:](#sphinx-themes)\r\n  - [Tests](#tests)\r\n  - [Some usefull commands:](#some-usefull-commands)\r\n  - [Packages needed](#packages-needed)\r\n  - [See also](#see-also)\r\n\r\n---\r\n\r\n## How it works?\r\n\r\nThis code uses the following approach:\r\n\r\n\u003cp align=\"center\"\u003e\r\n  \u003cimg\r\n    src=\"assets/project_workflow.png\"\r\n    width=\"500px\"\r\n    alt=\"Image #1\"/\u003e\r\n\u003c/p\u003e\r\n\r\n\r\n\r\n## How to run\r\n\r\nThe project was created and designed under \"jupyter researchers\" approach. Doing so, you'll need an jupyter\r\ninterface to open it.\r\n\r\n1. [Install python](https://www.python.org/downloads/)\r\n\r\n2. Install pip (only if you're running on linux):\r\n\r\n  ```bash\r\n  sudo apt install python3-pip\r\n  ```\r\n\r\n3. Install all dependencies\r\n\r\n  ```bash\r\n  cd src\r\n  # if you have the python2 installed too\r\n  pip3 install -r requirements.txt\r\n  # or\r\n  pip install -r requirements.txt\r\n\r\n  # uninstall\r\n  pip3 uninstall -r requirements.txt\r\n  ```\r\n\r\n4. You can install the graphviz by the package [site](https://graphviz.org/download/) or typing:\r\n    - in debian-based linux:\r\n\r\n      ```bash\r\n      sudo apt install graphviz\r\n      ```\r\n\r\n    - or using a choco (windows)\r\n\r\n      ```bash\r\n      choco install graphviz\r\n      ```\r\n5. You also need to install the [plotly support for jupyter][7]:\r\n\r\n```bash\r\n# basic packages\r\npip install plotly==4.14.3\r\npip install \"notebook\u003e=5.3\" \"ipywidgets\u003e=7.5\"\r\npip install jupyterlab \"ipywidgets\u003e=7.5\"\r\n\r\n# The following commands needs node to be installed:\r\n# JupyterLab renderer support\r\njupyter labextension install jupyterlab-plotly@4.14.3\r\n\r\n# OPTIONAL: Jupyter widgets extension\r\njupyter labextension install @jupyter-widgets/jupyterlab-manager plotlywidget@4.14.3\r\n```\r\n\r\n\r\n* Optionally, you can create an venv (virtual env):\r\n\r\n```bash\r\n# install venv\r\nsudo apt install python3-venv\r\n# create a new venv\r\npython3 -m venv pyenv\r\n\r\n# Activate the virtual environment\r\nsource pyenv/bin/activate\r\n# To deactivate the venv\r\ndeactivate\r\n```\r\n\r\n* You can install jupyter using:\r\n```bash\r\npip3 install jupyterlab\r\n```\r\n\r\n\u003e After that, opens the `./src/main.ipynb` file and run it.\r\n\r\n\r\n## Documentation\r\n\r\nThe [documentation](src/doc/build/html/index.html) of `modules` is inside `src/doc/build/html` folder.\r\n\r\n### How to document\r\n\r\nThe project was developed using _Google Docstrings_ and _RST (Reestructured text)_ guides.\r\nYou can check [here](https://www.google.com/search?q=google+documentation+python\u0026oq=google+pydocumention+\u0026aqs=chrome.1.69i57j0i13l9.8494j0j4\u0026sourceid=chrome\u0026ie=UTF-8) and\r\n[here](https://www.sphinx-doc.org/en/master/usage/restructuredtext/index.html).\r\n\r\n### Sphinx themes:\r\n\r\n- furo (black)\r\n- read the docs (default)\r\n- asteroid\r\n\r\n## Tests\r\n\r\n- You can test the python codes using the script \"tests.py\":\r\n\r\n  ```bash\r\n  # run all the tests\r\n  python src/tests.py\r\n  ```\r\n\r\n- You also can use the \"vscode testing tool\":\r\n  ```raw\r\n  Press: CTRL+SHIFT+P or F1\r\n  and then, type: Tests:\r\n  ```\r\n\r\n## Some usefull commands:\r\n\r\n```bash\r\n# autodocumentation sphinx api (inside source folder)\r\nsphinx-apidoc -o . ../../modules/\r\n\r\n# generate html pages for sphinx\r\nmake html\r\n\r\n# create a new documentation\r\nsphinx-quickstart\r\n\r\n\r\n# dump mongo\r\n$ mongodump --port 27017 -u USER_NICK -p USER_PASSWORD -d DATABASE --authenticationDatabase admin -o OUTPUT_FOLDER\r\n\r\n# restore mongo\r\n$ mongorestore --port 27017 -u USER_NICK -p USER_PASSWORD OUTPUT_FOLDER/\r\n\r\n# online restore\r\nmongorestore --uri URI_CONNECTION_STRING OUTPUT_FOLDER/\r\n```\r\n\r\n\u003e [Why don't use markdown to documentation](https://www.ericholscher.com/blog/2016/mar/15/dont-use-markdown-for-technical-docs/)\r\n\r\n\r\n## See also\r\n\r\n\u003e To install other themes you must search for a python package equivalent for that theme.\r\n\r\n1. [Sphinx themes][1]\r\n2. [Sphinx automodule/autoclass errors][2]\r\n3. [Generating automodules][3]\r\n4. [Sphinx cross references][4]\r\n5. [Sphinx pygments confs][5]\r\n6. [Getting started with sphinx][6]\r\n7. [Getting started with plotly][7]\r\n8. [Mongo mocking (Testing for mongo)][8]\r\n\r\n\u003c!-- Links --\u003e\r\n\r\n[1]: https://sphinx-themes.org/\r\n[2]: https://stackoverflow.com/questions/13516404/sphinx-error-unknown-directive-type-automodule-or-autoclass\r\n[3]: https://www.youtube.com/watch?v=b4iFyrLQQh4\u0026ab_channel=avcourt\r\n[4]: https://stackoverflow.com/questions/34533346/incorrect-cross-reference-syntax-gives-confusing-undefined-label-warning\r\n[5]: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-pygments_style\r\n[6]: https://docs.readthedocs.io/en/stable/intro/getting-started-with-sphinx.html\r\n[7]: https://plotly.com/python/getting-started/\r\n[8]: https://github.com/mongomock/mongomock/blob/develop/tests/test__mongomock.py\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppcamp%2Fsigaa-tcc-scraping-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fppcamp%2Fsigaa-tcc-scraping-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppcamp%2Fsigaa-tcc-scraping-analysis/lists"}