{"id":20335072,"url":"https://github.com/dimaj/moocs","last_synced_at":"2026-05-07T20:42:30.220Z","repository":{"id":7973944,"uuid":"9375341","full_name":"dimaj/moocs","owner":"dimaj","description":"Moocs project for CS160","archived":false,"fork":false,"pushed_at":"2013-05-20T22:51:53.000Z","size":4163,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-01-14T16:38:26.478Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dimaj.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-04-11T16:48:31.000Z","updated_at":"2015-04-04T16:25:35.000Z","dependencies_parsed_at":"2022-07-19T14:39:53.105Z","dependency_job_id":null,"html_url":"https://github.com/dimaj/moocs","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimaj%2Fmoocs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimaj%2Fmoocs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimaj%2Fmoocs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimaj%2Fmoocs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dimaj","download_url":"https://codeload.github.com/dimaj/moocs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241854666,"owners_count":20031478,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-14T20:39:07.672Z","updated_at":"2026-05-07T20:42:25.199Z","avatar_url":"https://github.com/dimaj.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"moocs\n=====\n\nMoocs project for CS160\n-----------------------\n\nThis project is a backend system that scrapes MOOCs sites and updates database with scraped information.\n\n## Adding new scrapers \n\nTo add a new scraper, you will need to modify ```config.php``` file, which is located in ```config``` folder.\n\nHere is the information that you will need to add:\n\n```\n$GLOBALS['scrapers']['name'] = array(\n\t\"className\" =\u003e \"class name that contains the scraper\",\n\t\"mainURL\" =\u003e \"Site's base url, where all classes are located\",\n\t\"mainSearchString\" =\u003e \"search string that will identify all the classes on the main page\",\n\t\"name\" =\u003e \"name of the site. Should be same as 'name' above\"\n);\n```\n\nSo, a sample entry for Canvase.net would look like this:\n\n```\n$GLOBALS['scrapers']['canvas'] = array(\n\t\"className\" =\u003e \"Canvas\",\n\t\"mainURL\" =\u003e \"https://www.canvas.net\",\n\t\"mainSearchString\" =\u003e \"div[class=block-box no-pad featured-course-outer-container]\",\n\t\"name\" =\u003e \"Canvas\"\n);\n```\n\n## Configuring Database connection\n\nTo configure a database connection, the following block has to be added to the ```config.php``` file:\n\n```\n$GLOBALS['db'] = array(\n\t\"host\" =\u003e \"ip of the server, where mysql is installed\",\n\t\"user\" =\u003e \"username that has access to the database\",\n\t\"pass\" =\u003e \"password for that user\",\n\t\"db\" =\u003e \"name of the database, where data is stored\"\n);\n```\n\nA sample database block will look something like this:\n\n```\n$GLOBALS['db'] = array(\n\t\"host\" =\u003e \"localhost\",\n\t\"user\" =\u003e \"root\",\n\t\"pass\" =\u003e \"password\",\n\t\"db\" =\u003e \"moocs\"\n);\n```\n\n## Running this project\n\n1) Open command prompt or terminal window\n\n2) Download this project to your computer\n```\ncd /home/user/\ngit clone https://github.com/dimaj/moocs.git\n```\n\n3) Navigate to the moocs folder\n```\ncd /home/user/moocs\n```\n\n4) Modify config.php to match your configuration\n\n5) Execute this command\n```\nphp ClassScraper.php\n```\n\nThe app will do the rest!\n\nNavigate to the phpMyAdmin to see the results.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimaj%2Fmoocs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdimaj%2Fmoocs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimaj%2Fmoocs/lists"}