{"id":24360407,"url":"https://github.com/dms-codes/scrape_dosen_fh_unibraw","last_synced_at":"2025-06-10T16:10:27.972Z","repository":{"id":199187209,"uuid":"702313712","full_name":"dms-codes/scrape_dosen_fh_unibraw","owner":"dms-codes","description":"Web Scraping for Faculty of Law Professors' Information This Python script is designed for web scraping the profile information of professors from the Faculty of Law at the University of Brawijaya. It collects data such as names, titles, sub-titles, profile URLs, image URLs, NIP","archived":false,"fork":false,"pushed_at":"2023-10-09T04:39:33.000Z","size":17,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-12T08:34:49.594Z","etag":null,"topics":["python","scraper","scraping-websites"],"latest_commit_sha":null,"homepage":"https://github.com/dms-codes/scrape_dosen_fh_unibraw","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dms-codes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-09T04:35:56.000Z","updated_at":"2023-10-09T04:40:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"52813c93-8528-4425-9fc0-0d52f59debc9","html_url":"https://github.com/dms-codes/scrape_dosen_fh_unibraw","commit_stats":null,"previous_names":["dms-codes/scrape_dosen_fh_unibraw"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dms-codes%2Fscrape_dosen_fh_unibraw","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dms-codes%2Fscrape_dosen_fh_unibraw/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dms-codes%2Fscrape_dosen_fh_unibraw/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dms-codes%2Fscrape_dosen_fh_unibraw/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dms-codes","download_url":"https://codeload.github.com/dms-codes/scrape_dosen_fh_unibraw/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dms-codes%2Fscrape_dosen_fh_unibraw/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259106678,"owners_count":22805941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","scraper","scraping-websites"],"created_at":"2025-01-18T21:19:40.494Z","updated_at":"2025-06-10T16:10:27.942Z","avatar_url":"https://github.com/dms-codes.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web Scraping for Faculty of Law Professors' Information\n\nThis Python script is designed for web scraping the profile information of professors from the Faculty of Law at the University of Brawijaya. It collects data such as names, titles, sub-titles, profile URLs, image URLs, NIP (Nomor Induk Pegawai) numbers, email addresses, education details, research information, publications, and books authored. The scraped data is saved in a CSV file for further analysis.\n\n## Prerequisites\n\nBefore running the script, make sure you have the following Python libraries installed:\n\n- `requests`: Used for making HTTP requests to web pages.\n- `BeautifulSoup` (imported as `bs`): A library for parsing HTML content.\n- `csv`: Used for writing data to a CSV file.\n\nYou can install these libraries using `pip`:\n\n```bash\npip install requests beautifulsoup4\n```\n\n## Usage\n\n1. Clone this repository or download the Python script to your local machine.\n\n2. Open the script in your favorite text editor or integrated development environment (IDE).\n\n3. Customize the script if needed:\n\n   - `BASE_URL`: The URL of the Faculty of Law professors' profiles page you want to scrape.\n   - `TIMEOUT`: The timeout for HTTP requests (in seconds).\n   - `HEADERS`: HTTP headers for requests.\n\n4. Run the script:\n\n   ```bash\n   python your_script_name.py\n   ```\n\n   Replace `your_script_name.py` with the actual name of the script.\n\n5. The script will start scraping professor information and print the names, titles, and sub-titles of each professor as it progresses. Once completed, the data will be saved to a CSV file named `data_dosen_fh_unibraw.csv` in the same directory as the script.\n\n## Output\n\nThe CSV file `data_dosen_fh_unibraw.csv` will contain the following columns:\n\n- `Name`: Professor's name.\n- `Title`: Professor's title.\n- `Sub`: Sub-title (if available).\n- `Profile URL`: URL to the professor's profile.\n- `Img URL`: URL to the professor's profile image.\n- `NIP`: Nomor Induk Pegawai (Employee Identification Number).\n- `Email`: Professor's email address.\n- `Education`: Education background.\n- `Research`: Research information.\n- `Publication`: Publication details with links (if available).\n- `Books`: Books authored by the professor.\n\n## Note\n\n- Make sure to respect the website's terms of use and scraping policies.\n- This script is provided as-is and may require adjustments to work with different websites or changes to the target website's structure.\n- Be aware of ethical and legal considerations when scraping websites for data. Always ensure that you have the necessary permissions and comply with applicable laws and terms of service.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdms-codes%2Fscrape_dosen_fh_unibraw","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdms-codes%2Fscrape_dosen_fh_unibraw","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdms-codes%2Fscrape_dosen_fh_unibraw/lists"}