{"id":20827779,"url":"https://github.com/kozhaakhmet/web-scrapping","last_synced_at":"2026-04-22T11:05:14.469Z","repository":{"id":176105048,"uuid":"493366571","full_name":"KozhaAkhmet/Web-Scrapping","owner":"KozhaAkhmet","description":"Simple web scrapping example on python.","archived":false,"fork":false,"pushed_at":"2022-05-17T21:14:36.000Z","size":603,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-18T17:56:59.221Z","etag":null,"topics":["python","python3","selenium","web","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KozhaAkhmet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-17T18:20:43.000Z","updated_at":"2022-10-16T17:58:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"3127a0c7-c7a6-43d9-9de9-6ecd5f126304","html_url":"https://github.com/KozhaAkhmet/Web-Scrapping","commit_stats":null,"previous_names":["kozhaakhmet/web-scrapping"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KozhaAkhmet%2FWeb-Scrapping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KozhaAkhmet%2FWeb-Scrapping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KozhaAkhmet%2FWeb-Scrapping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KozhaAkhmet%2FWeb-Scrapping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KozhaAkhmet","download_url":"https://codeload.github.com/KozhaAkhmet/Web-Scrapping/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243175369,"owners_count":20248465,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","python3","selenium","web","webscraping"],"created_at":"2024-11-17T23:12:59.475Z","updated_at":"2025-12-25T11:56:23.377Z","avatar_url":"https://github.com/KozhaAkhmet.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web_Scrapping\nThis is a little program that introduce web scrapping.\n## Goal\nMain purpose is to scrap names of products from internet market.\n## Pre requests\nyou need to install libraries:\n- BeautifulSoup     ``sudo pip install bs4``\n- Selenium          ``sudo pip install selenium``\n- Firefox Webdriver ``sudo apt-get install firefox-geckodriver `` \n\n## At First \nwe need to choose internet market. I prefer Trendyol.com. Then we need to copy url of page which interests us.\nIn this code I use this:\n\n``https://www.trendyol.com/sr?q=paten\u0026qt=paten\u0026st=paten\u0026os=1\u0026pi=1``\n\nBy right clicking a page, then clicking *inspect* you will see a new tab like this:\n![](images/Screenshot from 2022-05-17 23-26-56.png)\n\nThen by navigating a cursor you will see that every row is related to image boxes and also highlightes them. By expanding prefered boxes you can  reach to interested data. \n![](images/Screenshot%20from%202022-05-17%2023-38-22.png)\nIn this image you can see that we have rows ( classes ) of each product. By expending them, you can reach their detailes like name, price, etc. I remind that our goal is to find product names. It looks like this:\n```html\n\u003cspan class=\"prdct-desc-cntnr-name hasRatings\" title=\"Işıklı Ayarlanabilir Slikon Teker Metal Gövde Paten\"\u003eIşıklı Ayarlanabilir Slikon Teker Metal Gövde Paten\u003c/span\u003e\n```\nwe found every data we need.\n\n## Code\n```python\nfrom selenium import webdriver      #Importing Libraries\nfrom bs4 import BeautifulSoup       \n\ndriver = webdriver.Firefox()        #Defining Webdriver\n\nfor i in range(3):                  #Page scrolling loop\n    link=(\"https://www.trendyol.com/sr?q=paten\u0026qt=paten\u0026st=paten\u0026os=1\u0026pi=\" + str(i+1))      #Doing string addition to turn pages\n    driver.get(link)                          #Definding URL\n\n    content = driver.page_source\n    soup = BeautifulSoup(content, \"html.parser\")                      #Definfing site parser\n    products = soup.find_all('div', class_=\"p-card-wrppr add-to-bs-card\")             #In this row we search a class which include all products classes \n    for products in products:                                                #Then doing loop for every product class\n        name = products.find(\"span\", class_=\"prdct-desc-cntnr-name hasRatings\" )  #Searching for name string in product class\n        list = str(name).split('\"')                                              #Then by converting name to string, we split it and append to list\n        if \"None\" not in list :                                                  #We do not print,if list include \"None\"\n            print(list[3])\n\n```\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkozhaakhmet%2Fweb-scrapping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkozhaakhmet%2Fweb-scrapping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkozhaakhmet%2Fweb-scrapping/lists"}