{"id":19340300,"url":"https://github.com/edumats/flexible-price-tracker","last_synced_at":"2026-05-14T08:32:13.534Z","repository":{"id":47352109,"uuid":"505644072","full_name":"edumats/flexible-price-tracker","owner":"edumats","description":"Scrapper for tracking e-commerce prices. Uses XPath to keep track of the price element in a product's page. Uses Schedule to be able to schedule time intervals in which the scrapper will be activated.","archived":false,"fork":false,"pushed_at":"2022-09-29T20:35:41.000Z","size":45,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-01-06T11:36:47.859Z","etag":null,"topics":["python","scraping","selenium","selenium-python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/edumats.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-06-21T00:57:53.000Z","updated_at":"2022-09-15T19:59:29.000Z","dependencies_parsed_at":"2023-01-18T17:30:19.186Z","dependency_job_id":null,"html_url":"https://github.com/edumats/flexible-price-tracker","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edumats%2Fflexible-price-tracker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edumats%2Fflexible-price-tracker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edumats%2Fflexible-price-tracker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edumats%2Fflexible-price-tracker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/edumats","download_url":"https://codeload.github.com/edumats/flexible-price-tracker/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240443397,"owners_count":19802049,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","scraping","selenium","selenium-python"],"created_at":"2024-11-10T03:25:59.992Z","updated_at":"2026-05-14T08:32:08.512Z","avatar_url":"https://github.com/edumats.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Flexible Price Tracker\n\n\n### Video Demo: [https://youtu.be/ONO6ficUjug](https://youtu.be/ONO6ficUjug)\n\n\n### Description:\n\nE-commerce websites have different HTML structures that make scrapping prices a case by case task. Some websites might have a specific id attribute for the product's price, but others makes finding the price difficult by using identifiers that are not exclusive to the element that contains the price, like classes, or no usable identifiers at all.\n\n This price scrapper can be used with most of e-commerce websites by requiring the user to provide a [X-Path](https://www.w3schools.com/xml/xpath_intro.asp) of the element that contains the price. X-Path is similar to a path to a folder or file inside a file directory. As long as the path to a element does not change, the element containing the price can be retrieved.\n\n This scrapper can be run only once or run every X hours, minutes or days, using the --time argument. As an alternative, it is also possible to use OS specific schedulers like [Cron](https://en.wikipedia.org/wiki/Cron) if using Linux or MacOS, [Task Scheduler](https://en.wikipedia.org/wiki/Windows_Task_Scheduler) if using Windows.\n\n\n### Notifications\n\nThe scrapper will send a notification to the user's desktop if the price of the product is lower than the target price set by the user.\n\n\n### Price formats in various countries\n\nDifferent countries uses different currency formats, commas and dots will have different roles. For example:\n\n| Currency Name        | Currency Code | Format(s)        | Output       |\n|----------------------|---------------|------------------|--------------|\n| United States Dollar | USD           | $ ###,###,###.## | $123,000.50  |\n| Brazilian Real       | BRL           | R$ ###.###.###,## | R$123.000,50 |\n| Japanese Yen         | JPY           | ¥ ###,###,###    | ¥ 123,000,050    |\n\nIn order to correctly format the price used in different countries, the scraper needs a location setting. It is required to provide the locale in the arguments in order to run the scrapper. See [Usage](#usage) and [list of available locales](#list-of-locales) for more details.\n\n\n### Installation\n\nInstall all required modules by:\n\n```\npip install -r requirements.txt\n```\n\n#### Problems during installation\n\nIt is possible to encounter a problem related to pkg-config during installation of dbus-python on pip:\n\n````\nconfigure: error: The pkg-config script could not be found or is too old.  Make sure it\n      is in your PATH or set the PKG_CONFIG environment variable to the full\n      path to pkg-config.\n      \n      Alternatively, you may set the environment variables DBUS_CFLAGS\n      and DBUS_LIBS to avoid the need to call pkg-config.\n      See the pkg-config man page for more details.\n````\n\nTo solve this problem, it is necessary to install system packages using apt on Linux or brew on MacOS:\n\nOn Linux:\n\n```\napt install pkg-config build-essential libpython3-dev libdbus-1-dev\n```\n\nOn MacOS:\n\n```\nbrew install pkg-config dbus glib\n````\n\nthen install dbus-python again using pip to install it successfuly:\n\n```\npip install dbus-python\n```\n\n\n### Usage\n\n```\npython3 project.py \u003curl\u003e \u003cxpath\u003e \u003ctarget_price\u003e \u003clocale\u003e\n```\n\n| Required argument | Description                                             |\n|-------------------|---------------------------------------------------------|\n| url               | URL to be scrapped                                      |\n| xpath             | X-Path of the element that contains the product's price |\n| target_price      | Target price to activate a notification                 |\n| locale            | Locale setting in order to correctly convert price      |\n\n| Optional argument   | Description                                      |\n|---------------------|--------------------------------------------------|\n| -h, --help          | Show help message and exit                       |\n| -t, --time          | Run between X time itervals                      |\n| --all-notifications | Notify even if price is higher than target price |\n| --headless          | Will not open browser window when scrapping      |\n\n### **Important**\n\nIt is common for urls and XPaths to have characters such as / (forward slash) or \" (double quotes) that needs to be escaped to avoid unwanted behavior at the terminal shell. **Always use single quotes (') around the url and the XPath for the program to work correctly.**\n\n### Using scheduling\n\nUse the -t or --time argument to run between available intervals. \nAvailable intervals are:\n\n| Option | Description         |\n|--------|---------------------|\n| m      | Run every X minutes |\n| h      | Run every X hours   |\n| d      | Run every X days    |\n\nFor example:\n\n````\npython3 project.py 'www.shop.com' '//*[@id=\"price\"]' 49 en_US.UTF-8 --time m10\n````\nIt will run the scrapper once every 10 minutes\n\n````\npython3 project.py 'www.shop.com' '//*[@id=\"price\"]' 49 en_US.UTF-8 --time h2\n````\nIt will run the scrapper once every 2 hours\n\n````\npython3 project.py 'www.shop.com' '//*[@id=\"price\"]' 49 en_US.UTF-8 --time d1\n````\n\nIt will run the scrapper once every day\n\n**Important**: Notifications will appear only when the actual price of product is equal or below the targetprice. They also will not appear if user's computer is set on \"Do not disturb\" mode. Please deactivate this mode if desktop notifications are desired.\n\n\n### How to get the price element's X-Path\n\nThe X-Path can be obtained by using the browser's developer's tools. Specifics will vary form browser to browser, but the process is usually similar.\n\n#### In Chrome:\n\n1. Right click \"Inspect\" on the item you want to find the xpath\n1. Right click on the highlighted element in Devtools\n1. Select Copy \u003e Copy XPath\n\n#### In Firefox:\n\n1. Right click \"Inspect\" on the item you want to find the xpath\n1. Right click on the highlighted element in Devtools\n1. Select Copy \u003e XPath\n\n\n### File contents\n\n#### project.py\n\nContains the following functions:\n\n**main**: Calls the other functions for a single scrapping task\n\n**extract_time**: Used for the scheduling feature, validates schedule input\n\n**scrap_page**: Given an url, XPath and driver, returns the target element's contents\n\n**convert_string_to_float**: Given a price like string, considers locale for correctly converting to float\n\n**create_message**: Given a price and a target price, generates appropriate message for user\n\n#### test_project.py\n\nContains tests for each of the functions contained in project.py.\n\n#### requirements.txt\n\nContains all dependencies for the project.\n\n### List of locales\n\nUse the list below to get the appropriate locale for your currency format:\n\n| Locale             | Description                              |\n|--------------------|------------------------------------------|\n| af_ZA.UTF-8        | Afrikaans, South Africa                  |\n| ar_AE.UTF-8        | Arabic, United Arab Emirates             |\n| ar_BH.UTF-8        | Arabic, Bahrain                          |\n| ar_DZ.UTF-8        | Arabic, Algeria                          |\n| ar_EG.UTF-8        | Arabic, Egypt                            |\n| ar_IQ.UTF-8        | Arabic, Iraq                             |\n| ar_JO.UTF-8        | Arabic, Jordan                           |\n| ar_KW.UTF-8        | Arabic, Kuwait                           |\n| ar_LY.UTF-8        | Arabic, Libya                            |\n| ar_MA.UTF-8        | Arabic, Morocco                          |\n| ar_OM.UTF-8        | Arabic, Oman                             |\n| ar_QA.UTF-8        | Arabic, Qatar                            |\n| ar_SA.UTF-8        | Arabic, Saudi Arabia                     |\n| ar_TN.UTF-8        | Arabic, Tunisia                          |\n| ar_YE.UTF-8        | Arabic, Yemen                            |\n| as_IN.UTF-8        | Assamese, India                          |\n| az_AZ.UTF-8        | Azerbaijani, Azerbaijan                  |\n| be_BY.UTF-8        | Belarusian, Belarus                      |\n| bg_BG.UTF-8        | Bulgarian, Bulgaria                      |\n| bn_IN.UTF-8        | Bengali, India                           |\n| bs_BA.UTF-8        | Bosnian, Bosnia and Herzegovina          |\n| ca_ES.UTF-8        | Catalan, Spain                           |\n| cs_CZ.UTF-8        | Czech, Czech Republic                    |\n| da_DK.UTF-8        | Danish, Denmark                          |\n| de_AT.UTF-8        | German, Austria                          |\n| de_BE.UTF-8        | German, Belgium                          |\n| de_CH.UTF-8        | German, Switzerland                      |\n| de_DE.UTF-8        | German, Germany                          |\n| de_LI.UTF-8        | German, Liechtenstein                    |\n| de_LU.UTF-8        | German, Luxembourg                       |\n| el_CY.UTF-8        | Greek, Cyprus                            |\n| el_GR.UTF-8        | Greek, Greece                            |\n| en_AU.UTF-8        | English, Australia                       |\n| en_BW.UTF-8        | English, Botswana                        |\n| en_CA.UTF-8        | English, Canada                          |\n| en_GB.UTF-8        | English, United Kingdom                  |\n| en_HK.UTF-8        | English, Hong Kong SAR China             |\n| en_IE.UTF-8        | English, Ireland                         |\n| en_IN.UTF-8        | English, India                           |\n| en_MT.UTF-8        | English, Malta                           |\n| en_NZ.UTF-8        | English, New Zealand                     |\n| en_PH.UTF-8        | English, Philippines                     |\n| en_SG.UTF-8        | English, Singapore                       |\n| en_US.UTF-8        | English, U.S.A.                          |\n| en_ZW.UTF-8        | English, Zimbabwe                        |\n| es_AR.UTF-8        | Spanish, Argentina                       |\n| es_BO.UTF-8        | Spanish, Bolivia                         |\n| es_CL.UTF-8        | Spanish, Chile                           |\n| es_CO.UTF-8        | Spanish, Colombia                        |\n| es_CR.UTF-8        | Spanish, Costa Rica                      |\n| es_DO.UTF-8        | Spanish, Dominican Republic              |\n| es_EC.UTF-8        | Spanish, Ecuador                         |\n| es_ES.UTF-8        | Spanish, Spain                           |\n| es_GT.UTF-8        | Spanish, Guatemala                       |\n| es_HN.UTF-8        | Spanish, Honduras                        |\n| es_MX.UTF-8        | Spanish, Mexico                          |\n| es_NI.UTF-8        | Spanish, Nicaragua                       |\n| es_PA.UTF-8        | Spanish, Panama                          |\n| es_PE.UTF-8        | Spanish, Peru                            |\n| es_PR.UTF-8        | Spanish, Puerto Rico                     |\n| es_PY.UTF-8        | Spanish, Paraguay                        |\n| es_SV.UTF-8        | Spanish, El Salvador                     |\n| es_US.UTF-8        | Spanish, U.S.A.                          |\n| es_UY.UTF-8        | Spanish, Uruguay                         |\n| es_VE.UTF-8        | Spanish, Venezuela                       |\n| et_EE.UTF-8        | Estonian, Estonia                        |\n| fi_FI.UTF-8        | Finnish, Finland                         |\n| fr_BE.UTF-8        | French, Belgium                          |\n| fr_CA.UTF-8        | French, Canada                           |\n| fr_CH.UTF-8        | French, Switzerland                      |\n| fr_FR.UTF-8        | French, France                           |\n| fr_LU.UTF-8        | French, Luxembourg                       |\n| gu_IN.UTF-8        | Gujarati, India                          |\n| he_IL.UTF-8        | Hebrew, Israel                           |\n| hi_IN.UTF-8        | Hindi, India                             |\n| hr_HR.UTF-8        | Croatian, Croatia                        |\n| hu_HU.UTF-8        | Hungarian, Hungary                       |\n| hy_AM.UTF-8        | Armenian, Armenia                        |\n| id_ID.UTF-8        | Indonesian, Indonesia                    |\n| is_IS.UTF-8        | Icelandic, Iceland                       |\n| it_CH.UTF-8        | Italian, Switzerland                     |\n| it_IT.UTF-8        | Italian, Italy                           |\n| ja_JP.UTF-8        | Japanese, Japan                          |\n| ka_GE.UTF-8        | Georgian, Georgia                        |\n| kk_KZ.UTF-8        | Kazakh, Kazakhstan                       |\n| kn_IN.UTF-8        | Kannada, India                           |\n| ko_KR.UTF-8        | Korean, Korea                            |\n| ks_IN.UTF-8        | Kashmiri, India                          |\n| ku_TR.UTF-8        | Kurdish, Turkey                          |\n| ku_TR.UTF-8@sorani | Kurdish (Sorani), Turkey                 |\n| ky_KG.UTF-8        | Kirghiz, Kyrgyzstan                      |\n| lt_LT.UTF-8        | Lithuanian, Lithuania                    |\n| lv_LV.UTF-8        | Latvian, Latvia                          |\n| mk_MK.UTF-8        | Macedonian, Macedonia                    |\n| ml_IN.UTF-8        | Malayalam, India                         |\n| mr_IN.UTF-8        | Marathi, India                           |\n| ms_MY.UTF-8        | Malay, Malaysia                          |\n| mt_MT.UTF-8        | Maltese, Malta                           |\n| nb_NO.UTF-8        | Bokmal, Norway                           |\n| nl_BE.UTF-8        | Dutch, Belgium                           |\n| nl_NL.UTF-8        | Dutch, Netherlands                       |\n| nn_NO.UTF-8        | Nynorsk, Norway                          |\n| or_IN.UTF-8        | Oriya, India                             |\n| pa_IN.UTF-8        | Punjabi, India                           |\n| pl_PL.UTF-8        | Polish, Poland                           |\n| pt_BR.UTF-8        | Portuguese, Brazil                       |\n| pt_PT.UTF-8        | Portuguese, Portugal                     |\n| ro_RO.UTF-8        | Romanian, Romania                        |\n| ru_RU.UTF-8        | Russian, Russia                          |\n| ru_UA.UTF-8        | Russian, Ukraine                         |\n| sa_IN.UTF-8        | Sanskrit, India                          |\n| sk_SK.UTF-8        | Slovak, Slovakia                         |\n| sl_SI.UTF-8        | Slovenian, Slovenia                      |\n| sq_AL.UTF-8        | Albanian, Albania                        |\n| sr_ME.UTF-8        | Serbian, Montenegro                      |\n| sr_ME.UTF-8@latin  | Serbian, Montenegro (Latin)              |\n| sr_RS.UTF-8        | Serbian, Serbia                          |\n| sr_RS.UTF-8@latin  | Serbian, Serbia (Latin)                  |\n| sv_SE.UTF-8        | Swedish, Sweden                          |\n| ta_IN.UTF-8        | Tamil, India                             |\n| te_IN.UTF-8        | Telugu, India                            |\n| th_TH.UTF-8        | Thai, Thailand                           |\n| tr_TR.UTF-8        | Turkish, Turkey                          |\n| uk_UA.UTF-8        | Ukrainian, Ukraine                       |\n| vi_VN.UTF-8        | Vietnamese, Vietnam                      |\n| zh_CN.UTF-8        | Simplified Chinese, China                |\n| zh_HK.UTF-8        | Traditional Chinese, Hong Kong SAR China |\n| zh_SG.UTF-8        | Chinese, Singapore                       |\n| zh_TW.UTF-8        | Traditional Chinese, Taiwan              |\n\nTable from https://docs.oracle.com/cd/E23824_01/html/E26033/glset.html#glscx","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedumats%2Fflexible-price-tracker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fedumats%2Fflexible-price-tracker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedumats%2Fflexible-price-tracker/lists"}