{"id":23061804,"url":"https://github.com/rrwen/search_google","last_synced_at":"2025-08-15T08:32:59.575Z","repository":{"id":57465093,"uuid":"91630843","full_name":"rrwen/search_google","owner":"rrwen","description":"A command line tool and module for Google web and image search","archived":false,"fork":false,"pushed_at":"2017-11-08T00:09:56.000Z","size":372,"stargazers_count":12,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-02T12:58:15.806Z","etag":null,"topics":["api","cli","command","cse","custom","engine","google","image","interface","line","search","tool","web"],"latest_commit_sha":null,"homepage":"https://rrwen.github.io/search_google","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rrwen.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-18T00:09:57.000Z","updated_at":"2021-10-10T13:46:09.000Z","dependencies_parsed_at":"2022-09-17T17:52:12.206Z","dependency_job_id":null,"html_url":"https://github.com/rrwen/search_google","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrwen%2Fsearch_google","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrwen%2Fsearch_google/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrwen%2Fsearch_google/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rrwen%2Fsearch_google/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rrwen","download_url":"https://codeload.github.com/rrwen/search_google/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":229901936,"owners_count":18141799,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","cli","command","cse","custom","engine","google","image","interface","line","search","tool","web"],"created_at":"2024-12-16T03:18:32.488Z","updated_at":"2024-12-16T03:18:32.949Z","avatar_url":"https://github.com/rrwen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"search_google\n=============\n\n| Richard Wen\n| rrwen.dev@gmail.com\n\n* `Documentation \u003chttps://rrwen.github.io/search_google\u003e`_\n* `PyPi Package \u003chttps://pypi.python.org/pypi/search_google\u003e`_\n\nA command line tool and module for Google API web and image search.\n\n.. image:: https://badge.fury.io/py/search-google.svg\n    :target: https://badge.fury.io/py/search-google\n.. image:: https://travis-ci.org/rrwen/search_google.svg?branch=master\n    :target: https://travis-ci.org/rrwen/search_google\n.. image:: https://coveralls.io/repos/github/rrwen/search_google/badge.svg?branch=master\n    :target: https://coveralls.io/github/rrwen/search_google?branch=master\n.. image:: https://img.shields.io/github/issues/rrwen/search_google.svg\n    :target: https://github.com/rrwen/search_google/issues\n.. image:: https://img.shields.io/badge/license-MIT-blue.svg\n    :target: https://raw.githubusercontent.com/rrwen/search_google/master/LICENSE\n.. image:: https://img.shields.io/github/stars/rrwen/search_google.svg\n    :target: https://github.com/rrwen/search_google/stargazers\n.. image:: https://img.shields.io/twitter/url/https/github.com/rrwen/search_google.svg?style=social\n    :target: https://twitter.com/intent/tweet?text=%23python%20%23dataextraction%20tool%20for%20%23googlesearch%20results%20and%20%23googleimages:%20https://github.com/rrwen/search_google\n\nInstall\n-------\n\n1. Install `Python \u003chttps://www.python.org/downloads/\u003e`_\n2. Install `search_google \u003chttps://pypi.python.org/pypi/search-google\u003e`_ via ``pip``\n\n::\n  \n  pip install search_google\n  \nFor the latest developer version, see `Developer Install`_.\n  \nUsage\n-----\n\nFor help in the console::\n  \n  search_google -h\n  \nEnsure that a `CSE ID \u003chttps://support.google.com/customsearch/answer/2649143?hl=en\u003e`_ and a `Google API developer key \u003chttps://developers.google.com/api-client-library/python/auth/api-keys\u003e`_ are set::\n\n  search_google -s cx=\"your_cse_id\"\n  search_google -s build_developerKey=\"your_dev_key\"\n\nSearch the web for keyword \"cat\"::\n  \n  search_google \"cat\"\n  search_google \"cat\" --save_links=cat.txt\n  search_google \"cat\" --save_downloads=downloads\n\nSearch for \"cat\" images::\n  \n  search_google cat --searchType=image\n  search_google \"cat\" --searchType=image --save_links=cat_images.txt\n  search_google \"cat\" --searchType=image --save_downloads=downloads\n  \nUse as a Python module:\n\n.. code-block:: python\n\n  # Import the api module for the results class\n  import search_google.api\n  \n  # Define buildargs for cse api\n  buildargs = {\n    'serviceName': 'customsearch',\n    'version': 'v1',\n    'developerKey': 'your_api_key'\n  }\n  \n  # Define cseargs for search\n  cseargs = {\n    'q': 'keyword query',\n    'cx': 'your_cse_id',\n    'num': 3\n  }\n  \n  # Create a results object\n  results = search_google.api.results(buildargs, cseargs)\n  \n  # Download the search results to a directory\n  results.download_links('downloads')\n  \nFor more usage details, see the `Documentation \u003chttps://rrwen.github.io/search_google\u003e`_.\n\nContributions\n-------------\n\nReport Contributions\n********************\n\nReports for issues and suggestions can be made using the `issue submission \u003chttps://github.com/rrwen/search_google/issues\u003e`_ interface.  \n  \nWhen possible, ensure that your submission is:\n\n* **Descriptive**: has informative title, explanations, and screenshots\n* **Specific**: has details of environment (such as operating system and hardware) and software used\n* **Reproducible**: has steps, code, and examples to reproduce the issue\n\nCode Contributions\n******************\n\nCode contributions are submitted via `pull requests \u003chttps://help.github.com/articles/about-pull-requests\u003e`_:\n\n1. Ensure that you pass the `Tests`_\n2. Create a new `pull request \u003chttps://github.com/rrwen/search_google/pulls\u003e`_\n3. Provide an explanation of the changes\n\nA template of the code contribution explanation is provided below:\n\n::\n\n    ## Purpose\n    \n    The purpose can mention goals that include fixes to bugs, addition of features, and other improvements, etc.\n    \n    ## Description\n    \n    The description is a short summary of the changes made such as improved speeds, implementation\n    \n    ## Changes\n    \n    The changes are a list of general edits made to the files and their respective components.\n    * `file_path1`:\n        * `function_module_etc`: changed loop to map\n        * `function_module_etc`: changed variable value\n    * `file_path2`:\n        * `function_module_etc`: changed loop to map\n        * `function_module_etc`: changed variable value\n    \n    ## Notes\n    \n    The notes provide any additional text that do not fit into the above sections.\n\nFor more information, see `Developer Install`_ and `Implementation`_.\n\nDeveloper Notes\n---------------\n\nDeveloper Install\n*****************\n\nInstall the latest developer version with ``pip`` from github::\n  \n  pip install git+https://github.com/rrwen/search_google\n  \nInstall from ``git`` cloned source:\n\n1. Ensure `git \u003chttps://git-scm.com/\u003e`_ is installed\n2. Clone into current path\n3. Install via ``pip``\n\n::\n\n  git clone https://github.com/rrwen/search_google\n  cd search_google\n  pip install . -I\n  \nTests\n*****\n\n1. Clone into current path ``git clone https://github.com/rrwen/search_google``\n2. Enter into folder ``cd search_google``\n3. Ensure `unittest \u003chttps://docs.python.org/2.7/library/unittest.html\u003e`_ is available\n4. Set your `CSE ID \u003chttps://support.google.com/customsearch/answer/2649143?hl=en\u003e`_ and `Google API developer key \u003chttps://developers.google.com/api-client-library/python/auth/api-keys\u003e`_\n5. Run tests\n6. Reset config file to defaults\n7. Please note that this will use up 7 requests from your quota\n\n::\n  \n  pip install . -I\n  python -m search_google -s cx=\"your_cse_id\"\n  python -m search_google -s build_developerKey=\"your_dev_key\"\n  python -m unittest\n  python -m search_google -d\n\nDocumentation Maintenance\n*************************\n\n1. Ensure `sphinx \u003chttps://github.com/sphinx-doc/sphinx/\u003e`_ is installed ``pip install -U sphinx``\n2. Update the documentation in ``docs/``\n\n::\n  \n  pip install . -I\n  sphinx-build -b html docs/source docs\n\nUpload to github\n****************\n\n1. Ensure `git \u003chttps://git-scm.com/\u003e`_ is installed\n2. Add all files and commit changes\n3. Push to github\n\n::\n  \n  git add .\n  git commit -a -m \"Generic update\"\n  git push\n  \nUpload to PyPi\n**************\n\n1. Ensure `twine \u003chttps://pypi.python.org/pypi/twine\u003e`_ is installed ``pip install twine``\n2. Ensure `sphinx \u003chttps://github.com/sphinx-doc/sphinx/\u003e`_ is installed ``pip install -U sphinx``\n3. Run tests and check for OK status\n4. Delete ``dist`` directory\n5. Update the version ``search_google/__init__.py``\n6. Update the documentation in ``docs/``\n7. Create source distribution\n8. Upload to `PyPi \u003chttps://pypi.python.org/pypi\u003e`_\n\n::\n  \n  pip install . -I\n  python -m search_google -s cx=\"your_cse_id\"\n  python -m search_google -s build_developerKey=\"your_dev_key\"\n  python -m unittest\n  python -m search_google -d\n  sphinx-build -b html docs/source docs\n  python setup.py sdist\n  twine upload dist/*\n  \nImplementation\n**************\n\nThis command line tool uses the `Google Custom Search Engine (CSE) \u003chttps://developers.google.com/api-client-library/python/apis/customsearch/v1\u003e`_ to perform web and image searches. It relies on `googleapiclient.build \u003chttps://google.github.io/google-api-python-client/docs/epy/googleapiclient.discovery-module.html#build\u003e`_ and `cse.list \u003chttps://developers.google.com/resources/api-libraries/documentation/customsearch/v1/python/latest/customsearch_v1.cse.html\u003e`_, where ``build`` was used to create a Google API object and ``cse`` was used to perform the searches.\n\nThe class `search_google.api \u003chttps://rrwen.github.io/search_google/#module-api\u003e`_ simply passed a dictionary of arguments into ``build`` and ``cse`` to process the returned results with properties and methods. `search_google.cli \u003chttps://rrwen.github.io/search_google/#module-cli\u003e`_ was then used to create a command line interface for `search_google.api \u003chttps://rrwen.github.io/search_google/#module-api\u003e`_.\n\nIn order to use ``build`` and ``cse``, a `Google Developer API Key \u003chttps://developers.google.com/api-client-library/python/auth/api-keys\u003e`_ and a `Google CSE ID \u003chttps://cse.google.com/all\u003e`_ needs to be created for API access (see `search_google Setup \u003chttps://rrwen.github.io/search_google/#setup\u003e`_). Creating these keys also required a `Gmail \u003chttps://www.google.com/gmail\u003e`_ account for login access.\n\n::\n  \n          googleapiclient.build  \u003c-- Google API\n                    |                    \n                 cse.list        \u003c-- Google CSE\n                    |\n             search_google.api   \u003c-- search results\n                    |\n             search_google.cli   \u003c-- command line\n\nA rough example is provided below thanks to the `customsearch example \u003chttps://github.com/google/google-api-python-client/blob/master/samples/customsearch/main.py\u003e`_ from Google:\n\n.. code-block:: python\n  \n  from apiclient.discovery import build\n  \n  # Set developer key and CSE ID\n  dev_key = 'a_developer_key'\n  cse_id = 'a_cse_id'\n  \n  # Obtain search results from Google CSE\n  service = build(\"customsearch\", \"v1\", developerKey=dev_key)\n  results = service.cse().list(q='cat', cx=cse_id).execute()\n  \n  # Manipulate search results after ...\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frrwen%2Fsearch_google","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frrwen%2Fsearch_google","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frrwen%2Fsearch_google/lists"}