https://github.com/theprojectsx/articleparser
Parse Articles from WEB via Search Query
https://github.com/theprojectsx/articleparser
Last synced: 6 months ago
JSON representation
Parse Articles from WEB via Search Query
- Host: GitHub
- URL: https://github.com/theprojectsx/articleparser
- Owner: TheProjectsX
- Created: 2024-03-30T18:34:59.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-30T20:21:07.000Z (about 2 years ago)
- Last Synced: 2025-06-03T14:26:56.017Z (about 1 year ago)
- Language: Python
- Size: 5.86 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Article Parser
Get Articles Data just by Search Query
### Workflow:
- Uses googlesearch_python package to search for Google Search URLs
- Uses requests to get the URL's Webpage data
- Uses bs4, wikipedia and newspaper Libraries to Parse the actual Contents of the URL's
- Uses html2text to convert the HTML content to Markup. Which is decided by user
- Returns a Generator Object containing Search Results
### Installations:
Install using pip
```bash script
pip install git+https://github.com/TheProjectsX/ArticleParser.git
```
## Usages
### Get Articles via Search Query
```python
import articleparser
articlesData = articleparser.getArticles(query="What is Node JS?")
for article in articlesData:
print("URL:", article["url"])
print("Title:", article["title"])
print("Body:", article["body"][:400])
```
### Get Google Search Results
```python
import articleparser
searchResults = articleparser.getGoogleSearchResults(query="What is Node JS?")
for article in articlesData:
print("URL:", article["url"])
print("Title:", article["title"])
print("Description:", article["description"])
```
### Parse Article from a Certain URL
User can pass a certain Webpage URL to parse it's content
```python
import articleparser
article = articleparser.parseArticle(url="")
print("Title:", article["title"])
print("Content:", article["content"][:400])
```
## NOTE:
There are many useful Parameters in each Function.
You can get it's description Just by hovering in them or opening the file!