Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/gdsoumya/content_master
Content Master is a content aggregator that collects content from different sources, organizes them and puts them in one place for consumption.
https://github.com/gdsoumya/content_master
content-aggregation content-aggregator flask python3 web-scraping
Last synced: 23 days ago
JSON representation
Content Master is a content aggregator that collects content from different sources, organizes them and puts them in one place for consumption.
- Host: GitHub
- URL: https://github.com/gdsoumya/content_master
- Owner: gdsoumya
- License: mit
- Created: 2019-06-23T10:31:19.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2022-07-23T18:33:47.000Z (over 2 years ago)
- Last Synced: 2024-05-02T03:10:46.604Z (6 months ago)
- Topics: content-aggregation, content-aggregator, flask, python3, web-scraping
- Language: Python
- Size: 30.3 KB
- Stars: 7
- Watchers: 2
- Forks: 7
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Content Master
[](https://opensource.org/licenses/MIT) 
Content Master is a content aggregator that collects content from different sources, organizes them and puts them in one place for consumption.
## Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
### Prerequisites
**Content Master** requires [ **Python (> Python 3.6)**](https://www.python.org/) .
### Getting the project.
```sh
$ git clone https://github.com/gdsoumya/content_master.git
or
Download and extract the Zip-File
```
### Setting up Virtual Environemt
Setting up a virtual environment would be better for both development and normal execution purposes.
```sh
$ cd content_master
$ python -m virtualenv env
$ source env/bin/activate
or (Windows machine)
$ .\env\Scripts\activate
```
### Installing Dependencies
The Project has a few dependencies which can be installed by running.
```sh
$ pip install -r dependencies.txt
```
## Starting the Server
To start the Flask server run
```sh
$ python server.py
```
A Flask development server will be initialized at http://127.0.0.1:5000/
### Warnings
A possible warning that one might get is :
**WARNING: Do not use the development server in a production environment.**
This warning is displayed because currently a Flask Development Server is running but the default environment of Flask is set to Production, to remove this warning change the FLASK_ENV environment variable.
***Setting environment to development automatically sets the debugger on.**
```sh
$ export FLASK_ENV=development
or (Windows machine)
$ set FLASK_ENV=development
```
## Errors and Debugging options
The server by default does not start in debugger mode but to initialize debugger mode change the last line of the '[server.py](https://github.com/gdsoumya/content_master/blob/master/server.py)' file to :
```python
app.run() -> app.run(debug=True)
```
Most errors will be logged to the console and can be referenced later for debugging.
## Packages Used
- **[Requests](https://2.python-requests.org/en/master/)** : For fetching the source websites.
- **[Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)** : For scraping the source websites.
- **[Flask](http://flask.pocoo.org/)** : For hosting the web interface.
## Author
- **Soumya Ghosh Dastidar**
## Contributting
Any contribution/suggestions are welcomed.