https://github.com/garybake/siteanalyzer
Determine topic of a website from classifying the images.
https://github.com/garybake/siteanalyzer
Last synced: 3 months ago
JSON representation
Determine topic of a website from classifying the images.
- Host: GitHub
- URL: https://github.com/garybake/siteanalyzer
- Owner: garybake
- License: mit
- Created: 2020-05-14T13:40:25.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-07-06T20:58:34.000Z (about 2 years ago)
- Last Synced: 2024-12-06T20:38:26.190Z (over 1 year ago)
- Language: Python
- Size: 160 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
GB WebPage Classifier
=====================
App to determine what a web page is about by classifying the image
Uses serverless aws architecture
Architecture
============
Endpoints
=========
## Get list of all registered sites
curl https://gbimmageclassifier./url/list
## Get list of images for specific site
curl https://gbimmageclassifier./image/list?pageid=7213c7e3.www.garybake.com
## Request analysis of web page
curl --header "Content-Type: application/json" \
--request POST \
--data '{"url":"http://garybake.com"}' \
https://gbimmageclassifier./url/analyze
AWS
===
Cloudformation
Route53
API gateway
Lambda
S3
DynamoDB
SQS
Cloudwatch
Python
======
Boto3
BeautifulSoup
Requests
ENV
===
Needs the following environment variables set
export AWS_ACCOUNT_ID=***
export AWS_ACCESS_KEY_ID=***
export AWS_SECRET_ACCESS_KEY=***
export AWS_DEFAULT_REGION=***
export GBIMAGECLASSIFIER_BUCKET=***
export GBIMAGECLASSIFIER_TABLE=WebPage
export GBIMAGECLASSIFIER_CRAWLERQUEUE=CrawlerQueue
export GBIMAGECLASSIFIER_ANALYSISQUEUE=AnalysisQueue
export GBIMAGECLASSIFIER_DOMAIN=***
export GBIMAGECLASSIFIER_REGION=$AWS_DEFAULT_REGION
TODO
====
Front end UI
Add ui-service to deploy.sh
Refactor project layout
Move boto objects out of functions
AWS_DEFAULT_REGION is a reserved parameter (ui-service)