https://github.com/marekyggdrasil/5e3602d8-2d6e-4424-89eb-88a98a215074

Challenge project for Aftership
https://github.com/marekyggdrasil/5e3602d8-2d6e-4424-89eb-88a98a215074

Last synced: about 1 month ago
JSON representation

Challenge project for Aftership

• Host: GitHub
• URL: https://github.com/marekyggdrasil/5e3602d8-2d6e-4424-89eb-88a98a215074
• Owner: marekyggdrasil
• Created: 2015-08-14T08:53:05.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2015-08-26T09:03:55.000Z (about 9 years ago)
• Last Synced: 2023-03-03T12:23:04.251Z (over 1 year ago)
• Language: JavaScript
• Size: 137 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Introduction #

- Gets USD to HKD exchange rate once every minute, stores 10 succesful results in MongoDB.

- After 3 failed attempts, aborts.

- System is scalable horizontally as long as every worker gets different id and has connection to same Beanstalkd tube and same MongoDB.

- May require updating xe.com custom API in case if xe.com decides to change their HTML structure.

# Files #

- ./server.js

Initiates worker node.

- ./producer.js

Used only to seed the initial task.

- ./peak.js

Simple admin tools, let's see if there is a job delayed in the queue.

- ./remove.js

Simple admin tools, let's see if there is a job delayed in the queue and removes it.

- ./database.js

Contains all access functions to MongoDB, it is done the way allowing not reconnecting at each request (connection pool)

- ./apixe.js

Custom API to access xe.com service.

- ./handlers/getrates.js

Handler that performs processing of task obtained from Beanstalkd tube.

# Dependencies #

- `soupselect`, `htmlparser` and `http` for custom xe.com API.

- `fivebeans` for worker, peak and producer, to access Beanstalkd queue.

- `mongodb` for accessing MongoDB database.

# Remarks #

- System is designed to store 10 succesful readings, restarting without clearing record collection will not allow storage 10 new readings.

- Processing is stopped after 3 failures, not necessarily 3 consecutive failures.

- Database is called `test`, records are stored in `aftership` collection and failed attempts in `failures` collection, failures are purged once processing ends.