https://github.com/92hackers/big-data-practise

A Node.js app, which mock millions of data and save into Mysql database, optimize the app code and database to get a better performance
https://github.com/92hackers/big-data-practise

Last synced: about 1 month ago
JSON representation

A Node.js app, which mock millions of data and save into Mysql database, optimize the app code and database to get a better performance

• Host: GitHub
• URL: https://github.com/92hackers/big-data-practise
• Owner: 92hackers
• Created: 2019-09-06T11:10:23.000Z (about 5 years ago)
• Default Branch: master
• Last Pushed: 2023-02-24T22:08:28.000Z (over 1 year ago)
• Last Synced: 2023-03-01T03:11:51.983Z (over 1 year ago)
• Language: JavaScript
• Size: 103 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 5
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# A big data scenario to practise how to manage, optimize millions of data

Big data scenario is not patent of big company, we could design a scenario,

then mock millions, billions of data, store into database.

## Features

1. Super fast data generation, concurrently generate billions data in Node.js cluster mode.

2. Full problem list, to track detail problems we will facing under big data scenario.

3. Data models mocked from an living website: https://zhihu.com, a Quora-like Ask && Answer product.

4. Friendly to all levels developers, easy to set up, and full tutorials to help.

## Hardware requirements

You should owns a little bit high performance computer, which will speed up your

practise, give you my PC as an example:

* CPU: 4 x Core i5-7500 @ 3.4GHz

* Memory: 16G

* Disk: 256G SSD

Bigger data set requires more resource, especially large amount of disk,

when your practise involved backup, partion, replica and sharding

## Tutorial

Full tutorial to begin your practise

### Requirements