Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/hiyali/node-mongodb-dockerize

🐳 Dockerized repo of a Nodejs + MongoDB crawler (https://github.com/hiyali/node-crawler-on-mongodb) for daily job.
https://github.com/hiyali/node-mongodb-dockerize

crontab demo docker dockerization mongodb nodejs

Last synced: about 1 month ago
JSON representation

🐳 Dockerized repo of a Nodejs + MongoDB crawler (https://github.com/hiyali/node-crawler-on-mongodb) for daily job.

Awesome Lists containing this project

README

        

# Nodejs + Phantomjs Crawler on Mongodb dockerization

## Download & Build
```shell
mkdir -p /hunt/db
git clone https://github.com/hiyali/node-mongodb-dockerize.git
docker build -q -t hunt_crawler:v1 crawler-docker/ # remove -q with outputs
```

## Run background
```shell
docker run -d --restart always -v /hunt/db:/hunt/db:rw --add-host localhost:172.17.0.2 -p 5555:5555 -p 5556:80 --name crawler_container hunt_crawler:v1 /bin/bash -c "/hunt/run.sh"
```

## Run foreground
```shell
docker run -i -t -v /hunt/db:/hunt/db:rw --add-host localhost:172.17.0.2 -p 5555:5555 -p 5556:80 --name crawler_runner_1 hunt_crawler:v1 /bin/bash
./run.sh
```

## Start docker service & Start & Attach ...
```shell
systemctl start docker
docker start crawler_runner_1
docker attach crawler_runner_1
```
> To stop a container, use CTRL-c. This key sequence sends SIGKILL to the container. If --sig-proxy is true (the default),CTRL-c sends a SIGINT to the container. You can detach from a container and leave it running using the CTRL-p CTRL-q key sequence

## docker basic command

#### stop & remove all container
```shell
docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)
```

#### show & remove images
```shell
docker images
docker rmi # remove a image
docker rmi $(docker images | grep "^" | awk -F ' ' '{print $3}') # remove all none named images
docker rmi $(docker images -q) # remove all images
```

#### cp file from container
```shell
docker cp :/file/path/within/container /host/path/target
```

#### command officially documentation
[docker docs](https://docs.docker.com/engine/reference/commandline/build/)

## design
![Crawler design](https://raw.githubusercontent.com/hiyali/node-mongodb-dockerize/master/assets/crawler_design.png "Crawler design")