https://github.com/chemyl/api_scraper_template

Easy api data-crawler
https://github.com/chemyl/api_scraper_template

data-crawling java11 mvc-architecture rest-api spring-boot

Last synced: about 2 months ago
JSON representation

Easy api data-crawler

• Host: GitHub
• URL: https://github.com/chemyl/api_scraper_template
• Owner: chemyl
• Created: 2023-06-08T06:12:23.000Z (over 1 year ago)
• Default Branch: master
• Last Pushed: 2023-06-08T06:31:51.000Z (over 1 year ago)
• Last Synced: 2024-10-20T09:09:39.534Z (3 months ago)
• Topics: data-crawling, java11, mvc-architecture, rest-api, spring-boot
• Language: Java
• Homepage:
• Size: 4.88 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md

Awesome Lists containing this project

README

        
## api-response scraper ###

* SpringBoot

* Java 11

* Maven

* Apache POI

* Rest-Assured

* Lombok

* Log4j

### functionalities

* Scheduled

* Store URLs in application.properties

* ApiController sends prepared CRUD request to API 

* Response schema validation

* DataStorageImpl convert Response to POJO

* Create XLSX or find existed file in project dir 

* Add new row of POJO Data to the end of the file.

### how to run

* Create Maven project

* Add dependencies

* Configure project directories

* Start as SpringBoot Application

  * Save as .jar file

  * Create Docker Image

  * Run as Docker Container