https://github.com/itamarc/jobsdatadashboard

My personal project to create a system with various technologies, just for knowledge. A job seeker system to grab data from several sources, compile them and shows it in graphical ways.
https://github.com/itamarc/jobsdatadashboard

documentation java pet-project python study

Last synced: about 1 month ago
JSON representation

My personal project to create a system with various technologies, just for knowledge. A job seeker system to grab data from several sources, compile them and shows it in graphical ways.

• Host: GitHub
• URL: https://github.com/itamarc/jobsdatadashboard
• Owner: itamarc
• License: gpl-3.0
• Created: 2021-04-20T22:15:44.000Z (about 5 years ago)
• Default Branch: main
• Last Pushed: 2021-09-21T20:48:37.000Z (almost 5 years ago)
• Last Synced: 2025-01-20T10:48:19.988Z (over 1 year ago)
• Topics: documentation, java, pet-project, python, study
• Homepage: https://itamarc.github.io/JobsDataDashboard/
• Size: 1.19 MB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Jobs Data Dashboard Project

My personal project to create a system with various technologies, just for knowledge.

A jobs data system to grab data from several sources, compile them and shows it in a dashboard.

This repository have only the project documentation.

The documentation was produced using Libre Office.

Some documents about the project:

* [Logging](doc/Logging.md)

* [Grabber Job](doc/GrabberJob.md)

* [Web UI](doc/WebUI.md)

## Project Structure

Each component of the project will have a separate repository, with everything that the component needs to compile.

The documentation will have its own repository.

**List of repositories:**

- [JobsDataDashboard](https://github.com/itamarc/JobsDataDashboard)

  - *General documentation of the project (this repository).*

- [JobsDataDashboard-jddgrabber](https://github.com/itamarc/JobsDataDashboard-jddgrabber)

  - *Component made in Python to grab the data from the jobs online services.*

    - conf

    - src

    - test

    - requirements.txt

- [JobsDataDashboard-jddlogfunc](https://github.com/itamarc/JobsDataDashboard-jddlogfunc)

  - *Component made in Java to get the log messages out of a SQS queue and insert it into a Collection in MongoDB.*

    *It runs as an AWS Lambda function.*

    - src/main/java

    - src/test/java

    - pom.xml

- JobsDataDashboard-jddwebui

  - *Component with the web user interface.*

![Components overview diagram](doc/JDD-ComponentsOverview.svg "Components overview")

## Tools adopted

Here I will list the tools I'm using for each task in the project.

This list will be updated each time I change a tool.

* For development in Python (version 3.9)

  - Visual Studio Code

  - libraries listed in `requirements.txt`

* For development in Java 11

  - Visual Studio Code

  - Maven

    - dependencies listed in `pom.xml`

* For code versioning

  - GitHub and GitHub Desktop

  - Integration and unit tests with pytest/JUnit triggered by GH Actions

* Cloud services - AWS

  - Linux server with a cron job to grab data

  - Serverless webservices (Lambda functions)

  - MongoDB Atlas

  - *(TBD: web interface)*

* For documentation:

  - Markdown

  - Inkscape SVG

![Technology overview diagram](doc/JDD-TechnologyOverview.svg "Technology overview")