https://github.com/whatnick/austender_analyser

A learning project in Go to scrape and collate data from Austender Website
https://github.com/whatnick/austender_analyser

Last synced: 7 days ago
JSON representation

A learning project in Go to scrape and collate data from Austender Website

• Host: GitHub
• URL: https://github.com/whatnick/austender_analyser
• Owner: whatnick
• License: mit
• Created: 2023-01-22T05:09:08.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-11-05T11:02:27.000Z (about 2 months ago)
• Last Synced: 2024-11-05T11:42:02.465Z (about 2 months ago)
• Language: Go
• Size: 354 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md

Awesome Lists containing this project

README

        
# Austender Summarizer

There is always news around how much the government spends on contracting. However it is hard

to find a quick summary of figures in a transparent manner without digging through Austender

website, so public service report or relying on journalists' scraping skills.

Set this up as a serverless go development project to answer spend questions by organization name e.g. KPMG had 5136 tenders awarded, what was the total value ?

![KPMG Tenders](docs/KPMG_result_2023_01_22.png)

## Features

- CLI tool that scrapes for a keyword and returns roll-ups

- Cobra params parsing

- Standard 3-tier design (BE,FE,DB)

- Backend implementation in Golang

- Frontend implementation in Angular/React/HTMX TBD

- Fuzzy name matching in Google "did you mean" style

- Temporal aggregate spend on raw AUD values, not inflation adjusted (I am not an economist)

- Serverless hosting in AWS/Fly.io

- DynamoDB cache of Austender data downloadable as CSV

## Roadmap

- Download one search result - KPMG

- Identify fields to cache from Austender

- Cron to download and populate dynamodb/clickhouse with Austender info

- Webserver API to serve 1 set results

    - total spend on org all time

    - last 5 years

    - last year

- TDD

- Go project layout as it matures : https://github.com/golang-standards/project-layout