Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/blackcub3s/job-automation
While employed at my town's local administration, I played a key role in advancing its digital transformation. I accomplished this by developing an information management and automatic report generation system, utilizing Python and LaTeX.
https://github.com/blackcub3s/job-automation
google-sheets latex latex-template python
Last synced: 28 days ago
JSON representation
While employed at my town's local administration, I played a key role in advancing its digital transformation. I accomplished this by developing an information management and automatic report generation system, utilizing Python and LaTeX.
- Host: GitHub
- URL: https://github.com/blackcub3s/job-automation
- Owner: blackcub3s
- Created: 2022-11-14T22:45:00.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2022-11-14T22:53:56.000Z (about 2 years ago)
- Last Synced: 2024-11-09T20:17:46.554Z (3 months ago)
- Topics: google-sheets, latex, latex-template, python
- Language: TeX
- Homepage:
- Size: 1.49 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# MOTIVATION
I was working at a city hall and I created this program to reduce a considerable ammount of administrative load.
I had to generate around 20 - 25 pdf reports each months for the relatives of the users of our service (one for each user of the service). Thus, I created a piece of software that allowed me to reduce around 10 hours of administrative load per month by automating the generation of these PDF reports using *google forms* & *google spreadsheets*, *Python* and *LaTeX.
The need for that aroused when I was managing a team of 8 social care assistants: their role was to support old people in town (dependent people, with slow mobility, etc.) and I was responsible for communicating with user's relatives in a comprehensive way, and inform them what our team did with their old relatives, while carrying social suport labour with the elderly.
# REPOSITORY DESCRIPTION
## INTRODUCTION
This repository contains the files with which PDF reports were automatically generated once a month. The pdfs were generated by adjusting a *LaTeX* template to each user using a *Python* script. Those templates were fitted with the google form spreadsheets data from the support workers.
## PROCEDURE TO AUTOMATICALLY GENERATE PDFs (execute)
The procedure, once a month, was as follows:
1. Downloading the [spreadsheets file](https://github.com/blackcub3s/job-automation/blob/main/Formulari%20actes%20(treball%20als%20barris)%20(Responses)%20bo.xlsx)[^1] with the data the social care assistants updated every day via google forms, replacing the previous file.
2. Gramatically correct the column *DESCRIPCIO*[^1], within [spreadsheets file](https://github.com/blackcub3s/job-automation/blob/main/Formulari%20actes%20(treball%20als%20barris)%20(Responses)%20bo.xlsx) with the data the social care assistants updated every day via google forms, replacing the previous file, as contents need to be polished.
3. Go to [__main__.py](https://github.com/blackcub3s/job-automation/blob/main/volcatgeActesScript/__main__.py), within [volcatgeActesScript](https://github.com/blackcub3s/job-automation/tree/main/volcatgeActesScript/) folder, and update the value arguments of the function call `main(data_incial, data_final)` where data_inicial and data_final are the first day of the month and the last day of the month, respectively (the program will just filter out the social care services between those two days so as the report for each user just contains one month, as we intended to).[^2]
4. By executing [__main__.py](https://github.com/blackcub3s/job-automation/blob/main/volcatgeActesScript/__main__.py) the program will, for one, take the data from the [spreadsheets file](https://github.com/blackcub3s/job-automation/blob/main/Formulari%20actes%20(treball%20als%20barris)%20(Responses)%20bo.xlsx) and, for the other, it'll use the string variables filled with *LaTeX* syntax that are stored in the template [plantillaLatex.py](https://github.com/blackcub3s/job-automation/blob/main/volcatgeActesScript/plantillaLatex.py)[^3] to finally generate a *.tex* file for each user, ready to be compiled with pdflatex. All these *.tex* files will be compiled at once using pdfLATEX by the same script, automatically creating the .pdfs[^4]. If you dive in the template [plantillaLatex.py](https://github.com/blackcub3s/job-automation/blob/main/volcatgeActesScript/plantillaLatex.py) you'll find the decision of writing latex code within the .py file was efficient, not a mere whim to use the language I feel most comfortable in: you can see that the variable *inici* has two python placeholders that will get the data in from the [spreadsheets file](https://github.com/blackcub3s/job-automation/blob/main/Formulari%20actes%20(treball%20als%20barris)%20(Responses)%20bo.xlsx) and that the variable *taula_long_variable* is responsible for generating, at each iteration [^5], the rows of the table presented to the families. The loop I am talking about is the following:
* `{{\\Large\\bfseries {} \\par}}` would get us the name of each user for each .tex template, being exported in the .tex file as `{\\Large\\bfseries NAME SURNAME SURNAME' \\par}`;
* `{{\\large del {} al {}.\\par}}`would get us `{\\large del dd/mm/yyyy al DD/MM/YYYY.\\par}`
* `"""\t\t\t {} & {} & {} \\\\ \n"""` would get us `11/05/2021 & Natasha Romanov & consequuntur ullam dolorum pariatur mollitia animi eum nam? \\`
5. You can consult the results seeing the .pdfs within the folder [informesPerUsuari](https://github.com/blackcub3s/job-automation/tree/main/volcatgeActesScript/informesPerUsuari). As an example, see, for a given user:
1. This are some of the rows from the [spreadsheets file](https://github.com/blackcub3s/job-automation/blob/main/Formulari%20actes%20(treball%20als%20barris)%20(Responses)%20bo.xlsx) (i.e. each highlighted row is a social service care day given to *user 16* by *Diana Prince* or *Natasha Romanov* that will comprehend part of the subset query that will appear only for user 16 on the pdf file):
2. And this one is the [.tex](https://github.com/blackcub3s/job-automation/blob/main/volcatgeActesScript/informesPerUsuari/_NOM16%20COGNOM16%20COGNOM16.tex) file for *user 16*, which will generate after compiling it the [.pdf](https://github.com/blackcub3s/job-automation/blob/main/volcatgeActesScript/informesPerUsuari/_NOM16%20COGNOM16%20COGNOM16.pdf) file as you can see down below (you can find all those files within [informesPerUsuari](https://github.com/blackcub3s/job-automation/tree/main/volcatgeActesScript/informesPerUsuari) subfolder):
* Cover letter
* table with social care workers for that given user:
[^1]: The column DESCRIPCIÓ contained a description of what each worker (see *ACOMPANYANT* column) made with a user (see *USUARI* column) for a given day (see *DATA* column).
[^2]: The dates within this arguments need to be strings, in format `dd/mm/yyyy`
[^3]: Note that the LaTeX code has a syntax such as `\\usepackage{{package name}}`instead of what a LaTeX code would have to look like `\usepackage{package name}`. This happens because the latex code was *not* stored within a *.tex* file but a string variable within a *.py* file. Python has certain characters that are not to be used in a string (for example, { } is used within strings as placeholders under the `"the value ten is: {}".format(10)` ). So, in order to write these strings in an actual .tex file and avoid python to mistake them they needed to be written twice, in order to override their syntactic behavious and, later on, render correctly in a .tex file.
[^4]: As you can see, .xlx files were also generated (they contain the same data, for other internal purposes).
[^5]: The code snippet that, at each iteration, fills the placeholders of *taula_long_variable* is as follows. It gets filled with the pandas dataframe that contains the filtered data for the given user: https://github.com/blackcub3s/job-automation/blob/5d606c4ecfa75dba8caf880f39d13debc30e0926/volcatgeActesScript/__main__.py#L81-L84