{"id":17339377,"url":"https://github.com/lubien/ufpa-advanced-databases-2018-assignment-01","last_synced_at":"2025-03-27T08:24:05.424Z","repository":{"id":112976007,"uuid":"151512936","full_name":"lubien/ufpa-advanced-databases-2018-assignment-01","owner":"lubien","description":"Implement some queries over self made db table with 1 billion entries of 8 bytes each for @gustavopinto class","archived":false,"fork":false,"pushed_at":"2019-05-04T19:23:42.000Z","size":7929,"stargazers_count":2,"open_issues_count":2,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-01T13:24:07.982Z","etag":null,"topics":["college-assignment","dbms","elixir"],"latest_commit_sha":null,"homepage":"http://gustavopinto.org/teaching/bd2/exercise","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lubien.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-04T03:20:19.000Z","updated_at":"2019-05-04T19:23:44.000Z","dependencies_parsed_at":"2023-03-10T16:00:35.434Z","dependency_job_id":null,"html_url":"https://github.com/lubien/ufpa-advanced-databases-2018-assignment-01","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lubien%2Fufpa-advanced-databases-2018-assignment-01","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lubien%2Fufpa-advanced-databases-2018-assignment-01/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lubien%2Fufpa-advanced-databases-2018-assignment-01/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lubien%2Fufpa-advanced-databases-2018-assignment-01/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lubien","download_url":"https://codeload.github.com/lubien/ufpa-advanced-databases-2018-assignment-01/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245807289,"owners_count":20675499,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["college-assignment","dbms","elixir"],"created_at":"2024-10-15T15:41:37.491Z","updated_at":"2025-03-27T08:24:05.405Z","avatar_url":"https://github.com/lubien.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Queries over binary files using the BEAM ecosystem\n\nCode used for [@gustavopinto](https://github.com/gustavopinto) Advanced Databases classroom [first assignment](http://gustavopinto.org/teaching/bd2/exercise).\n\n[Full report](https://github.com/lubien/ufpa-advanced-databases-2018-assignment-01-report/releases) is written in portuguese.\n\nOverview:\n\n  * Create a single-file single-table database to save 1 billion people.\n  * Each person is a 64 bits tuple with gender (1), age (7), monthly income (10), scholarity (2), idiom (12), country (8) and coordinates (24).\n  * Implement 7 queries from the assignment and 3 of your choice.\n  * Export the binary database to a relational DBMS.\n  * Compare results.\n\n## Setup\n\nGenerate the binary file.\n\n```sh\nhead -c 8000000000 \u003c/dev/urandom \u003epeople.db\n```\n\nSetup PostgreSQL. You may change env variables, but it's optional. Below are the default.\n\n```sh\nexport DB_NAME=ufpa-databases-2\nexport DB_USER=postgres\n\nmake prepare\n```\n\nExport the binary database to CSV then import to PostgreSQL.\n\n```sh\nmake dump-database\nmake import-dump\n```\n\nPrepare Elixir.\n\n```sh\nmix deps.get\nmix compile\n```\n## Query\n\nElixir.\n\n```sh\n# from 1 to 10\nmix query --db people.db --query 1\n```\n\nPostgreSQL.\n\n```sh\n# from 1 to 10\nmake query-1\n```\n\n## Results\n\nThis machine was used:\n\n```sh\nλ neofetch\nOS: Manjaro Linux x86_64 \nHost: B250M-D3H \nKernel: 4.14.77-1-MANJARO \nUptime: 12 hours, 58 mins \nPackages: 1024 (pacman) \nShell: bash 4.4.23 \nResolution: 1360x768, 2560x1080 \nDE: Xfce \nTheme: Vertex-Maia [GTK2], Breath [GTK3] \nIcons: Vertex-Maia [GTK2], hicolor [GTK3] \nTerminal: xfce4-terminal \nTerminal Font: Monospace 12 \nCPU: Intel i5-7400 (4) @ 3.500GHz \nGPU: NVIDIA GeForce GT 610 \nMemory: 3204MiB / 7939MiB\n\nλ lscpu\nArquitetura:                x86_64\nModo(s) operacional da CPU: 32-bit, 64-bit\nOrdem dos bytes:            Little Endian\nCPU(s):                     4\nLista de CPU(s) on-line:    0-3\nThread(s) per núcleo:       1\nNúcleo(s) por soquete:      4\nSoquete(s):                 1\nNó(s) de NUMA:              1\nID de fornecedor:           GenuineIntel\nFamília da CPU:             6\nModelo:                     158\nNome do modelo:             Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz\nStep:                       9\nCPU MHz:                    800.050\nCPU MHz máx.:               3500,0000\nCPU MHz mín.:               800,0000\nBogoMIPS:                   6002.00\nVirtualização:              VT-x\ncache de L1d:               32K\ncache de L1i:               32K\ncache de L2:                256K\ncache de L3:                6144K\nCPU(s) de nó0 NUMA:         0-3\n\nλ sudo hdparm -Tt /dev/sda\n\n/dev/sda:\n Timing cached reads:   26046 MB in  1.99 seconds = 13068.87 MB/sec\n Timing buffered disk reads: 392 MB in  3.00 seconds = 130.50 MB/sec\n```\n\nQuery       | First (DBMS)    | First (Bin)     | Mean (DBMS)  | Mean (Bin)\n------------|-----------------|-----------------|--------------|-----------\n1           | 748s            | 134s            | 640s         | 134s\n2           | -               | 456s            | -            | 453s\n3           | 629s            | 160s            | 622s         | 161s\n4           | 620s            | 173s            | 614s         | 163s\n5           | 705s            | 59s             | 618s         | 59s\n6           | 861s            | 63s             | 623s         | 61s\n7           | 631s            | 67s             | 604s         | 62s\n8           | 619s            | 60s             | 622s         | 59s\n9           | 639s            | 59s             | 626s         | 60s\n10          | 622s            | 61s             | 628s         | 60s\n\nQuery 2 never finished on PostgreSQL so...\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flubien%2Fufpa-advanced-databases-2018-assignment-01","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flubien%2Fufpa-advanced-databases-2018-assignment-01","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flubien%2Fufpa-advanced-databases-2018-assignment-01/lists"}