{"id":19907297,"url":"https://github.com/captainirs/transfinitte","last_synced_at":"2025-10-10T16:35:24.166Z","repository":{"id":67075961,"uuid":"551693203","full_name":"CaptainIRS/transfinitte","owner":"CaptainIRS","description":null,"archived":false,"fork":false,"pushed_at":"2022-11-09T20:11:31.000Z","size":1346,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-01T08:12:49.797Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://family-tree.captainirs.dev/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CaptainIRS.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-14T22:44:02.000Z","updated_at":"2022-10-20T21:13:35.000Z","dependencies_parsed_at":"2023-03-16T22:00:18.531Z","dependency_job_id":null,"html_url":"https://github.com/CaptainIRS/transfinitte","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CaptainIRS/transfinitte","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainIRS%2Ftransfinitte","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainIRS%2Ftransfinitte/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainIRS%2Ftransfinitte/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainIRS%2Ftransfinitte/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CaptainIRS","download_url":"https://codeload.github.com/CaptainIRS/transfinitte/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CaptainIRS%2Ftransfinitte/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279004693,"owners_count":26083750,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T20:39:57.113Z","updated_at":"2025-10-10T16:35:24.146Z","avatar_url":"https://github.com/CaptainIRS.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Family tree builder for TransfiNITTe'22\n\nSubmission for TransfiNITTe'22 hackathon under the problem statement \"Family Tree\" by BharatX\n\n## Problem Statement\n\nCreatre a Family Tree of all families in India (Pan India) using Electoral Roll Pdf available at nsvp.in. You are free to use any technology or any hack to crack the problem statement. [Link](https://quartz-artichoke-67d.notion.site/Hackathon-Problem-Statement-7f6ebf8bbc694cd18c355eb9433d1197) to elaborated problem statement.\n\n## Solution\n\nWe have developed an approach to dynamically generate the family tree:\n\n* Query the electoral search website with user's details.\n* Get the user's polling booth from the details.\n* Download the electoral roll PDF for that booth from the website of that state's Chief Electoral Officer.\n* Since the PDFs contain the electoral roll as image data, we use Optical Character Recognition (OCR) to extract the text from the PDF.\n* This text is parsed to generate a JSON file with all the people's details, which are cached in the local filesystem for faster access.\n* This JSON file is then used to generate the family tree of a person or a constituency.\n* The family trees are generated by finding ancestors and spouses from this data for people belonging to the same house.\n* The trees can be visualised using our web application.\n\n## ETL pipelines\nETL, which stands for extract, transform and load, is a data integration process that combines data from multiple data sources into a single, consistent data store that is loaded into a data warehouse or other target system.\n\nFor our project, we have used the following ETL pipeline:\n* Extract: We extract the data from the electoral roll PDFs. This is done for state of Tamil Nadu. This code is present in the `spark` folder `extract_tn.ipynb`.\n* Transform: We transform the extracted data using the `transform.ipynb` notebook in the `spark` folder. This notebook generates the JSON file for the constituency.\n* Load: We load the JSON file into the local filesystem for the APIs to use.\n\n## NEO4J\n\nFeatures:\n* Visualise the family tree of a person or a constituency.\n* Query the database to find the relationships between people.\n\n#### Example Queries\n* Find the spouse of a person\n```\nMATCH (n:Person {name: \"S. S. Rajendran\"})-[:SPOUSE]-\u003e(spouse) RETURN spouse\n```\n* Find the children of a person\n```\nMATCH (n:Person {name: \"S. S. Rajendran\"})-[:CHILD]-\u003e(child) RETURN child\n```\n* Find the person in house number 1\n```\nMATCH (n:Person {house_number: \"1\"}) RETURN n\n```\n\n\n## Tech Stack\n\n![tech-stack-image](https://i.imgur.com/IrmcFe8.png)\n\n## Deployment\n\nThe web application is hosted [here](https://family-tree.captainirs.dev).\nThe backend API is hosted [here](https://api.family-tree.captainirs.dev).\n\n## API Information\n\n[Postman collection link](https://documenter.getpostman.com/view/5489887/2s84Dmy4Vq)\n[Swagger documentation link](https://api.family-tree.captainirs.dev/docs)\n\n* GET /state_list - Returns a list of states and its state codes\n\n* GET /district_list - Returns a list of districts for a state and its district number\n\n  Query Parameters:\n  * state_no - State code for which the districts are to be fetched\n\n* GET /assembly_list - Returns a list of of assembly constituencies for a district and its assembly number\n  \n  Query Parameters:\n  * state_no - State code for which the districts are to be fetched\n  * dist_no - District number for which the assembly constituencies are to be fetched\n\n* POST /tree - Returns a family tree for a person\n  \n  Query Parameters:\n  * name - Name of the person\n  * relative_name - Name of the relative\n  * dob - Date of birth of the person in YYYY-MM-DD\n  * state - State code of the state in which the person resides\n  * gender (Optional param) - Gender of the person: M - Male, F - Female, O - Other\n  * district (Optional param) - District number of the district in which the person resides\n  * ac (Optional param) - Assembly constituency number of the constituency in which the person resides\n\n* POST /trees - Returns all family trees for a polling booth\n\n  Query Parameters:\n  * state - State code of the state in which the constituency resides\n  * district - District number of the district in which the constituency resides\n  * ac - Assembly constituency number of the constituency\n  * part_no - Part number of the polling booth\n\n## Local Docker deployment instructions\n\n### Backend\n\n* Go to `backend` directory\n* Copy the `.env.example` file to `.env` and fill in the required values\n* Run `git submodule update --init --recursive`\n* Go to `./indic-trans`\n* Run `pip install -r requirements.txt`. It is required for the below command.\n* Run `python setup.py sdist`\n\n### Frontend\n\n* Change BACKEND_URL const to your backend URL in `frontend/src/config.js`\n\n### Spark\n\n* Go to `spark` directory\n* Run `build.sh` to build the docker images\n\nRun `docker-compose up -d` in the root directory to start the application.\n\n\u003e **_NOTE:_**  Some of the states have ratelimiting on their electoral roll PDFs or have blacklisted the IP address of the cloud server. In such cases, the PDFs cannot be downloaded. The application will still work for other states.\n\n## Screenshots\n\n#### Family tree search for a single person\n\n![single_person](./screenshots/single_person.png)\n![family_tree](./screenshots/family_tree.png)\n\n#### Family tree search for a polling booth\n\n![polling_booth](./screenshots/polling_booth.png)\n![family_trees](./screenshots/family_trees.png)\n\n#### NEO4J\n\n![neo4j](./screenshots/neo4j.png)\n\n#### Swagger\n![swagger](./screenshots/swagger.jpg)\n\n#### Spark\n![spark](./screenshots/spark.jpg)\n![master](./screenshots/master.jpg)\n![worker](./screenshots/workers.jpg)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcaptainirs%2Ftransfinitte","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcaptainirs%2Ftransfinitte","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcaptainirs%2Ftransfinitte/lists"}