{"id":19664425,"url":"https://github.com/ljnath/getdomainage","last_synced_at":"2025-06-15T21:33:12.833Z","repository":{"id":47232228,"uuid":"191649430","full_name":"ljnath/getDomainAge","owner":"ljnath","description":"getDomainAge is a web application which can provide the age of a given domain name. The intention of this project is to demonstrate flask for API development, SQLAlchemy for ORM and beautifulsoup for HTML parsing.","archived":false,"fork":false,"pushed_at":"2025-06-13T11:48:39.000Z","size":3659,"stargazers_count":2,"open_issues_count":16,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-13T12:45:45.000Z","etag":null,"topics":["beautifulsoup","docker","domain","flask","flask-api","flask-application","flask-restful","flask-server","flask-sqlalchemy","flask-web","job-queue","job-scheduler","producer-consumer","python","python3","scraping-websites","sqlalchemy","sqlalchemy-database","sqllite","whois"],"latest_commit_sha":null,"homepage":"https://ljnath.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ljnath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-06-12T21:48:45.000Z","updated_at":"2023-04-08T12:36:03.000Z","dependencies_parsed_at":"2024-01-11T23:19:17.789Z","dependency_job_id":"713ad066-be94-4657-b22d-bc9f6c34eeae","html_url":"https://github.com/ljnath/getDomainAge","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ljnath/getDomainAge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljnath%2FgetDomainAge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljnath%2FgetDomainAge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljnath%2FgetDomainAge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljnath%2FgetDomainAge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ljnath","download_url":"https://codeload.github.com/ljnath/getDomainAge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ljnath%2FgetDomainAge/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260053248,"owners_count":22951844,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautifulsoup","docker","domain","flask","flask-api","flask-application","flask-restful","flask-server","flask-sqlalchemy","flask-web","job-queue","job-scheduler","producer-consumer","python","python3","scraping-websites","sqlalchemy","sqlalchemy-database","sqllite","whois"],"created_at":"2024-11-11T16:17:47.225Z","updated_at":"2025-06-15T21:33:12.786Z","avatar_url":"https://github.com/ljnath.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# getDomainAge\n### Version : 0.3\n\nAuthor: Lakhya Jyoti Nath (ljnath)\u003cbr\u003e\nDate: June 2019 - July 2021\u003cbr\u003e\nEmail: ljnath@ljnath.com\u003cbr\u003e\nWebsite: https://ljnath.com\n\n[![](https://img.shields.io/github/license/ljnath/getdomainage)](https://github.com/ljnath/getDomainAge)\n![Lines of code](https://img.shields.io/tokei/lines/github/ljnath/getDomainAge)\n![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/ljnath/getDomainAge)\n![GitHub search hit counter](https://img.shields.io/github/search/ljnath/getDomainAge/getDomainAge)\n[![codecov](https://codecov.io/gh/ljnath/getDomainAge/branch/master/graph/badge.svg?token=nwi6SmUvjF)](https://codecov.io/gh/ljnath/getDomainAge)\n\n![GitHub Workflow Status(https://github.com/ljnath/getDomainAge/actions)](https://img.shields.io/github/workflow/status/ljnath/getDomainAge/E2E%20getDomainAge%20workflow)\n[![](https://img.shields.io/docker/pulls/ljnath/getdomainage)](https://hub.docker.com/r/ljnath/getdomainage)\n![Docker Image Size (tag)](https://img.shields.io/docker/image-size/ljnath/getdomainage/latest)\n[![](https://images.microbadger.com/badges/version/ljnath/getdomainage.svg)](https://microbadger.com/images/ljnath/getdomainage)\n\n\n\n\n## What is it ?\n\ngetDomainAge is a web application which can provide the age of a given domain name.\nUser can login using their email and they can add a job with one or more URLs. The application will pick these jobs sequentially and process these.\nJob processing includes parsing the domain name from the URL, fetching its registration date, calculating its age (in days) and saving the result to a CSV file. It will also send an email to the user with the result file attached.\n\nThe intention of this project is to demonstrate flask for API development, SQLAlchemy for ORM and beautifulsoup for HTML parsing.\n\u003c/br\u003e\u003c/br\u003e\n\n## How it works ?\n\nThis application mainly consists of 2 components viz. application-server and worker-service.\n1. Application server - This is built around flask which has a handful of APIs for user login, logout, addition of job, viewing of job and file download.\n\n2. Worker service - This is an independent component which runs on a separate thread. It is responsible for picking up task from the database and completing those.\n\u003c/br\u003e\u003c/br\u003e\n\n## What is the workflow ?\n\n- It works by scrapping the domain registration date from https://www.whois.com/\n- User places their job request using the web portal where they can enter one or more URLs in comma or new-line separated manner. Jobs are stored in SQLite DB using SQLAlchemy\n- A worker service looks for pending jobs in the database and picks it up.\n- The worker service updates the job status to `RUNNING` and starts processing it using `DomainService`.\n- `DomainService` checks for the domain registered date in local cache, if it is not there it parse that information from https://www.whois.com using `Beautiful Soup` and finally the result is saved as a CSV file.\n- Next `EmailService` sends out an email to the job requestor with the result CSV attached.\n\u003c/br\u003e\u003c/br\u003e\n\n## Application configuration\nThe configuration of the application is done using a JSON file. This file needs to be passed as argument while starting the application.\nFollowing is the explained config file \n```json\n{\n    # complete path to the workspace directory\n    # incase of docker, this should prefix with /workspace as shown below\n    \"workspace_path\": \"/workspace/my_workspace\",\n\n    # application server configuration\n    \"server\": {\n      \"host\": \"0.0.0.0\",\n      \"port\": 80,\n      \"debug\": false\n    },\n\n    # email configuration\n    \"smtp\": {\n      \"host\": \"\",\n      \"port\": 25,\n      \"username\": \"\",\n      \"password\": \"\",\n      \"sender_email\": \"\"\n    },\n\n    # application configuration\n    \"application\": {\n      \"jobs_per_page\": 10,\n      \"session_timeout\": 10\n    }\n  }\n```\n\u003c/br\u003e\n\n\n## How to run in local ?\n\n- Clone the git repository\n```\ngit clone https://github.com/ljnath/getDomainAge.git\n```\n- update the config.json file as per your need\n- start the application using the following command\n```\npython runserver.py --config config.json\n```\n\u003c/br\u003e\n\n## How to run using docker ?\n\nTo run the getdomainage docker container, you need to mount a local directory to docker volume `/workspace`.\nThis directory should contain the application configuration file (`config.json`).\nThe SQLite database, cache, application logs and results will be created under the `/workspace` volume, so please make to have correct permissions.\n\n```docker\ndocker run -p 5000:{server-port-as-per-config-json} -v /home/user/getDomainAge:/workspace -e CONFIG_FILE=config.json ljnath/getdomainage\n```\n\u003c/br\u003e\u003c/br\u003e\n\n## How to use ?\n\n- Login with your email ID\n- Add a new job where you need to enter your list of URLs in a form, which can be comma or new-line separated\n- Wait for your job to be processed. Once processed, you will receive an email with the age of your selected domain.\n\u003c/br\u003e\u003c/br\u003e\n\n## Sample\n\n\u003cimg src=\"demo.jpg\" aligh=\"center\"/\u003e\n\u003cbr\u003e\u003cbr\u003e\n\n\n## Give a Star!⭐️\n\nIf you find this repository useful, please give it a star.\nThanks in advance !\n\u003c/br\u003e\u003c/br\u003e\n\n## License\n\nCopyright © 2021 [Lakhya's Innovation Inc.](https://github.com/ljnath/) under the [MIT License](https://github.com/ljnath/getDomainAge/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fljnath%2Fgetdomainage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fljnath%2Fgetdomainage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fljnath%2Fgetdomainage/lists"}