{"id":18368840,"url":"https://github.com/rpstreef/urlshortener","last_synced_at":"2026-04-09T06:42:35.290Z","repository":{"id":212011246,"uuid":"91766276","full_name":"rpstreef/urlshortener","owner":"rpstreef","description":"URL Shortener","archived":false,"fork":false,"pushed_at":"2019-12-24T08:06:06.000Z","size":303,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-15T20:53:16.350Z","etag":null,"topics":["express-js","javascript","node-js","react-native","reactjs","url-shortener","webpack"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rpstreef.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-05-19T04:44:24.000Z","updated_at":"2022-09-08T14:44:37.000Z","dependencies_parsed_at":"2023-12-12T04:39:29.703Z","dependency_job_id":null,"html_url":"https://github.com/rpstreef/urlshortener","commit_stats":null,"previous_names":["rpstreef/urlshortener"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpstreef%2Furlshortener","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpstreef%2Furlshortener/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpstreef%2Furlshortener/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rpstreef%2Furlshortener/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rpstreef","download_url":"https://codeload.github.com/rpstreef/urlshortener/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248281426,"owners_count":21077423,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["express-js","javascript","node-js","react-native","reactjs","url-shortener","webpack"],"created_at":"2024-11-05T23:27:38.648Z","updated_at":"2025-12-30T18:59:25.080Z","avatar_url":"https://github.com/rpstreef.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# URL Shortener\n\n## Install instructions\nEverything runs locally, there's no database to install since it uses json-server to host the database from a json file.\n\nnpm start will run the development environment with nodemon, compile with babel and start the express server backend.\n\n```\ngit clone https://github.com/rpstreef/urlshortener.git\nnpm install\nnpm start\n```\n\n## Code structure\n- API\n    - Services\n        - Database\n            - Checks existing ShortURL entries, if not present writes new shortURL entry.\n        - Validation URL library\n            - Check via regex if URL entered via input field conforms to URL format.\n        - Random numbers for database entry\n            - Generates a random number, used for generating the shorturl and as ID in the database for that ShortURL entry.\n            - Note: If there are many URL's generated, collision is likely(!)\n- Public, files served on the client-side.\n    - css\n        - style sheets\n    - views\n        - React layouts, routes and react partial view implementations.\n    - bundle.js, generated javascript from Babel compilation.\n- Root\n    - Server initialization files and certificates for SSL, HTTPS\n    - Configuration files for development, webpack, npm packager, eslint.\n\n## Security issues\n    Security issues with this solution:\n    - SQL Injection\n        - Unprotect database REST api's can be vulnerable to scripting atacks deleting/aletering the database content.\n        - Solution:\n            - Only allow server ip/subnet to execute on the database and put the database in a subnet not accessible to the internet.\n    - Brute force shortened URL's:\n        - In this case of OneDrive, if the end user does not enforce strict sharing rules, their files are out in the open.\n        - http://www.pcmag.com/news/343781/url-shorteners-convenient-but-a-potential-security-risk\n        - Solution:\n            - Create longer URL's that are much harder to bruteforce, ultimately it's a battle that can't be won. Plus, shortened URL's will loose their purpose if they're too long.\n    - Mallware\n        - I can shorten a url, hide mallware behind it that will auto-download and execute\n        - Solution:\n            - Implement a URL scanning solution to verify integrity of the URL and it's contents. Hard to implement and hard to verify. Contents can be altered after URL shortening process has been completed.\n    - Not secured REST API's\n        - REST API's for URL shortening is open to the world for access.\n        - Solution: use API keys to secure access to private Rest endpoints. Only the application with access to the API secret key can execute on the endpoint.\n            - https://stormpath.com/blog/express-sample-api-key-management\n\n## Scalability issues\n    - Database writes\n        - Once more users use the service, writes will increase.\n        - Solution:\n            - Scale vertically, increase the size of the machine running your database.\n            - Scale horizontal, increase the number of database instances that can write data. Requires master-master setup for your database solution. MySQL, Oracle, NoSQL (OrientDB, ArangoDB)\n    - Database reads\n        - For every shorturl entry into the database, there's a read action to confirm there's no duplicate. It's a simple key - value based search. Either NoSQL or Relational database would be suitable.\n        - Solution:\n            - Scale vertically, increase the size of the database instance.\n            - Scale horizontal, increase the number of database \"slaves\" to scale out read capacity.\n    - Parsing URLS\n        - Conversion of a number to a shortURL.\n        - Requires quite a lot conversions to put load on the hosting machine(!).\n        - Solution:\n            - Host conversion on a separate EC2 instance, load-balanced.\n            - OR host on Serverless infrastructure like AWS Lambda which can scale automatically according to load.\n    - Serving website\n        - There's very little in the way of server processing done, again this requires quite high numbers of traffic.\n        - Solution:\n            - Combination of static file serving via a service like Amazon S3 and dynamic on a load balanced EC2 instance.\n            - AWS CloudFront for localized caching and taking load off your EC2 instance. It can cache dynamic content with TTL=0.\n\n    - General architecture:\n        - This solution is all hosted on one machine and a client side for the fronted.\n        - Solution:\n            - Host services such as database; bijection algorithm from separate instances. \n            - Let the page serving instance call the services via rest endpoints.\n            - Database can be installed on a dedicated instance.\n            - Conversion algorithm can be served from a Serverless function.\n            - Web tier, static on S3, dynamic on EC2 load-balanced and served via CloudFront with caching.\n            \n## Sources\n\n### URL shortener\n- Theory, Wiki bijection\n    - https://en.wikipedia.org/wiki/Bijection\n- How To:\n    - http://phpden.info/How-to-Build-a-URL-Shortner-like-bit.ly-or-goo.gl\n    - http://stackoverflow.com/questions/742013/how-to-code-a-url-shortener\n    - http://blog.gainlo.co/index.php/2016/03/08/system-design-interview-question-create-tinyurl-system/?utm_campaign=quora\u0026utm_medium=What+are+the+http%3A%2F%2Fbit.ly+and+t.co+shortening+algorithms%3F\u0026utm_source=quora\n\n### Promise based libraries\n- Fetch\n    - https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch\n- Request, request promise\n    - https://github.com/request/request-promise\n\n### ExpressJS best practices\n- Express 4 API documentation:\n    - https://expressjs.com/en/4x/api.html#router\n- Best practices\n    - https://expressjs.com/en/advanced/best-practice-performance.html#in-code\n- Error handling\n    - http://expressjs.com/en/guide/error-handling.html\n- Setting up SSL\n    - http://blog.ayanray.com/2015/06/adding-https-ssl-to-express-4-x-applications/\n\n### Frontend, React\n- React Engine:\n    - https://github.com/paypal/react-engine/\n### Various\n- URL Regex\n    - https://mathiasbynens.be/demo/url-regex\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frpstreef%2Furlshortener","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frpstreef%2Furlshortener","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frpstreef%2Furlshortener/lists"}