{"id":20033913,"url":"https://github.com/tristanlatr/masswappalyzer","last_synced_at":"2025-05-05T05:31:24.309Z","repository":{"id":108632011,"uuid":"292441093","full_name":"tristanlatr/MassWappalyzer","owner":"tristanlatr","description":" Run Wappalyzer asynchronously on a list of URLs and generate a Excel file containing all results.   ","archived":false,"fork":false,"pushed_at":"2022-03-14T18:00:59.000Z","size":1294,"stargazers_count":12,"open_issues_count":1,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-08T17:21:36.432Z","etag":null,"topics":["bulk","excel","mass","multiple","urls","wappalyzer","websites"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tristanlatr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-03T02:05:14.000Z","updated_at":"2024-12-22T13:29:14.000Z","dependencies_parsed_at":"2023-05-02T14:31:45.903Z","dependency_job_id":null,"html_url":"https://github.com/tristanlatr/MassWappalyzer","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanlatr%2FMassWappalyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanlatr%2FMassWappalyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanlatr%2FMassWappalyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanlatr%2FMassWappalyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tristanlatr","download_url":"https://codeload.github.com/tristanlatr/MassWappalyzer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252446009,"owners_count":21749154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bulk","excel","mass","multiple","urls","wappalyzer","websites"],"created_at":"2024-11-13T09:47:57.822Z","updated_at":"2025-05-05T05:31:24.302Z","avatar_url":"https://github.com/tristanlatr.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mass Wappalyzer\n\nRun [Wappalyzer](https://github.com/aliasio/wappalyzer) asynchronously on a list of URLs and generate a Excel file containing all results.   \n\nThe generated Excel file will have 2 sheets.  \n\nFirst sheet contains one column per technology seen and one row per analyzed website, additionnaly, a `\"Url\"`  column will aways be present.   \n\nSecond sheet contains one column per analyzed website and one row per seen technology.    \n\nCSV and JSON format are also supported.      \n\n### Install\n\nInstall **Python module**  \n\n    python3 -m pip install -U git+https://github.com/tristanlatr/MassWappalyzer.git\n\n### Requirements\n\n- **Wappalyzer CLI** if you want to use the official Javascript Wappalyzer CLI (shows more details and configurable with `--wappalyzerargs`)  \n\n  - [Docker](https://hub.docker.com/r/wappalyzer/cli/), pull image with `docker pull wappalyzer/cli`\n\n  - [NPM](https://www.npmjs.com/package/wappalyzer), install with `npm i -g wappalyzer`  \n\n- **None** if you use the full-python Wappalyzer implementation: [python-Wappalyzer](https://github.com/chorsley/python-Wappalyzer)\n\n*MassWappalyzer should detect if Wappalyzer CLI is installed and use appropriate implementation*\n\n### Usage\n\n    python3 -m masswappalyzer -i sample/top-20-websites-2020.txt -o sample/top-20-websites-2020.xlsx\n\nOutput: \n```\nMass Wappalyzer\nUsing wappalyzer/cli: docker run --rm wappalyzer/cli\nAnalyzing...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:46\u003c00:00,  2.31s/it]\nAll technologies seen: \n['AWSCertificateManager', 'Akamai', 'AkamaiBotManager', 'AmazonCloudfront', 'AmazonWebServices', 'Apache', 'AppleSignin', 'Babel', 'Bootstrap', 'BugSnag', 'CartFunctionality', 'DigiCert', 'DoubleClickforPublishersDFP', 'Envoy', 'Express', 'Facebook', 'FacebookSignin', 'Fastly', 'GoogleAnalytics', 'GoogleFontAPI', 'GoogleSignin', 'GoogleTagManager', 'GoogleWorkspace', 'Hammerjs', 'Hoganjs', 'Lodash', 'MediaWiki', 'Microsoft365', 'MobX', 'Nextjs', 'Nginx', 'Nodejs', 'OneTrust', 'PHP', 'Polyfill', 'Polymer', 'Prebid', 'Python', 'React', 'Reddit', 'RequireJS', 'Sentry', 'Sizmek', 'Stripe', 'Underscorejs', 'Varnish', 'YouTube', 'Zipkin', 'comScore', 'jQuery', 'reCAPTCHA', 'webpack']\nCreating Excel file sample/top-20-websites-2020.xlsx\nDone\n```\n\n### Excel file\n\n![Excel file](https://raw.githubusercontent.com/tristanlatr/MassWappalyzer/master/sample/top-20-websites-2020.png \"Excel file\")\n\n### Full help\n\n```\nusage: python3 -m masswappalyzer [-h] -i Input file [-o Output file]\n                                 [-f Format] [-w Wappalyzer path]\n                                 [-c Wappalyzer arguments] [-a Number] [-p]\n                                 [-v]\n\nRun Wappalyzer asynchronously on a list of URLs and generate a Excel file\ncontaining all results.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -i Input file, --inputfile Input file\n                        Input file, the file must contain 1 host URL per line.\n                        (default: None)\n  -o Output file, --outputfile Output file\n                        Output file containning all Wappalyzer informations.\n                        (default: MassWappalyzerResults)\n  -f Format, --outputformat Format\n                        Indicate output format. Choices: 'xlsx', 'csv',\n                        'json'. (default: xlsx)\n  -w Wappalyzer path, --wappalyzerpath Wappalyzer path\n                        Indicate the path to the Wappalyzer CLI executable.\n                        Auto detect by default. Use \"python-Wappalyzer\" if\n                        Wappalyzer CLI not found. (default: None)\n  -c Wappalyzer arguments, --wappalyzerargs Wappalyzer arguments\n                        Indicate the arguments of the Wappalyzer CLI command\n                        as string. Not applicable if using \"python-\n                        Wappalyzer\". (default: --pretty --probe --user-\n                        agent=\"Mozilla/5.0\")\n  -a Number, --asynch_workers Number\n                        Number of websites to analyze at the same time\n                        (default: 5)\n  -p, --python          Use full Python Wappalyzer implementation \"python-\n                        Wappalyzer\" even if Wappalyzer CLI is installed with\n                        NPM or docker. (default: False)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftristanlatr%2Fmasswappalyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftristanlatr%2Fmasswappalyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftristanlatr%2Fmasswappalyzer/lists"}