{"id":13464189,"url":"https://github.com/jamesgpearce/confess","last_synced_at":"2025-03-25T10:32:05.088Z","repository":{"id":51362219,"uuid":"1611617","full_name":"jamesgpearce/confess","owner":"jamesgpearce","description":"[ARCHIVED] Uses PhantomJS to headlessly analyze web pages and generate manifests. You may be able to use the puppeteer-har package instead.","archived":true,"fork":false,"pushed_at":"2016-08-03T19:10:24.000Z","size":356,"stargazers_count":874,"open_issues_count":13,"forks_count":101,"subscribers_count":48,"default_branch":"master","last_synced_at":"2024-10-29T16:23:14.423Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jamesgpearce.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2011-04-13T21:56:10.000Z","updated_at":"2024-08-26T06:47:40.000Z","dependencies_parsed_at":"2022-08-28T14:00:11.703Z","dependency_job_id":null,"html_url":"https://github.com/jamesgpearce/confess","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamesgpearce%2Fconfess","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamesgpearce%2Fconfess/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamesgpearce%2Fconfess/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jamesgpearce%2Fconfess/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jamesgpearce","download_url":"https://codeload.github.com/jamesgpearce/confess/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245444187,"owners_count":20616337,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T14:00:36.475Z","updated_at":"2025-03-25T10:32:04.766Z","avatar_url":"https://github.com/jamesgpearce.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","Analyzers"],"sub_categories":["Meetups"],"readme":"# confess.js\n\nA small script library that uses [PhantomJS 1.2](http://www.phantomjs.org/) (or later) to headlessly analyze web pages. Currently it can perform the following tasks:\n\n * Generation of an *appcache* manifest for a web app\n * Simple *performance* analysis of a web page and its resources\n * List of *CSS properties* used on the page\n\n## Examples\n\nconfess.js can enumerate a web app's resources for the purposes of creating a cache manifest file to make your apps run offline.\n\nFor example...\n\n    \u003e phantomjs confess.js http://functionsource.com appcache\n\n...will write, to stdout:\n\n    CACHE MANIFEST\n\n    # This manifest was created by confess.js, http://github.com/jamesgpearce/confess\n    #\n    # Time: Fri Dec 23 2011 13:12:32 GMT-0800 (PST)\n    # Retrieved URL: http://functionsource.com/\n    # User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X) AppleWebKit/...\n\n    CACHE:\n    /images/icons/netflix.png\n    /javascripts/lib/legacy.js\n    /stylesheets/light.css\n    /stylesheets/screen.css\n    /stylesheets/syntax.css\n    http://functionscopedev.files.wordpress.com/2011/12/dabblet.png\n    http://functionsource.com/images/avatars/ben.png\n    http://functionsource.com/images/avatars/dion.png\n    http://functionsource.com/images/avatars/kevin.png\n    http://functionsource.com/images/avatars/ryan.png\n    http://functionsource.com/images/header.png\n    http://functionsource.com/images/header_wht.png\n    http://functionsource.com/images/icons/facebook.png\n    http://functionsource.com/images/icons/firehose.png\n    http://functionsource.com/images/icons/linkedin.png\n    http://functionsource.com/images/icons/podcast.png\n    http://functionsource.com/images/icons/rss.png\n    http://functionsource.com/images/icons/twitter.png\n    http://use.typekit.com/k/tqz3zpc-b.css?3bb2a6e53c9684f...\n    http://use.typekit.com/tqz3zpc.js\n    http://www.google-analytics.com/ga.js\n\n    NETWORK:\n    *\n\nThe other thing it can do is generate a simple performance analysis of a site:\n\n    \u003e phantomjs confess.js http://functionsource.com performance\n\n...which will write, to stdout, something like:\n\n    Elapsed load time:   6370ms\n       # of resources:       21\n\n     Fastest resource:    147ms; http://functionsource.com/...egacy.js\n     Slowest resource:   2766ms; http://use.typekit.com/...z3zpc-b.css\n      Total resources:  17898ms\n\n    Smallest resource:      35b; http://www.google-analytics.com/__...\n     Largest resource:   40183b; http://functionsource.com/...ader.png\n      Total resources:  107594b; (at least)\n\n    Elapsed load time:   2927ms\n       # of resources:       21\n\n     Fastest resource:     59ms; http://functionsource.com/...egacy.js\n     Slowest resource:   1306ms; http://use.typekit.com/...78ed8d47e4b\n      Total resources:   7573ms\n\n    Smallest resource:      35b; http://www.google-analytics.com/...q~\n     Largest resource:   32614b; http://www.google-analytics.com/ga.js\n      Total resources:  149860b\n\n...and which can also generate an ASCII-art waterfall of the waits and receipts of each requested resource:\n\n     1|---------------------=                                           |\n     2|                     ----------                                  |\n     3|                     ---=                                        |\n     4|                     --------=                                   |\n     5|                     ---------=                                  |\n     6|                     -----------                                 |\n     7|                     -------                                     |\n     8|                              -----------=========               |\n     9|                              ---======                          |\n    10|                               ----==                            |\n      ...\n\n     1:    577ms;   17669b; http://functionsource.com/\n     2:    255ms;   24990b; http://use.typekit.com/tqz3zpc.js\n     3:     59ms;    1733b; http://functionsource.com/...ts/lib/legacy.js\n     4:    216ms;    5291b; http://functionsource.com/...sheets/light.css\n     5:    240ms;    4150b; http://functionsource.com/...heets/syntax.css\n     6:    283ms;   12049b; http://functionsource.com/...heets/screen.css\n     7:    176ms;   32614b; http://www.google-analytics.com/ga.js\n     8:    537ms;    1448b; http://functionscopedev.files.wordpress.com..\n     9:    228ms;    5792b; http://functionsource.com/...cons/netflix.png\n    10:    134ms;    1153b; http://functionsource.com/...avatars/dion.png\n      ...\n\nAs well as using the final command line argument to perform different tasks, you can use the local \u003ccode\u003econfig.json\u003c/code\u003e file to affect the behavior of the way in which confess.js runs.\n\nFor example, you can set the user-agent header of the request made by PhantomJS to request the page, in case you're serving mobile apps off similar entry-point URLs to your desktop content.\n\nSimilarly, if you're creating an appcache manifest, you can use filters to indicate which files should be included or excluded from the generated \u003ccode\u003eCACHE\u003c/code\u003e list.\n\n## Installation \u0026 usage\n\nThe one-and-only dependency is the one-and-only [PhantomJS](http://www.phantomjs.org/), version 1.2 or later. Install this, and ensure it's all good by trying out some of its example scripts.\n\nThen, assuming \u003ccode\u003ephantomjs\u003c/code\u003e is on your path, and from the directory containing \u003ccode\u003econfess.js\u003c/code\u003e and \u003ccode\u003econfig.json\u003c/code\u003e, run the tasks with:\n\n    \u003e phantomjs confess.js URL TASK [CONFIG]\n\nWhere the \u003ccode\u003eURL\u003c/code\u003e is mandatory, and points to the page or app you're analyzing. \u003ccode\u003eTASK\u003c/code\u003e is what you want the tool to do, and \u003ccode\u003eCONFIG\u003c/code\u003e is the location of an alternative configuration file, if you don't want to use the default \u003ccode\u003econfig.json\u003c/code\u003e.\n\n## The appcache task\n\nUsing the \u003ccode\u003eappcache\u003c/code\u003e task argument will get confess.js to load the page, and then search the DOM and the CSSOM (and then the results of applying the latter to the former) for references to any external resources that the app needs. It can optionally also look for resource request events.\n\nThe results go to stdout, but of course you can pipe it to a file. If you want to create a cache manifest for an app, this might be called something like \u003ccode\u003emy.appcache\u003c/code\u003e:\n\n    \u003e phantomjs confess.js http://functionsource.com appcache \u003e my.appcache\n\nYou would then need to attach add this file to your app, and reference the manifest in the \u003ccode\u003ehtml\u003c/code\u003e element:\n\n    \u003c!DOCTYPE html\u003e\n    \u003chtml manifest=\"my.appcache\"\u003e\n        \u003chead\u003e\n\n(And ensure that the manifest file gets emitted from your web server with a content type of \u003ccode\u003etext/cache-manifest\u003c/code\u003e.)\n\nTo check the resulting manifest's syntax, you might like to use Frederic Hemberger's great [cache manifest validator](http://manifest-validator.com/).\n\n## The performance task\n\nUsing the \u003ccode\u003eperformance\u003c/code\u003e task argument will get confess.js to load the page, and then log the sizes and timings of its constituent parts.\n\nIt will list the fastest and slowest resources, and also the largest and smallest (subject to the availability of the \u003ccode\u003econtent-length\u003c/code\u003e header). With the verbose configuration enabled, it will also list out all the resources loaded as part of the page, and display an ASCII-art waterfall chart of their timings.\n\n## The cssproperties task\n\nUsing the \u003ccode\u003ecssproperties\u003c/code\u003e task argument will get confess.js to load the page, and then parse the styles in the DOM and CSSOM to identify which CSS properties are being used by the page.\n\n\n## Configuration\n\nThe following is the default \u003ccode\u003econfig.json\u003c/code\u003e file for confess.js, but you can of course alter any of the values in this file, or a new config file of your own.\n\n    {\n        \"task\": \"appcache\",\n        \"userAgent\": \"default\",\n        \"userAgentAliases\": {\n            \"iphone\": \"Mozilla/5.0 (iPhone; ...\",\n            \"android\": \"Mozilla/5.0 (Linux; U; Android ...\",\n            \"chrome\": \"Mozilla/5.0 (Macintosh; Intel Mac OS X ...\"\n        },\n        \"wait\": 0,\n        \"consolePrefix\": \"#\",\n        \"verbose\": true,\n        \"appcache\": {\n            \"urlsFromDocument\": true,\n            \"urlsFromRequests\": false,\n            \"cacheFilter\": \".*\",\n            \"networkFilter\": null\n        }\n    }\n\nThese properties are used as follows:\n\n * \u003ccode\u003etask\u003c/code\u003e - the default type of task you want confess.js to perform, if not specified on the command line. \u003ccode\u003eappcache\u003c/code\u003e, \u003ccode\u003eperformance\u003c/code\u003e, and \u003ccode\u003ecssproperties\u003c/code\u003e are the supported values\n\n * \u003ccode\u003euserAgent\u003c/code\u003e - the user-agent to make the request as, or \u003ccode\u003edefault\u003c/code\u003e to use PhantomJS's usual user-agent string\n\n * \u003ccode\u003euserAgentAliases\u003c/code\u003e - common aliases for the userAgent setting. For example, you could use the setting \u003ccode\u003e\"userAgent\": \"iphone\"\u003c/code\u003e and it will resolve to the iPhone's full UA provided here.\n\n * \u003ccode\u003ewait\u003c/code\u003e - the number of milliseconds to wait after the document has loaded before parsing for resources. This might be useful if you know that a deferred script might be making relevant additions to the DOM.\n\n * \u003ccode\u003econsolePrefix\u003c/code\u003e - if set, confess.js will output the *browser's* console to the standard output. Useful for detecting if there are also any issues with the app's execution itself.\n\n * \u003ccode\u003everbose\u003c/code\u003e - whether to add extra output about the generator's configuration into the output. Set to true for the \u003ccode\u003eperformance\u003c/code\u003e task in order to see the waterfall chart.\n\n * \u003ccode\u003eappcache.urlsFromDocument\u003c/code\u003e - whether to traverse the DOM and CSSOM looking for resources; the default behaviour\n\n * \u003ccode\u003eappcache.urlsFromRequests\u003c/code\u003e - whether to also (or alternatively) listen for resource loading events to detect required dependencies\n\n * \u003ccode\u003eappcache.cacheFilter\u003c/code\u003e - a regex to indicate which files to include in the \u003ccode\u003eCACHE\u003c/code\u003e block of the manifest. If set to \u003ccode\u003enull\u003c/code\u003e, none will. As a better example, \u003ccode\u003e\\\\\\\\.png$\u003c/code\u003e will indicate that only PNG files should be cached. (Note the double escaping: once for the regex, and once for the JSON.)\n\n * \u003ccode\u003eappcache.networkFilter\u003c/code\u003e - a regex to indicate which files *not* to include in the \u003ccode\u003eCACHE\u003c/code\u003e block of the manifest, and which a browser will request from the network. If set to \u003ccode\u003enull\u003c/code\u003e, none will. Note that matching files will *not* be explicitly listed in the \u003ccode\u003eNETWORK\u003c/code\u003e block of the manifest, since there is always a catch-all \u003ccode\u003e*\u003c/code\u003e wildcard added.\n\nNote that there are currently no specific configuration options for the \u003ccode\u003eperformance\u003c/code\u003e task.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjamesgpearce%2Fconfess","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjamesgpearce%2Fconfess","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjamesgpearce%2Fconfess/lists"}