{"id":26244223,"url":"https://github.com/scrapfly/fingerprint-generator","last_synced_at":"2025-04-08T04:16:32.720Z","repository":{"id":280912982,"uuid":"934537189","full_name":"scrapfly/fingerprint-generator","owner":"scrapfly","description":"Browser fingerprint data generator","archived":false,"fork":false,"pushed_at":"2025-04-02T07:40:06.000Z","size":121,"stargazers_count":46,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-06T06:53:52.686Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://pypi.org/project/fpgen","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scrapfly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-18T02:06:19.000Z","updated_at":"2025-04-06T06:04:14.000Z","dependencies_parsed_at":"2025-03-06T01:31:54.395Z","dependency_job_id":"9ee13a43-cf3d-4e40-9392-39c6fc463968","html_url":"https://github.com/scrapfly/fingerprint-generator","commit_stats":null,"previous_names":["scrapfly/fingerprint-generator"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ffingerprint-generator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ffingerprint-generator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ffingerprint-generator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scrapfly%2Ffingerprint-generator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scrapfly","download_url":"https://codeload.github.com/scrapfly/fingerprint-generator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247445651,"owners_count":20939956,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-13T11:18:50.348Z","updated_at":"2025-04-08T04:16:32.705Z","avatar_url":"https://github.com/scrapfly.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eFingerprint Generator\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003eA fast browser data generator that mimics actual traffic patterns in the wild. With \u003ci\u003eextensive\u003c/i\u003e data coverage.\u003c/p\u003e\n\n\u003cp align=\"center\"\u003eCreated by \u003ca href=\"https://github.com/daijro\"\u003edaijro\u003c/a\u003e. Data provided by Scrapfly.\u003c/p\u003e\n\n---\n\n## Features\n\n- Uses a Bayesian generative network to mimic real-world web traffic patterns\n- Extensive data coverage for **nearly all known** browser data points\n- Creates complete fingerprints in a few milliseconds ⚡\n- Easily specify custom criteria for any data point (e.g. \"only Windows + Chrome, with Intel GPUs\")\n- Simple for humans to use 🚀\n\n## Demo Video\n\nHere is a demonstration of what fpgen generates \u0026 its ability to filter data points:\n\nhttps://github.com/user-attachments/assets/5c56691a-5804-4007-b179-0bae7069a111\n\n---\n\n# Installation\n\nInstall the package using pip:\n\n```bash\npip install fpgen\n```\n\n\u003chr width=50\u003e\n\n### Downloading the model\n\nFetch the latest model:\n\n```bash\nfpgen fetch\n```\n\nThis will be ran automatically on the first import, or every 5 weeks.\n\nTo decompress the model for faster generation (_up to 10-50x faster!_), run:\n\n```bash\nfpgen decompress\n```\n\nNote: This action will use an additional 100mb+ of storage.\n\n\u003cdetails\u003e\n\u003csummary\u003eCLI Usage\u003c/summary\u003e\n\n```\nUsage: python -m fpgen [OPTIONS] COMMAND [ARGS]...\n\nOptions:\n  --help  Show this message and exit.\n\nCommands:\n  decompress  Decompress model files for speed efficiency (will take 100mb+)\n  fetch       Fetch the latest model from GitHub\n  recompress  Compress model files after running decompress\n  remove      Remove all downloaded and/or extracted model files\n```\n\n\u003c/details\u003e\n\n---\n\n# Usage\n\n### Generate a fingerprint\n\nSimple usage:\n\n```python\n\u003e\u003e\u003e import fpgen\n\u003e\u003e\u003e fpgen.generate(browser='Chrome', os='Windows')\n```\n\nOr use the Generator object to pass filters downward:\n\n```python\n\u003e\u003e\u003e gen = fpgen.Generator(browser='Chrome')  # Filter by Chrome\n\u003e\u003e\u003e gen.generate(os='Windows')  # Generate Windows \u0026 Chrome fingerprints\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\nParameters list\n\u003c/summary\u003e\n\n```\nInitializes the Generator with the given options.\nValues passed to the Generator object will be inherited when calling Generator.generate()\n\nParameters:\n    conditions (dict, optional): Conditions for the generated fingerprint.\n    window_bounds (WindowBounds, optional): Constrain the output window size.\n    strict (bool, optional): Whether to raise an exception if the conditions are too strict.\n    flatten (bool, optional): Whether to flatten the output dictionary\n    target (Optional[Union[str, StrContainer]]): Only generate specific value(s)\n    **conditions_kwargs: Conditions for the generated fingerprint (passed as kwargs)\n```\n\n\u003c/details\u003e\n\n[See example output.](https://raw.githubusercontent.com/scrapfly/fingerprint-generator/refs/heads/main/assets/example-output.json)\n\n---\n\n## Filtering the output\n\n### Setting fingerprint criteria\n\nYou can narrow down generated fingerprints by specifying filters for **any** data field.\n\n```python\n# Only generate fingerprints with Windows, Chrome, and Intel GPU:\n\u003e\u003e\u003e fpgen.generate(\n...     os='Windows',\n...     browser='Chrome',\n...     gpu={'vendor': 'Google Inc. (Intel)'}\n... )\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\nThis can also be passed as a dictionary.\n\u003c/summary\u003e\n\n```python\n\u003e\u003e\u003e fpgen.generate({\n...     'os': 'Windows',\n...     'browser': 'Chrome',\n...     'gpu': {'vendor': 'Google Inc. (Intel)'},\n... })\n```\n\n\u003c/details\u003e\n\n### Multiple constraints\n\nPass in multiple constraints for the generator to select from using a tuple.\n\n```python\n\u003e\u003e\u003e fpgen.generate({\n...     'os': ('Windows', 'MacOS'),\n...     'browser': ('Firefox', 'Chrome'),\n... })\n```\n\nIf you are passing many nested constraints, run `fpgen decompress` to improve model performance.\n\n## Custom filters\n\nData can be filtered by passing in callable functions.\n\n### Examples\n\nSet the minimum browser version:\n\n```python\n# Constrain client:\n\u003e\u003e\u003e fpgen.generate(client={'browser': {'major': lambda ver: int(ver) \u003e= 130}})\n# Or, just pass a dot seperated path to client.browser.major:\n\u003e\u003e\u003e fpgen.generate({'client.browser.major': lambda ver: int(ver) \u003e= 130})\n```\n\nOnly allow NVIDIA GPUs:\n\n```python\n# Note: Strings are lowercased before they're passed.\n\u003e\u003e\u003e fpgen.generate(gpu={'vendor': lambda vdr: 'nvidia' in vdr})\n```\n\nLimit the maximum/minimum window size:\n\n```python\n# Set allowed ranges for outerWidth \u0026 outerHeight:\n\u003e\u003e\u003e fpgen.generate(\n...   window={\n...     'outerWidth': lambda width: 1000 \u003c= width \u003c= 2000,\n...     'outerHeight': lambda height: 500 \u003c= height \u003c= 1500\n...   }\n... )\n```\n\n\n\u003cdetails\u003e\n\u003csummary\u003e\nOr, filter the window dictionary directly.\n\u003c/summary\u003e\n\n```python\ndef window_filter(window):\n    if not (1000 \u003c= window['outerWidth'] \u003c= 2000):\n        return False\n    if not (500 \u003c= window['outerHeight'] \u003c= 1500):\n        return False\n    return True\n\nfpgen.generate(window=window_filter)\n```\n\u003c/details\u003e\n\u003c/details\u003e\n\n---\n\n## Only generate specific data\n\nTo generate specific data fields, use the `target` parameter with a string or a list of strings.\n\n### Examples\n\nOnly generate HTTP headers:\n\n```python\n\u003e\u003e\u003e fpgen.generate(target='headers')\n{'accept': '*/*', 'accept-encoding': 'gzip, deflate, br, zstd', 'accept-language': 'en-US,en;q=0.9', 'priority': 'u=1, i', 'sec-ch-ua': '\"Google Chrome\";v=\"131\", \"Chromium\";v=\"131\", \"Not_A Brand\";v=\"24\"', 'sec-ch-ua-mobile': None, 'sec-ch-ua-platform': '\"Windows\"', 'sec-fetch-dest': 'empty', 'sec-fetch-mode': 'cors', 'sec-fetch-site': 'same-site', 'sec-gpc': None, 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36'}\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\nOr, by using the generate_target shortcut.\n\u003c/summary\u003e\n\n```python\n\u003e\u003e\u003e fpgen.generate_target('headers')\n{'accept': '*/*', 'accept-encoding': 'gzip, deflate, br, zstd', 'accept-language': 'en-GB,en;q=0.9,en-US;q=0.8,sk;q=0.7', 'priority': 'u=1, i', 'sec-ch-ua': '\"Google Chrome\";v=\"131\", \"Chromium\";v=\"131\", \"Not_A Brand\";v=\"24\"', 'sec-ch-ua-mobile': None, 'sec-ch-ua-platform': '\"Windows\"', 'sec-fetch-dest': 'empty', 'sec-fetch-mode': 'cors', 'sec-fetch-site': 'same-site', 'sec-gpc': None, 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36'}\n```\n\n\u003c/details\u003e\n\n\nGenerate a User-Agent for Windows \u0026 Chrome:\n\n```python\n\u003e\u003e\u003e fpgen.generate(\n...     os='Windows',\n...     browser='Chrome',\n...     # Nested targets must be seperated by dots:\n...     target='headers.user-agent'\n... )\n'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:134.0) Gecko/20100101 Firefox/134.0'\n```\n\nGenerate a Firefox TLS fingerprint:\n\n```python\n\u003e\u003e\u003e fpgen.generate(\n...     browser='Firefox',\n...     target='network.tls.scrapfly_fp'\n... )\n{'version': '772', 'ch_ciphers': '4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-156-157-47-53', 'ch_extensions': '0-5-10-11-13-16-23-27-28-34-35-43-45-51-65037-65281', 'groups': '4588-29-23-24-25-256-257', 'points': '0', 'compression': '0', 'supported_versions': '772-771', 'supported_protocols': 'h2-http11', 'key_shares': '4588-29-23', 'psk': '1', 'signature_algs': '1027-1283-1539-2052-2053-2054-1025-1281-1537-515-513', 'early_data': '0'}\n```\n\nYou can provide multiple targets as a list.\n\n---\n\n## Get the probabilities of a target\n\nCalculate the probability distribution of a target given any filter:\n\n```python\n\u003e\u003e\u003e fpgen.trace(target='browser', os='Windows')\n[\u003cChrome: 71.29276%\u003e, \u003cEdge: 12.96372%\u003e, \u003cFirefox: 12.64484%\u003e, \u003cOpera: 2.12217%\u003e, \u003cYandex Browser: 0.94575%\u003e, \u003cWhale: 0.03076%\u003e]\n```\n\nMultiple targets can be passed as a list/tuple.\nHere is an example of tracking the probability of browser \u0026 OS given a GPU vendor:\n\n```python\n\u003e\u003e\u003e fpgen.trace(\n...   target=('browser', 'os'),\n...   gpu={'vendor': 'Google Inc. (Intel)'}\n... )\n{'browser': [\u003cChrome: 76.46641%\u003e, \u003cEdge: 13.02665%\u003e, \u003cFirefox: 8.48189%\u003e, \u003cOpera: 1.36188%\u003e, \u003cYandex Browser: 0.65133%\u003e, \u003cWhale: 0.01184%\u003e],\n 'os': [\u003cWindows: 84.08380%\u003e, \u003cLinux: 8.07652%\u003e, \u003cMacOS: 7.46072%\u003e, \u003cChromeOS: 0.37896%\u003e]}\n```\n\nThis also works in the Generator object:\n\n```python\n\u003e\u003e\u003e gen = fpgen.Generator(os='ChromeOS')\n\u003e\u003e\u003e gen.trace(target='browser')\n[\u003cChrome: 100.00000%\u003e]\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\nParameters for trace\n\u003c/summary\u003e\n\n```\nCompute the probability distribution(s) of a target variable given conditions.\n\nParameters:\n    target (str): The target variable name.\n    conditions (Dict[str, Any], optional): A dictionary mapping variable names\n    flatten (bool, optional): If True, return a flattened dictionary.\n    **conditions_kwargs: Additional conditions to apply\n\nReturns:\n    A dictionary mapping probabilities to the target's possible values.\n```\n\n\u003c/details\u003e\n\n\u003chr width=50\u003e\n\n### Reading TraceResult\n\nTo read the output `TraceResult` object:\n\n```python\n\u003e\u003e\u003e chrome = fpgen.trace(target='browser', os='ChromeOS')[0]\n\u003e\u003e\u003e chrome.probability\n1.0\n\u003e\u003e\u003e chrome.value\n'Chrome'\n```\n\n---\n\n## Query possible values\n\nYou can get a list of a target's possible values by passing it into `fpgen.query`:\n\nList all possible browsers:\n\n```python\n\u003e\u003e\u003e fpgen.query('browser')\n['Chrome', 'Edge', 'Firefox', 'Opera', 'Safari', 'Samsung Internet', 'Yandex Browser']\n```\n\nPassing a nested target:\n\n```python\n\u003e\u003e\u003e fpgen.query('navigator.maxTouchPoints') # Dot seperated path\n[0, 1, 2, 5, 6, 9, 10, 17, 20, 40, 256]\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e\nParameters for query\n\u003c/summary\u003e\n\n```\nQuery a list of possibilities given a target.\n\nParameters:\n    target (str): Target node to query possible values for\n    flatten (bool, optional): Whether to flatten the output dictionary\n    sort (bool, optional): Whether to sort the output arrays\n```\n\n\u003c/details\u003e\n\n\u003e [!NOTE]\n\u003e Since fpgen is trained on live data, queries may occasionally return invalid or anomalous values. Values lower than a 0.001% probability will not appear in traces or generated fingerprints.\n\n---\n\n## Generated data\n\nHere is a rough list of the data fpgen can generate:\n\n- **Browser data:**\n  - All navigator data\n  - All mimetype data: Audio, video, media source, play types, PDF, etc\n  - All window viewport data (position, inner/outer viewport sizes, toolbar \u0026 scrollbar sizes, etc)\n  - All screen data\n  - Supported \u0026 unsupported DRM modules\n  - Memory heap limit\n\n* **System data:**\n  - GPU data (vendor, renderer, WebGL/WebGL2, extensions, context attributes, parameters, shader precision formats, etc)\n  - Battery data (charging, charging time, discharging time, level)\n  - Screen size, color depth, taskbar size, etc.\n  - Full fonts list\n  - Cast receiver data\n\n- **Network data:**\n  - HTTP headers\n  - TLS fingerprint data\n  - HTTP/2 fingerprint \u0026 frames\n  - RTC video \u0026 audio capabilities, codecs, clock rates, mimetypes, header extensions, etc\n\n* **Audio data:**\n  - Audio signal\n  - All Audio API constants (AnalyserNode, BiquadFilterNode, DynamicsCompressorNode, OscillatorNode, etc)\n\n- **Internationalization data:**\n  - Regional internationalization (Locale, calendar, numbering system, timezone, date format, etc)\n  - Voices\n\n* **_And much more!_**\n\nFor a more complete list, see the [full example output](https://raw.githubusercontent.com/scrapfly/fingerprint-generator/refs/heads/main/assets/example-output.json).\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapfly%2Ffingerprint-generator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscrapfly%2Ffingerprint-generator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscrapfly%2Ffingerprint-generator/lists"}