{"id":13936372,"url":"https://github.com/matiskay/html-similarity","last_synced_at":"2025-07-19T21:32:31.347Z","repository":{"id":48440704,"uuid":"108414116","full_name":"matiskay/html-similarity","owner":"matiskay","description":"Compare html similarity using structural and style metrics","archived":false,"fork":false,"pushed_at":"2023-05-11T13:04:52.000Z","size":66,"stargazers_count":212,"open_issues_count":6,"forks_count":23,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-06-20T20:50:07.744Z","etag":null,"topics":["html","jaccard-similarity","python36","similarity"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matiskay.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-10-26T13:19:43.000Z","updated_at":"2025-05-23T13:19:18.000Z","dependencies_parsed_at":"2024-06-18T23:59:27.595Z","dependency_job_id":"3c2d85e3-e794-4e38-a354-044846f71628","html_url":"https://github.com/matiskay/html-similarity","commit_stats":{"total_commits":69,"total_committers":5,"mean_commits":13.8,"dds":"0.42028985507246375","last_synced_commit":"d1aff81d8873586986baf3fa9fe01b6060523b30"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/matiskay/html-similarity","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matiskay%2Fhtml-similarity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matiskay%2Fhtml-similarity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matiskay%2Fhtml-similarity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matiskay%2Fhtml-similarity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matiskay","download_url":"https://codeload.github.com/matiskay/html-similarity/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matiskay%2Fhtml-similarity/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266019657,"owners_count":23864916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["html","jaccard-similarity","python36","similarity"],"created_at":"2024-08-07T23:02:36.560Z","updated_at":"2025-07-19T21:32:26.330Z","avatar_url":"https://github.com/matiskay.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"===============\nHTML Similarity\n===============\n\n.. image:: https://travis-ci.org/matiskay/html-similarity.svg?branch=master\n    :target: https://travis-ci.org/matiskay/html-similarity\n\n.. image:: https://codebeat.co/badges/304915eb-48a3-46a8-9ce9-2790c82dc2b8\n    :target: https://codebeat.co/projects/github-com-matiskay-html-similarity-master\n\nThis package provides a set of functions to measure the similarity between web pages.\n\nInstall\n=======\n\nThe quick way::\n\n    pip install html-similarity\n\nHow it works?\n=============\n\nStructural Similarity\n---------------------\n\nUses sequence comparison of the html tags to compute the similarity.\n\nWe not implement the similarity based on tree edit distance because it is slower than sequence comparison.\n\n\nStyle Similarity\n----------------\n\nExtracts css classes of each html document and calculates the jaccard similarity of the sets of classes.\n\n\nJoint Similarity (Structural Similarity and Style Similarity)\n-------------------------------------------------------------\n\nThe joint similarity metric is calculated as::\n\n    k * structural_similarity(document_1, document_2) + (1 - k) * style_similarity(document_1, document_2)\n\n\nAll the similarity metrics takes values between 0 and 1.\n\nRecommendations for joint similarity\n------------------------------------\n\nUsing `k=0.3` give use better results. The style similarity gives more information about the similarity rather than the structural similarity.\n\nExamples\n========\n\nHere is a example::\n\n    In [1]: html_1 = '''\n    \u003ch1 class=\"title\"\u003eFirst Document\u003c/h1\u003e\n    \u003cul class=\"menu\"\u003e\n        \u003cli class=\"active\"\u003eDocuments\u003c/li\u003e\n        \u003cli\u003eExtra\u003c/li\u003e\n    \u003c/ul\u003e\n    '''\n\n    In [2]: html_2 = '''\n    \u003ch1 class=\"title\"\u003eSecond document Document\u003c/h1\u003e\n    \u003cul class=\"menu\"\u003e\n        \u003cli class=\"active\"\u003eExtra Documents\u003c/li\u003e\n    \u003c/ul\u003e\n    '''\n\n    In [3] from html_similarity import style_similarity, structural_similarity, similarity\n\n    In [4]: style_similarity(html_1, html_2)\n    Out[4]: 1.0\n\n    In [7]: structural_similarity(html_1, html_2)\n    Out[7]: 0.9090909090909091\n\n    In [8]: similarity(html_1, html_2)\n    Out[8]: 0.9545454545454546\n\nReferences\n==========\n\n- The idea of sequence comparision was taken from `Page Compare \u003chttps://github.com/TeamHG-Memex/page-compare\u003e`_.\n- The other ideas were taken from `T. Gowda and C. A. Mattmann, Clustering Web Pages Based on Structure and Style Similarity, 2016 IEEE 17th International Conference on Information Reuse and Integration (IRI), Pittsburgh, PA, 2016, pp. 175-180. \u003chttp://ieeexplore.ieee.org/document/7785739/\u003e`_\n- Use case `Clustering web pages based on structure and style similarity \u003chttps://www.slideshare.net/thammegowda/ieee-iri-16-clustering-web-pages-based-on-structure-and-style-similarity?qid=7deea5f8-157d-4e57-a413-16ec7c6a22d9\u0026v=\u0026b=\u0026from_search=1\u003e`_\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatiskay%2Fhtml-similarity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatiskay%2Fhtml-similarity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatiskay%2Fhtml-similarity/lists"}