{"id":44905217,"url":"https://github.com/iscc/iscc-core","last_synced_at":"2026-02-17T22:28:55.108Z","repository":{"id":41331833,"uuid":"397389876","full_name":"iscc/iscc-core","owner":"iscc","description":"ISCC - Codec \u0026 Algorithms","archived":false,"fork":false,"pushed_at":"2025-11-25T08:15:58.000Z","size":7128,"stargazers_count":21,"open_issues_count":6,"forks_count":5,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-11-28T15:26:31.753Z","etag":null,"topics":["hashing","identifier","iscc","media","python","similarity"],"latest_commit_sha":null,"homepage":"https://core.iscc.codes/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iscc.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":"titusz","custom":"https://iscc.foundation/support/"}},"created_at":"2021-08-17T20:59:35.000Z","updated_at":"2025-09-17T09:58:04.000Z","dependencies_parsed_at":"2024-03-17T21:44:30.833Z","dependency_job_id":"bf256ef9-6bdb-4f25-9ff6-656dbfe5eb00","html_url":"https://github.com/iscc/iscc-core","commit_stats":{"total_commits":519,"total_committers":3,"mean_commits":173.0,"dds":"0.0077071290944122905","last_synced_commit":"d616ec61a1800349095f0986f43086bc7ee3c9fd"},"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"purl":"pkg:github/iscc/iscc-core","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iscc%2Fiscc-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iscc%2Fiscc-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iscc%2Fiscc-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iscc%2Fiscc-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iscc","download_url":"https://codeload.github.com/iscc/iscc-core/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iscc%2Fiscc-core/sbom","scorecard":{"id":495318,"data":{"date":"2025-08-11","repo":{"name":"github.com/iscc/iscc-core","commit":"7872485584bd667276ab41cb9f4879e9df54f919"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4.1,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Code-Review","score":0,"reason":"Found 1/29 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql-analysis.yml:28","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql-analysis.yml:29","Warn: no topLevel permission defined: .github/workflows/codeql-analysis.yml:1","Warn: no topLevel permission defined: .github/workflows/tests.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Pinned-Dependencies","score":2,"reason":"dependency not pinned by hash detected -- score normalized to 2","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/codeql-analysis.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:45: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/codeql-analysis.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:56: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/codeql-analysis.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:70: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/codeql-analysis.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/tests.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/tests.yml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/tests.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/tests.yml:36: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yml:62: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/tests.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/tests.yml:71: update your workflow using https://app.stepsecurity.io/secureworkflow/iscc/iscc-core/tests.yml/main?enable=pin","Info:   0 out of   8 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 third-party GitHubAction dependencies pinned","Info:   1 out of   1 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":7,"reason":"3 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-48p4-8xcf-vxj5","Warn: Project is vulnerable to: GHSA-pq67-6m6q-mj2v"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":7,"reason":"SAST tool detected but not run on all commits","details":["Info: SAST configuration detected: CodeQL","Warn: 0 commits out of 2 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-19T20:18:49.755Z","repository_id":41331833,"created_at":"2025-08-19T20:18:49.755Z","updated_at":"2025-08-19T20:18:49.755Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29560559,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-17T21:50:49.831Z","status":"ssl_error","status_checked_at":"2026-02-17T21:46:15.313Z","response_time":100,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hashing","identifier","iscc","media","python","similarity"],"created_at":"2026-02-17T22:28:54.295Z","updated_at":"2026-02-17T22:28:55.093Z","avatar_url":"https://github.com/iscc.png","language":"Python","funding_links":["https://github.com/sponsors/titusz","https://iscc.foundation/support/"],"categories":[],"sub_categories":[],"readme":"# ISCC - Codec \u0026 Algorithms\n\n[![Build](https://github.com/iscc/iscc-core/actions/workflows/tests.yml/badge.svg)](https://github.com/iscc/iscc-core/actions/workflows/tests.yml)\n[![Version](https://img.shields.io/pypi/v/iscc-core.svg)](https://pypi.python.org/pypi/iscc-core/)\n[![Coverage](https://codecov.io/gh/iscc/iscc-core/branch/main/graph/badge.svg?token=7BJ7HJU815)](https://codecov.io/gh/iscc/iscc-core)\n[![Quality](https://app.codacy.com/project/badge/Grade/ad1cc48ac0c0413ea2373a938148f019)](https://www.codacy.com/gh/iscc/iscc-core/dashboard)\n[![Downloads](https://pepy.tech/badge/iscc-core)](https://pepy.tech/project/iscc-core)\n\n**Create similarity-preserving identifiers for digital content**\n\n`iscc-core` is the reference implementation of the core algorithms of\n[ISO 24138](https://www.iso.org/standard/77899.html) – [ISCC](https://iscc.io) (*International\nStandard Content Code*)\n\n## Key Features\n\n- **Similarity-Preserving**: Detect similar content even after modifications\n- **Multi-Level Identification**: Identify content at metadata, perceptual, and data levels\n- **Self-Describing**: Each component contains its own type and version information\n- **ISO Standardized**: Implements the official ISO 24138:2024 specification\n- **Highly Tested**: 100% test coverage with conformance test vectors\n\n## What is the ISCC\n\nThe ISCC is a similarity preserving fingerprint and identifier for digital media assets.\n\nISCCs are generated algorithmically from digital content, just like cryptographic hashes. However,\ninstead of using a single cryptographic hash function to identify data only, the ISCC uses various\nalgorithms to create a composite identifier that exhibits similarity-preserving properties (soft\nhash).\n\nThe component-based structure of the ISCC identifies content at multiple levels of abstraction. Each\ncomponent is self-describing, modular, and can be used separately or with others to aid in various\ncontent identification tasks. The algorithmic design supports content deduplication, database\nsynchronization, indexing, integrity verification, timestamping, versioning, data provenance,\nsimilarity clustering, anomaly detection, usage tracking, allocation of royalties, fact-checking and\ngeneral digital asset management use-cases.\n\n## What is `iscc-core`\n\n`iscc-core` is the python based reference implementation of the ISCC core algorithms as defined by\n[ISO 24138](https://www.iso.org/standard/77899.html). It is also a good reference for porting ISCC\nto other programming languages.\n\n!!! tip\n    This is a low level reference implementation that does not inlcude features like mediatype\n    detection, metadata extraction or file format specific content extraction. Please have a look at\n    [iscc-sdk](https://github.com/iscc/iscc-sdk) which adds those higher level features on top of the\n    `iscc-core` library.\n\n## Implementors Guide\n\n### Reproducible Environment\n\nFor reproducible installation of the reference implementation we included a `poetry.lock` file with\npinned dependencies. Install them using [Python Poetry](https://pypi.org/project/poetry/) with the\ncommand `poetry install` in the root folder.\n\n### Repository structure\n\n```\niscc-core\n├── docs       # Markdown and other assets for mkdocs documentation\n├── examples   # Example scripts using the reference code\n├── iscc_core  # Actual source code of the reference implementation\n├── tests      # Tests for the reference implementation\n└── tools      # Development tools\n```\n\n### Testing \u0026 Conformance\n\nThe reference implementation comes with 100% test coverage. To run the conformance selftest from the\nrepository root use `poetry run python -m iscc_core`. To run the complete test suite use\n`poetry run pytest`.\n\nTo build a conformant implementation work through the follwing top level entrypoint functions:\n\n```\ngen_meta_code_v0\ngen_text_code_v0\ngen_image_code_v0\ngen_audio_code_v0\ngen_video_code_v0\ngen_mixed_code_v0\ngen_data_code_v0\ngen_instance_code_v0\ngen_iscc_code_v0\n```\n\nThe corresponding test vectors can be found in `iscc_core/data.json`.\n\n## ISCC Architecture\n\n![ISCC Architecture](https://raw.githubusercontent.com/iscc/iscc-core/master/docs/images/iscc-codec-light.png)\n\n## ISCC MainTypes\n\n| Idx | Slug     | Bits | Purpose                                                |\n| --- | :------- | ---- | ------------------------------------------------------ |\n| 0   | META     | 0000 | Match on metadata similarity                           |\n| 1   | SEMANTIC | 0001 | Match on semantic content similarity                   |\n| 2   | CONTENT  | 0010 | Match on perceptual content similarity                 |\n| 3   | DATA     | 0011 | Match on data similarity                               |\n| 4   | INSTANCE | 0100 | Match on data identity                                 |\n| 5   | ISCC     | 0101 | Composite of two or more components with common header |\n\n## Installation\n\nUse the package manager [pip](https://pip.pypa.io/en/stable/) to install `iscc-core` as a library.\n\n```bash\npip install iscc-core\n```\n\n## Quick Start\n\n```python\nimport json\nimport iscc_core as ic\n\nmeta_code = ic.gen_meta_code(name=\"ISCC Test Document!\")\n\nprint(f\"Meta-Code:     {meta_code['iscc']}\")\nprint(f\"Structure:     {ic.iscc_explain(meta_code['iscc'])}\\n\")\n\n# Extract text from file\nwith open(\"demo.txt\", \"rt\", encoding=\"utf-8\") as stream:\n    text = stream.read()\n    text_code = ic.gen_text_code_v0(text)\n    print(f\"Text-Code:     {text_code['iscc']}\")\n    print(f\"Structure:     {ic.iscc_explain(text_code['iscc'])}\\n\")\n\n# Process raw bytes of textfile\nwith open(\"demo.txt\", \"rb\") as stream:\n    data_code = ic.gen_data_code(stream)\n    print(f\"Data-Code:     {data_code['iscc']}\")\n    print(f\"Structure:     {ic.iscc_explain(data_code['iscc'])}\\n\")\n\n    stream.seek(0)\n    instance_code = ic.gen_instance_code(stream)\n    print(f\"Instance-Code: {instance_code['iscc']}\")\n    print(f\"Structure:     {ic.iscc_explain(instance_code['iscc'])}\\n\")\n\n# Combine ISCC-UNITs into ISCC-CODE\niscc_code = ic.gen_iscc_code(\n    (meta_code[\"iscc\"], text_code[\"iscc\"], data_code[\"iscc\"], instance_code[\"iscc\"])\n)\n\n# Create convenience `Code` object from ISCC string\niscc_obj = ic.Code(iscc_code[\"iscc\"])\nprint(f\"ISCC-CODE:     {ic.iscc_normalize(iscc_obj.code)}\")\nprint(f\"Structure:     {iscc_obj.explain}\")\nprint(f\"Multiformat:   {iscc_obj.mf_base32}\\n\")\n\n# Compare with changed ISCC-CODE:\nnew_dc, new_ic = ic.Code.rnd(mt=ic.MT.DATA), ic.Code.rnd(mt=ic.MT.INSTANCE)\nnew_iscc = ic.gen_iscc_code((meta_code[\"iscc\"], text_code[\"iscc\"], new_dc.uri, new_ic.uri))\nprint(f\"Compare ISCC-CODES:\\n{iscc_obj.uri}\\n{new_iscc['iscc']}\")\nprint(json.dumps(ic.iscc_compare(iscc_obj.code, new_iscc[\"iscc\"]), indent=2))\n```\n\nThe output of this example is as follows:\n\n```\nMeta-Code:     ISCC:AAAT4EBWK27737D2\nStructure:     META-NONE-V0-64-3e103656bffdfc7a\n\nText-Code:     ISCC:EAAQMBEYQF6457DP\nStructure:     CONTENT-TEXT-V0-64-060498817dcefc6f\n\nData-Code:     ISCC:GAA7UJMLDXHPPENG\nStructure:     DATA-NONE-V0-64-fa258b1dcef791a6\n\nInstance-Code: ISCC:IAA3Y7HR2FEZCU4N\nStructure:     INSTANCE-NONE-V0-64-bc7cf1d14991538d\n\nISCC-CODE:     ISCC:KACT4EBWK27737D2AYCJRAL5Z36G76RFRMO4554RU26HZ4ORJGIVHDI\nStructure:     ISCC-TEXT-V0-MCDI-3e103656bffdfc7a060498817dcefc6ffa258b1dcef791a6bc7cf1d14991538d\nMultiformat:   bzqavabj6ca3fnp757r5ambeyqf6457dp7isywhoo66i2npd46hiutektru\n\nCompare ISCC-CODES:\nISCC:KACT4EBWK27737D2AYCJRAL5Z36G76RFRMO4554RU26HZ4ORJGIVHDI\nISCC:KACT4EBWK27737D2AYCJRAL5Z36G7Y7HA2BMECKMVRBEQXR2BJOS6NA\n{\n  \"meta_dist\": 0,\n  \"content_dist\": 0,\n  \"data_dist\": 33,\n  \"instance_match\": false\n}\n```\n\n## Documentation\n\nDocumentation is published at \u003chttps://core.iscc.codes\u003e\n\n## Development\n\n**Requirements**\n\n- [Python 3.9](https://www.python.org/) or higher for code generation and static site building.\n- [Poetry](https://python-poetry.org/) for installation and dependency management.\n\n**Development Setup**\n\n```shell\ngit clone https://github.com/iscc/iscc-core.git\ncd iscc-core\npoetry install\n```\n\n**Development Tasks**\n\nTests, coverage, code formatting and other tasks can be run with the `poe` command:\n\n```shell\npoe\n\nPoe the Poet - A task runner that works well with poetry.\nversion 0.18.1\n\nResult: No task specified.\n\nUSAGE\n  poe [-h] [-v | -q] [--root PATH] [--ansi | --no-ansi] task [task arguments]\n\nGLOBAL OPTIONS\n  -h, --help     Show this help page and exit\n  --version      Print the version and exit\n  -v, --verbose  Increase command output (repeatable)\n  -q, --quiet    Decrease command output (repeatable)\n  -d, --dry-run  Print the task contents but don't actually run it\n  --root PATH    Specify where to find the pyproject.toml\n  --ansi         Force enable ANSI output\n  --no-ansi      Force disable ANSI output\nCONFIGURED TASKS\n  gentests       Generate conformance test data\n  format         Code style formating with black\n  docs           Copy README.md to /docs\n  format-md      Markdown formating with mdformat\n  lf             Convert line endings to lf\n  test           Run tests with coverage\n  sec            Security check with bandit\n  all\n```\n\nUse `poe all` to run all tasks before committing any changes.\n\n## Maintainers\n\n[@titusz](https://github.com/titusz)\n\n## Contributing\n\nPull requests are welcome. For significant changes, please open an issue first to discuss your\nplans. Please make sure to update tests as appropriate.\n\nYou may also want join our developer chat on Telegram at \u003chttps://t.me/iscc_dev\u003e.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiscc%2Fiscc-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiscc%2Fiscc-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiscc%2Fiscc-core/lists"}