{"id":22669058,"url":"https://github.com/wlynxg/chardet","last_synced_at":"2025-04-19T17:22:08.907Z","repository":{"id":266465941,"uuid":"897228760","full_name":"wlynxg/chardet","owner":"wlynxg","description":"Go character encoding detector","archived":false,"fork":false,"pushed_at":"2024-12-16T07:40:20.000Z","size":2101,"stargazers_count":18,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-04T09:33:27.384Z","etag":null,"topics":["chardet","encoding","go","golang","language"],"latest_commit_sha":null,"homepage":"https://pkg.go.dev/github.com/wlynxg/chardet","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wlynxg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-02T09:12:43.000Z","updated_at":"2025-01-14T09:08:50.000Z","dependencies_parsed_at":"2024-12-04T12:31:44.843Z","dependency_job_id":"9cd7bd94-1ff4-4a50-9cb2-0359e3f0a722","html_url":"https://github.com/wlynxg/chardet","commit_stats":null,"previous_names":["wlynxg/chardet"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wlynxg%2Fchardet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wlynxg%2Fchardet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wlynxg%2Fchardet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wlynxg%2Fchardet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wlynxg","download_url":"https://codeload.github.com/wlynxg/chardet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246174469,"owners_count":20735410,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chardet","encoding","go","golang","language"],"created_at":"2024-12-09T15:18:15.639Z","updated_at":"2025-03-29T10:44:17.065Z","avatar_url":"https://github.com/wlynxg.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=center\u003e\n\n## chardet: Go character encoding detector\n[![Go Reference](https://pkg.go.dev/badge/github.com/wlynxg/chardet.svg)](https://pkg.go.dev/github.com/wlynxg/chardet)\n[![License](https://img.shields.io/github/license/wlynxg/chardet.svg?style=flat)](https://github.com/wlynxg/chardet)\n[![Go Report Card](https://goreportcard.com/badge/github.com/wlynxg/chardet)](https://goreportcard.com/report/github.com/wlynxg/chardet)\n\n\u003c/div\u003e\n\n# Introduction\n\nThis is a Go port of the python's [chardet](https://github.com/chardet/chardet) library. Much respect and appreciation to the original authors for their excellent work.\n\nchardet is a character encoding detector library written in Go. It helps you automatically detect the character encoding of text content.\n\n# Installation\n\nTo install chardet, use `go get`:\n\n```bash\ngo get github.com/wlynxg/chardet\n```\n\n## Supported Encodings \u0026 Languages\n\n**Support Encodings**:\n\n\u003cdetails\u003e\n  \u003csummary\u003eExpand the list of supported encodings\u003c/summary\u003e\n\n- **Ascii**\n- **UTF-8**\n- **UTF-8-SIG**\n- **UTF-16**\n- **UTF-16LE**\n- **UTF-16BE**\n- **UTF-32**\n- **UTF-32BE**\n- **UTF-32LE**\n- **GB2312**\n- **HZ-GB-2312**\n- **SHIFT_JIS**\n- **Big5**\n- **Johab**\n- **KOI8-R**\n- **TIS-620**\n- **MacCyrillic**\n- **MacRoman**\n- **EUC-TW**\n- **EUC-KR**\n- **EUC-JP**\n- **CP932**\n- **CP949**\n- **Windows-1250**\n- **Windows-1251**\n- **Windows-1252**\n- **Windows-1253**\n- **Windows-1254**\n- **Windows-1255**\n- **Windows-1256**\n- **Windows-1257**\n- **ISO-8859-1**\n- **ISO-8859-2**\n- **ISO-8859-5**\n- **ISO-8859-6**\n- **ISO-8859-7**\n- **ISO-8859-8**\n- **ISO-8859-9**\n- **ISO-8859-13**\n- **ISO-2022-CN**\n- **ISO-2022-JP**\n- **ISO-2022-KR**\n- **X-ISO-10646-UCS-4-3412**\n- **X-ISO-10646-UCS-4-2143**\n- **IBM855**\n- **IBM866**\n\n\u003c/details\u003e\n\n**Support Languages**:\n\u003cdetails\u003e\n\u003csummary\u003eExpand the list of supported languages\u003c/summary\u003e\n- Chinese\n- Japanese\n- Korean\n- Hebrew\n- Russian\n- Greek\n- Bulgarian\n- Thai\n- Turkish\n\n\u003c/details\u003e\n\n# Usage\n\n## Basic Usage\n\nThe simplest way to use chardet is with the `Detect` function:\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"github.com/wlynxg/chardet\"\n)\n\nfunc main() {\n\tdata := []byte(\"Your text data here...\")\n\tresult := chardet.Detect(data)\n\tfmt.Printf(\"Detected result: %+v\\n\", result) \n    //Output: Detected result: {Encoding:Ascii Confidence:1 Language:}\n}\n```\n\n## Advanced Usage\n\nFor handling large amounts of text, you can use the detector incrementally. This allows the detector to stop as soon as it reaches sufficient confidence in its result.\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"github.com/wlynxg/chardet\"\n)\n\nfunc main() {\n\t// Create a detector instance\n\tdetector := chardet.NewUniversalDetector(0)\n\t// Process text in chunks\n\tchunk1 := []byte(\"First chunk of text...\")\n\tchunk2 := []byte(\"Second chunk of text...\")\n\tdetector.Feed(chunk1)\n\tdetector.Feed(chunk2)\n\t// Get the result\n\tresult := detector.GetResult()\n\tfmt.Printf(\"Detected result: %+v\\n\", result)\n\t// Output: Detected result: {Encoding:Ascii Confidence:1 Language:}\n}\n```\n\n## Processing Multiple Files\n\nYou can reuse the same detector instance for multiple files by using the `Reset()` method:\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"github.com/wlynxg/chardet\"\n)\n\nfunc main() {\n\tdetector := chardet.NewUniversalDetector(0)\n\tfiles := []string{\"file1.txt\", \"file2.txt\"}\n\tfor _, file := range files {\n\t\tdetector.Reset()\n\t\tdata, err := os.ReadFile(file)\n\t\tif err != nil {\n\t\t\tcontinue\n\t\t}\n\t\tdetector.Feed(data)\n\t\tresult := detector.GetResult()\n\t\tfmt.Printf(\"File %s encoding: %+v\\n\", file, result)\n\t}\n}\n```\n\n# License\n\n`chardet` is licensed under the [MIT License](LICENSE), 100% free and open-source, forever.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwlynxg%2Fchardet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwlynxg%2Fchardet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwlynxg%2Fchardet/lists"}