{"id":36726374,"url":"https://github.com/mtnmunuklu/lescatit","last_synced_at":"2026-01-12T12:02:06.817Z","repository":{"id":147937868,"uuid":"376865380","full_name":"mtnmunuklu/lescatit","owner":"mtnmunuklu","description":"Provides to crawl and categorize URL addresses","archived":false,"fork":false,"pushed_at":"2024-05-08T06:49:06.000Z","size":275921,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-05-08T07:43:07.589Z","etag":null,"topics":["docker","fiber","go","golang","grpc","kubernetes","mongo","nlp","protocol-buffers","rest-api","traefik"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mtnmunuklu.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["mtnmunuklu"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2021-06-14T15:08:27.000Z","updated_at":"2024-05-08T06:49:03.000Z","dependencies_parsed_at":"2024-05-08T07:40:21.418Z","dependency_job_id":"bf8ba859-a1f8-41d5-a5e8-6c3bfbcd18c9","html_url":"https://github.com/mtnmunuklu/lescatit","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/mtnmunuklu/lescatit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtnmunuklu%2Flescatit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtnmunuklu%2Flescatit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtnmunuklu%2Flescatit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtnmunuklu%2Flescatit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mtnmunuklu","download_url":"https://codeload.github.com/mtnmunuklu/lescatit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtnmunuklu%2Flescatit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28338973,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T10:58:46.209Z","status":"ssl_error","status_checked_at":"2026-01-12T10:58:42.742Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","fiber","go","golang","grpc","kubernetes","mongo","nlp","protocol-buffers","rest-api","traefik"],"created_at":"2026-01-12T12:02:05.911Z","updated_at":"2026-01-12T12:02:06.806Z","avatar_url":"https://github.com/mtnmunuklu.png","language":"Go","funding_links":["https://github.com/sponsors/mtnmunuklu"],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg width=\"300\" height=\"300\" src=\"images/logo.svg\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://pkg.go.dev/github.com/mtnmunuklu/lescatit\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%93%9A%20godoc-pkg-informational.svg\" alt=\"Go Doc\"\u003e\u003c/a\u003e \u003ca href=\"https://goreportcard.com/report/github.com/mtnmunuklu/lescatit\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%93%9D%20goreport-A+-success.svg\" alt=\"Go Report\"\u003e\u003c/a\u003e \u003ca href=\"https://travis-ci.com/\"\u003e\u003cimg src=\"https://img.shields.io/badge/%E2%9A%99%20build-X-success.svg\" alt=\"Build Status\"\u003e\u003c/a\u003e \u003ca href=\"https://lescatit.com/\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%93%BD%20demo-online-red.svg\" alt=\"Live Demo\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n# Lescatit \u003csub\u003e\u003csmall\u003e\u003csmall\u003e(Let's categorized it)\u003c/small\u003e\u003c/small\u003e\u003c/sub\u003e\n\nLescatit is a project developed in **Go**, **Mongo**, **Docker**, **Kubernetes**, **gRPC** and **Fiber** technologies, providing URL crawling and categorization functionality.\n\n## Table of Contents\n\n* [Features](#features)\n* [Setup](#setup)\n* [Usage](#usage)\n* [License](#license)\n\n## Features\n\nLescatit offers the following features:\n\n* Getting user(s) information\n* Deleting a user\n* Changing user roles\n* Updating user passwords\n* Updating user email addresses\n* Updating usernames\n* Getting content of URL(s)\n* Crawling URL(s)\n* Categorizing URL(s)\n* Generating a classification model\n* Getting the classification model\n* Updating the classification model\n* Deleting classification model(s)\n* Listing all classification models\n* Getting URL categories\n* Updating URL categories\n* Reporting miscategorization\n* Adding URL addresses\n* Deleting URL(s)\n* Listing all URLs\n\n## Setup\n\nTo set up Lescatit, follow these steps:\n\n1. Download the latest version:\n\n    ```\n    LATEST_VERSION=$(wget -qO - https://api.github.com/repos/mtnmunuklu/lescatit/releases/latest \\\n    | grep tag_name \\\n    | cut -d  '\"' -f 4)\n\n    curl -LJO https://github.com/mtnmunuklu/lescatit/archive/refs/tags/$LATEST_VERSION.tar.gz\n    ```\n\n2. Extract the downloaded file:\n\n    ```\n    FILE_NAME=lescatit-$(echo $LATEST_VERSION | cut -d 'v' -f 2)\n    tar -xvf $FILE_NAME.tar.gz\n    ```\n\n3. Execute the setup scripts:\n\n    ```\n    cd $FILE_NAME/scripts\n    # Execute on worker and control plane servers.\n    bash tools/setup_tools.sh\n    bash k8s/setup_k8s.sh\n    # Execute only on the first control plane server.\n    # It will create setup_k8s_control_plane.sh and setup_k8s_worker.sh files.\n    # Control plane and worker scripts are for joining the Kubernetes cluster.\n    # You can use these scripts on new nodes when you add new nodes as control plane or worker.\n    bash k8s/setup_k8s_first_control_plane.sh\n    # Execute only on first control plane server.\n    bash setup_lescatit.sh\n    ```\n\n## Usage\n\nLescatit consists of 6 different services: [authentication](authentication), [crawler](crawler), [categorizer](categorizer), [categorization](categorization), [api](api) and [web](web).  Unlike the other services, incoming requests to the web service are directly routed without passing through the API service. The requested URL plays a role in the decision-making process.\n\nTo understand the features of each service, the available endpoints, how to make requests, and the expected responses, refer to the [api.pdf](docs/api/api.pdf) file under the [docs](docs) folder.\n\nYou can also access the documents describing the `software structure` of each service under the [docs](docs) folder.\n\n## License\n\nAlterix is licensed under the MIT License. See [LICENSE](LICENSE) for the full text of the license.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtnmunuklu%2Flescatit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmtnmunuklu%2Flescatit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtnmunuklu%2Flescatit/lists"}