{"id":41226764,"url":"https://github.com/sanchitram1/pyspam","last_synced_at":"2026-01-23T00:37:15.359Z","repository":{"id":328219051,"uuid":"1091358305","full_name":"sanchitram1/pyspam","owner":"sanchitram1","description":"An ML model to detect spammy python packages","archived":false,"fork":false,"pushed_at":"2025-12-31T08:08:25.000Z","size":1946,"stargazers_count":0,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-04T11:34:51.682Z","etag":null,"topics":["mcp-server","spam-classification"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sanchitram1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-11-06T23:00:16.000Z","updated_at":"2025-12-31T08:08:27.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sanchitram1/pyspam","commit_stats":null,"previous_names":["sanchitram1/pyspam"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sanchitram1/pyspam","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanchitram1%2Fpyspam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanchitram1%2Fpyspam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanchitram1%2Fpyspam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanchitram1%2Fpyspam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sanchitram1","download_url":"https://codeload.github.com/sanchitram1/pyspam/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sanchitram1%2Fpyspam/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28676145,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T20:48:19.482Z","status":"ssl_error","status_checked_at":"2026-01-22T20:48:14.968Z","response_time":144,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mcp-server","spam-classification"],"created_at":"2026-01-23T00:37:14.886Z","updated_at":"2026-01-23T00:37:15.345Z","avatar_url":"https://github.com/sanchitram1.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pyspam\n\n![Tests Passing](https://github.com/sanchitram1/pyspam/actions/workflows/ci.yml/badge.svg)\n[![Coverage Status](https://coveralls.io/repos/github/sanchitram1/pyspam/badge.svg?branch=main)\n](https://coveralls.io/github/sanchitram1/pyspam?branch=main)\n\nA tool for identifying spam packages on Python's registry.\n\n## Pre-requisites\n\n1. [`uv`](https://astral.sh/uv)\n2. [`gcloud`](https://docs.cloud.google.com/sdk/docs/install-sdk)\n3. Not required, but [`pkgx`](https://pkgx.sh) is useful as well.\n4. Not required, but [`xc`](https://xcfile.dev) is useful as well.\n\n\u003e [!TIP]\n\u003e If you have pkgx, all you need to do is prefix everything with `pkgx ...` and it works\n\u003e like magic\n\n## Installation\n\n1. Install uv from [astral.sh/uv](https://astral.sh/uv)\n2. Clone the repository\n3. Run `uv sync` to install dependencies\n  - Alternatively, for all dependencies, including API ones,\nrun `uv sync --all-extras`\n\n## Training\n\nTraining can be broken down into three steps:\n\n### 1. Raw Data\n\nOur data source is from Google BigQuery – `pypi.distribution_metadata`, which contains\nall metadata information for every single package published to PyPI. The\n[training.sql](sql/training.sql) query will generate a labeled dataset of spam vs. non-\nspam python packages\n\n\u003e [!WARNING]\n\u003e In the training.sql file, we reference `project.ground_truth`, which is a labeled\n\u003e dataset that we authored to training the data. We haven't published the dataset to BQ\n\u003e yet, but for now, you would need need to create a table called `ground_truth` which\n\u003e contains two columns: `package_name` and `is_spam`, which is your source for the\n\u003e labels for spam python packages.\n\n### 2. Feature Engineering\n\nThe code to translate the raw data into a set of features for an ML model is in\n[feature_engineering](./feature_engineering) . To run it, just:\n\n```sh\nuv run feature_engineering/pipeline.py \\\n  --input /path/to/input/file.json \\\n  --output /path/to/output/file.json\n```\n\nYou can use defaults set in [settings.py](./feature_engineering/settings.py) as well.\nWe're gonna author a README in that folder to explain how the pipeline works.\n\n### 3. Model training\n\nCurrently the notebook [models.ipynb](training/models.ipynb) is the source to generate\nall the joblib models that we use for our analysis.\n[#10](https://github.com/sanchitram1/pyspam/issues/10) tracks the changes we need to\nmake to this process\n\nThe output is a set of joblib files that are written to [models](models/) .\n\n## API\n\nWe implemented a secured API to demonstrate how an external service (like an MCP server\nor LLM) could use this model to evaluate PyPI packages. The code lives in\n[api](api/main.py) .\n\n### Local Setup\n\nTo run it locally, you need two things:\n1. **Google Credentials:** Authenticate so [bq.py](api/bq.py) can query BigQuery.\n2. **Local Secret:** Set a dummy secret key for JWT generation.\n\n```bash\n# 1. Authenticate with Google Cloud\ngcloud auth application-default login\n\n# 2. Set a temporary secret for local testing\nexport API_TOKEN_SECRET=\"local-dev-secret\"\n\n# 3. Start the server\nuv run uvicorn api.main:app --reload\n```\n\n### Usage\n\n**Step 1: Generate an API Key**\n\nThe API is protected by JWT authentication. You must first generate a temporary access\ntoken, simulating how a user on the portfolio website would gain access.\n\n```bash\ncurl -X POST [http://127.0.0.1:8000/generate-key](http://127.0.0.1:8000/generate-key)\n```\n\n*Copy the `token` string from the JSON response.*\n\n**Step 2: Scan a package**: Replace `\u003cYOUR_TOKEN\u003e` with the token from Step 1:\n\n```bash\ncurl -H \"Authorization: Bearer \u003cYOUR_TOKEN\u003e\" http://127.0.0.1:8000/scan/requests\n```\n\n## Dashboard\n\n\u003e [!NOTE]\n\u003e To run the dashboard locally, you need to have two terminals, one that runs the API,\n\u003e and one that runs the dashboard. See [API](#api) for instructions on how to run the\n\u003e API\n\nTo run the dashboard, execute the following in your terminal\n\n```bash\nuv run streamlit run dashboard.py\n```\n\n## Testing\n\nWe use pytest:\n\n```bash\npytest tests/\n```\n\n## Deploy\n\nRequires: `gcloud`\n\n```bash\ngcloud auth login\ngcloud config set project $PROJECT_ID\ngcloud services enable cloudbuild.googleapis.com run.googleapis.com\ngcloud run deploy pyspam-api --source .\n```\n\n### Troubleshooting\n\n**`Error: \"BigQuery execution failed... Project [old-project-id] has been deleted\"`**\n\nForce a refresh of the local credentials for your current project:\n\n```bash\ngcloud auth application-default login\n```\n\nMake sure to sign in with the Google account associated with the active project.\n\n## Model Context Protocol (MCP)\n\nThis repository includes a standalone MCP server (mcp_server/) that allows AI agents\n(like Claude Desktop or Cursor) to natively \"consult\" the PySpam API before suggesting\npackages.\n\n### Quick Start (Requires [`pkgx`](https://pkgx.sh))\n\nThe server script is self-bootstrapping. It uses a shebang to automatically pull the\ncorrect Python version and dependencies (mcp, httpx) via pkgx + uv. You will need to\nmake mcp_server/server.py executable:\n\n```bash\nchmod +x mcp_server/server.py\n```\n\n### Client Configuration\n\nTo use this with your AI editor, add the configuration below to your MCP Settings file.\n\n- Cursor: Cmd+Shift+P \u003e MCP: Open Settings File\n- Claude Desktop: ~/Library/Application\nSupport/Claude/claude_desktop_config.json\n\n**Option 1:** The pkgx Method (Recommended) Since the script is executable, you can\npoint the client directly to the file. Note: You must use the absolute path to the repo.\n\n```json\n{\n  \"mcpServers\": {\n    \"pyspam\": {\n      \"command\": \"/ABSOLUTE/PATH/TO/pyspam/mcp_server/server.py\",\n      \"args\": []\n    }\n  }\n}\n```\n\n**Option 2:** The Standard uv Method If you do not use pkgx, you can invoke the server\nusing standard uv.\n\n```json\n{\n  \"mcpServers\": {\n    \"pyspam\": {\n      \"command\": \"uv\",\n      \"args\": [\n        \"run\",\n        \"--with\", \n        \"mcp\", \n        \"--with\", \n        \"httpx\",\n        \"/ABSOLUTE/PATH/TO/pyspam/mcp_server/server.py\"\n      ]\n    }\n  }\n}\n```\n\n## Tasks\n\nCollection of repeatable tasks runnable via `xc`\n\n### install\n\nInstalls the packages exposed by this repo\n\n```bash\nuv pip install -e .\n```\n\n### sync\n\nGets all the requirements you need for developing everything\n\nRequires: install\n\n```bash\nuv sync --all-extras\n```\n\n### test \n\nRuns all the tests\n\n```bash\npytest .\n```\n\n### lint\n\n```bash\nruff format . \nruff check . --fix\n```\n\n### deploy\n\n```bash\ngcloud run deploy pyspam-api \\\n  --source . \\\n  -- region us-west1 \\\n  --max-instances 1 \\\n  --allow-unauthenticated\n  --set-secrets=\"API_TOKEN_SECRET=pyspam-jwt-secret:latest\" \\\n```\n\n### api\n\n```bash\nuv run uvicorn api.main:app --reload\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsanchitram1%2Fpyspam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsanchitram1%2Fpyspam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsanchitram1%2Fpyspam/lists"}