{"id":30071406,"url":"https://github.com/graph-com/llm_bp","last_synced_at":"2025-08-08T12:42:46.740Z","repository":{"id":300208355,"uuid":"933834082","full_name":"Graph-COM/LLM_BP","owner":"Graph-COM","description":"[ICML 2025] Generalization Principles for Inference over Text-Attributed Graphs with Large Language","archived":false,"fork":false,"pushed_at":"2025-06-29T19:04:38.000Z","size":320,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-29T20:19:00.802Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Graph-COM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-16T19:41:36.000Z","updated_at":"2025-06-29T19:04:41.000Z","dependencies_parsed_at":"2025-06-20T12:47:04.195Z","dependency_job_id":null,"html_url":"https://github.com/Graph-COM/LLM_BP","commit_stats":null,"previous_names":["graph-com/llm_bp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Graph-COM/LLM_BP","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Graph-COM%2FLLM_BP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Graph-COM%2FLLM_BP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Graph-COM%2FLLM_BP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Graph-COM%2FLLM_BP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Graph-COM","download_url":"https://codeload.github.com/Graph-COM/LLM_BP/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Graph-COM%2FLLM_BP/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269423882,"owners_count":24414615,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-08T02:00:09.200Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-08T12:42:40.671Z","updated_at":"2025-08-08T12:42:46.713Z","avatar_url":"https://github.com/Graph-COM.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM_BP\n\n[[Paper]](https://arxiv.org/html/2502.11836v2)\n\n![model](model-arch.png)\n\n## Overview\nThis is the official implementation of ICML 2025 paper ['Model Generalization on Text Attribute Graphs: Principles with Large Language Models'](http://arxiv.org/abs/2502.11836), Haoyu Wang, Shikun Liu, Rongzhe Wei, Pan Li.\n\n## Repository Structure\n\nThe repository structure is as follows:\n\n```\nLLM_BP (Root Directory)\n│── dataset/        # Contains dataset files\n│── model/          # Stores model implementation of LLM-BP and LLM-BP (appr.)\n│── results/        # Contains generated results from GPT-4o (the predictions on testset) and GPT-4o-mini (predictions on homophily ratio)\n│── zero_shot.py  # zero shot inference\n│── few_shot.py        # few shot inference\n│── run_gpt.py     # run openai GPT to predict the results by taking raw node texts\n│── pred_h.py     # predict the homophily ratio r by sampling edges\n│── generate_llm.py     # generate the embeddings of vanilla LLM2Vec or task-adaptive encoder\n│── generate_lm.py     # generate the embeddings of sbert or Roberta\n│── generate_llm_gpt.py     # generate the embeddings of text-embedding-3-large\n│── README.md       # Documentation file\n```\n\n## STEP 0.1 Environment Setup\n\nTo set up the environment, follow these steps:\n\n```\nconda create -n llmbp python==3.8.18 \nconda activate llmbp\nconda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia\npip install pyg_lib==0.3.1+pt21cu121 -f https://data.pyg.org/whl/torch-2.1.0+cu121.html \npip install torch_scatter==2.1.2 -f https://data.pyg.org/whl/torch-2.1.0+cu121.html \npip install torch_sparse==0.6.18+pt21cu121 -f https://data.pyg.org/whl/torch-2.1.0+cu121.html \npip install torch_cluster==1.6.3+pt21cu121 -f https://data.pyg.org/whl/torch-2.1.0+cu121.html \npip install torch_spline_conv==1.2.2+pt21cu121 -f https://data.pyg.org/whl/torch-2.1.0+cu121.html\npip install transformers==4.46.3 \npip install sentence_transformers==2.2.2\npip install dgl==2.4.0+cu121 -f https://data.dgl.ai/wheels/torch-2.1/cu121/repo.html \npip install openai \npip install torch_geometric==2.5.0 \npip install protobuf \npip install accelerate\n```\n-------------\n\n## STEP 0.2 Dataset Preparation\n\nThe dataset structure should be organized as follows:\n\n```plaintext\n/dataset/\n│── [dataset_name]/\n│   │── processed_data.pt    # Contains labels and graph information\n│   │── [encoder]_x.pt       # Features extracted by different encoders\n│   │── categories.csv       # label name raw texts\n│   │── raw_texts.pt       # raw text of each node\n```\n\n### File Descriptions\n- **`processed_data.pt`**: A PyTorch file storing the processed dataset, including graph structure and node labels. Note that in heterophilic datasets, thie is named as [Dataset].pt, where Dataset could be Cornell, etc, and should be opened with DGL.\n- **`[encoder]_x.pt`**: Feature matrices extracted using different encoders, where `[encoder]` represents the encoder name.\n- **`categories.csv`**: raw label names.\n- **`raw_texts.pt`**: raw node texts. Note that in heterophilic datasets, this is named as [Dataset].csv, where Dataset can be Cornell, etc.\n\n### Dataset Naming Convention\n`[dataset_name]` should be one of the following:\n- `cora`\n- `citeseer`\n- `pubmed`\n- `bookhis`\n- `bookchild`\n- `sportsfit`\n- `wikics`\n- `cornell`\n- `texas`\n- `wisconsin`\n- `washington`\n\n### Encoder Naming Convention\n`[encoder]` can be one of the following:\n- `sbert` (the sentence-bert encoder)\n- `roberta` (the Roberta encoder)\n- `llmicl_primary` (the vanilla LLM2Vec)\n- `llmicl_class_aware` (the task-adaptive encoder)\n- `llmgpt_text-embedding-3-large` (the embedding api text-embedding-3-large by openai)\n\n\nEnsure the datasets are placed correctly for smooth execution.\n\n### Download Pre-Calculated Embeddings and datasets\n\nThey could be found at: [huggingface repository](https://huggingface.co/datasets/Graph-COM/Text-Attributed-Graphs), one could directly download, place under /dataset/ folder.\n\n## STEP 1: Generating Dataset Embeddings\n\n```\npython generate_llm.py --dataset [DATASET] --version [VERSION]\n```\n\n### Example: \n```\nCUDA_VISIBLE_DEVICES=0,1 python generate_llm.py --dataset cora --version class_aware\n```\n\n### Parameters:\n- `[DATASET]`: The name of the dataset.\n- `[VERSION]`: \n  - `primary` → Vanilla LLM2Vec\n  - `class_aware` → Task-adaptive encoding\n\nEnsure that the appropriate CUDA devices are set before running the script.\n\n### Download pre-calculated embeddings\n\nWe have enclosed the pre-calculated embeddings for the encoders in: [huggingface repository](https://huggingface.co/datasets/Graph-COM/Text-Attributed-Graphs), one may directly download and put them under the /dataset folder\n\n------------\n\n## STEP 2: Generate the predictions from GPT-4o\n\n```\npython run_gpt.py --mode [MODE] --model [MODEL] --dataset [DATASET]\n```\n\n### Parameters:\n- `[MODEL]`: The model selection (e.g., 4o for GPT-4o).\n- `[DATASET]`: The name of the dataset.\n- `[MODE]`: when set as inference, it do inference and save results, when set as evaluate, it evaluate the results of the model\n\n\n\n### Download pre-calculated predictions\n\nWe have enclosed the pre-calculated predictions from GPT-4o in: [huggingface repository](https://huggingface.co/datasets/Graph-COM/Text-Attributed-Graphs), one may directly download and put them under the /results folder\n\n----------------\n\n## STEP 3: Predict the homophily ratio  of the dataset\n\n```\npython pred_r.py --mode [MODE] --dataset [DATASET] --model [MODEL]\n```\n### Parameters:\n- `[DATASET]`: The name of the dataset.\n- `[MODEL]`: The model selection (e.g., 4o_mini).\n- `[MODE]`: when set as inference, it do inference and save results, when set as evaluate, it makes prediction with the model\n\n### Fill the value\nFill the predicted value in H_dict in zero_shot.py or few_shot.py\n\n### Download pre-calculated predictions\n\nWe have enclosed the pre-calculated predictions from GPT-4o-mini in: [huggingface repository](https://huggingface.co/datasets/Graph-COM/Text-Attributed-Graphs), one may directly download and put them under the /results folder\n\n## STEP 4: Zero-shot Inference\n\n```\npython zero_shot.py --dataset [DATASET] --encoder [ENCODER] --model 4o\n```\n\n### Parameters:\n- `[DATASET]`: The name of the dataset.\n- `[ENCODER]`: The encoder model (e.g., sbert, roberta, llmicl_primary, llmicl_class_aware, llmicl_text-embedding-3-large, etc.).\n- `4o`: Specifies the use of GPT-4o as averaged class embeddings.\n\n## STEP 5: Few-shot Inference\n\n```\npython few_shot.py --dataset [DATASET] --encoder [ENCODER]\n```\n\n### Parameters:\n- `[DATASET]`: The name of the dataset.\n- `[ENCODER]`: The encoder model (e.g., sbert, roberta, llmicl_primary, llmicl_class_aware, llmicl_text-embedding-3-large, etc.).\n\n\n## Acknowledgements\nThe dataset pre-processing, formats and code implementations are inspired by or built upon [GLBench](https://github.com/NineAbyss/GLBench), [Text-space graph foundation model](https://github.com/CurryTang/TSGFM), and [LLaGA](https://github.com/VITA-Group/LLaGA).\n\n\n## Citation\n\nIf you find our work helpful, please consider citing:\n\n```\n@article{wang2025model,\ntitle={Model Generalization on Text Attribute Graphs: Principles with Large Language Models},\nauthor={Wang, Haoyu and Liu, Shikun and Wei, Rongzhe and Li, Pan},\njournal={arXiv preprint arXiv:2502.11836},\nyear={2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraph-com%2Fllm_bp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgraph-com%2Fllm_bp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraph-com%2Fllm_bp/lists"}