{"id":21022596,"url":"https://github.com/aspadax/dim","last_synced_at":"2026-02-11T06:03:21.630Z","repository":{"id":263439851,"uuid":"887335650","full_name":"AspadaX/dim","owner":"AspadaX","description":"Use LLMs for effective and refined vectorizations. ","archived":false,"fork":false,"pushed_at":"2025-02-01T05:48:46.000Z","size":83,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-19T17:54:16.725Z","etag":null,"topics":["cv","embedding-models","gpt","machine-learning","nlp","rust","vector"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AspadaX.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-12T15:11:16.000Z","updated_at":"2025-02-01T05:48:49.000Z","dependencies_parsed_at":"2024-11-18T15:51:58.736Z","dependency_job_id":"17d7c3cd-7bd1-4fd0-9b32-e564bf4828dc","html_url":"https://github.com/AspadaX/dim","commit_stats":null,"previous_names":["aspadax/dim"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/AspadaX/dim","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AspadaX%2Fdim","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AspadaX%2Fdim/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AspadaX%2Fdim/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AspadaX%2Fdim/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AspadaX","download_url":"https://codeload.github.com/AspadaX/dim/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AspadaX%2Fdim/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29327871,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T03:52:29.695Z","status":"ssl_error","status_checked_at":"2026-02-11T03:52:23.094Z","response_time":97,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cv","embedding-models","gpt","machine-learning","nlp","rust","vector"],"created_at":"2024-11-19T11:12:57.382Z","updated_at":"2026-02-11T06:03:21.617Z","avatar_url":"https://github.com/AspadaX.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dim\n\n`dim` is a Rust library for flexible and extensible vectorization of different types of data (images, text, etc.) using Large Language Models (LLMs). It allows concurrent processing of multiple prompts to generate meaningful vector representations.\n\n## Features\n\n- Support for multiple data types (Image, and Text for now. Other data formats in the future)\n- Concurrent vectorization using multiple prompts\n- Compatible with OpenAI API format. You may use Ollama API or so as a drop-in replacement \n- Flexible vector dimension control through prompt design\n- Built-in validation for vectorization results\n\n## Installation\n\nAdd this to your `Cargo.toml`:\n\n```toml\n[dependencies]\ndim-rs = \"0.2.0\"\n```\n\n## Quick Start\n\n### Vectorize Text\n\n```rust\nuse dim_rs::{prelude::*, vectorization::ModelParameters};\nuse tokio;\nuse anyhow::{Error, Result};\nuse async_openai;\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Error\u003e {\n    // Load text\n    let test_text: String = \"Hi, this is dim. I am here to vectorize whatever your want.\"\n        .to_string();\n\n    // Create a Vector object from the image\n    let mut vector: Vector\u003cString\u003e = Vector::from_text(\n        test_text\n    );\n\n    // Initialize client\n    let client: async_openai::Client\u003casync_openai::config::OpenAIConfig\u003e = async_openai::Client::with_config(\n        async_openai::config::OpenAIConfig::new()\n            .with_api_base(\"http://192.168.0.101:11434/v1\") // comment this out if you use OpenAI instead of Ollama\n            .with_api_key(\"your_api_key\")\n    );\n\n    // Initialize prompts\n    let prompts: Vec\u003cString\u003e = vec![\n        \"Score the sentiment intensity of the text from 1 (extremely negative) to 9 (extremely positive). Consider emotional language, tone, and context. Format your response exactly like this example: {'sentiment_score': 7}\".to_string(),\n        \"Rate the formality of the text from 1 (highly informal, slang-heavy) to 9 (highly formal, academic/professional). Format your response exactly like this example: {'formality_score': 4}\".to_string(),\n        \"Assess the emotional intensity of the text from 1 (neutral/clinical) to 9 (highly emotional, passionate, or provocative). Format your response exactly like this example: {'emotional_score': 8}\".to_string(),\n        \"Score how subjective the text is from 1 (purely factual/objective) to 9 (heavily opinionated/subjective). Format your response exactly like this example: {'subjectivity_score': 6}\".to_string(),\n        \"Rate the linguistic complexity of the text from 1 (simple vocabulary/short sentences) to 9 (dense jargon/long, intricate sentences). Format your response exactly like this example: {'complexity_score': 3}\".to_string(),\n        \"Score the dominant intent: 1-3 (informative/educational), 4-6 (persuasive/argumentative), 7-9 (narrative/storytelling). Format your response exactly like this example: {'intent_score': 5}\".to_string(),\n        \"Rate how urgent or time-sensitive the text feels from 1 (no urgency) to 9 (immediate action required). Format your response exactly like this example: {'urgency_score': 2}\".to_string(),\n        \"Score the specificity of details from 1 (vague/abstract) to 9 (highly specific/concrete examples). Format your response exactly like this example: {'specificity_score': 7}\".to_string(),\n        \"Rate the politeness of the tone from 1 (rude/confrontational) to 9 (extremely polite/deferential). Format your response exactly like this example: {'politeness_score': 8}\".to_string(),\n        \"Categorize the text's primary domain: 1-3 (technical/scientific), 4-6 (casual/everyday), 7-9 (artistic/creative). Format your response exactly like this example: {'domain_score': 4}\".to_string(),\n    ];\n\n    // Vectorize image\n    let model_parameters = ModelParameters::new(\"minicpm-v\".to_string(), None, None);\n    vectorize_string_concurrently(\n        prompts,\n        \u0026mut vector, \n        client,\n        model_parameters\n    ).await?;\n\n    // Print vectorized result\n    println!(\"Vector: {:?}\", vector.get_vector());\n    println!(\"Vector Length: {:?}\", vector.get_vector().len());\n\n    Ok(())\n}\n```\nThe result should be something like this:\n```rust\nVector: [7, 4, 8, 6, 3, 5, 2, 7, 8, 4]\n```\nNotice that each prompt generates a value between 1 and 9. The final vector is a combination of these values.\n\n### Vectorize Images\n\n```rust\nuse dim_rs::{prelude::*, vectorization::ModelParameters};\nuse image::DynamicImage;\nuse tokio;\nuse anyhow::{Error, Result};\nuse async_openai::{Client, config::OpenAIConfig};\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Error\u003e {\n    // Load image\n    let image_path: \u0026str = \"./examples/images/54e2c8ea-58ef-4871-ae3f-75eabd9a2c6c.jpg\";\n    let test_image: DynamicImage = image::open(image_path).unwrap();\n\n    // Create a Vector object from the image\n    let mut vector: Vector\u003cDynamicImage\u003e = Vector::from_image(test_image);\n\n    // Initialize client\n    let client: Client\u003cOpenAIConfig\u003e = Client::with_config(\n        OpenAIConfig::new()\n            .with_api_base(\"http://192.168.0.101:11434/v1\") // comment this out if you use OpenAI instead of Ollama\n            .with_api_key(\"your_api_key\")\n    );\n\n    // Initialize prompts\n    let prompts: Vec\u003cString\u003e = vec![\n        \"output in json. Rate the image's offensiveness from 0.0 to 10.0. {'offensiveness': your score}\".to_string(),\n        \"output in json. Rate the image's friendliness from 0.0 to 10.0. {'friendliness': your score}\".to_string(),\n    ];\n\n    // Initialize model parameters\n    let model_parameters = ModelParameters::new(\n        \"minicpm-v\".to_string(), \n        Some(0.7), \n        None\n    );\n\n    // Vectorize image\n    vectorize_image_concurrently(\n        prompts,\n        \u0026mut vector, \n        client,\n        model_parameters\n    ).await?;\n\n    // Print vectorized result\n    println!(\"Vector: {:?}\", vector.get_vector());\n    println!(\"Vector Length: {:?}\", vector.get_vector().len());\n\n    Ok(())\n}\n```\nOnce again, the result should be something like this:\n```rust\nVector: [0.0, 10.0]\n```\nNotice that each prompt generates a value between 0.0 and 10.0. The final vector is a combination of these values.\n\n## How It Works\n\n1. The library takes your data (text/image) and creates a `Vector` object\n2. You provide multiple prompts that will be used to analyze different aspects of the data\n3. The prompts are processed concurrently using the specified LLM\n4. Results are combined into a single vector representation\n5. The dimensionality of the final vector is determined by the number of prompts and their specified outputs\n\n## Configuration\n\n- Works with OpenAI API style. Also, this project uses `async_openai` for API calls. \n- Customize API endpoint using:\n```rust\n.with_api_base(\"your_api_endpoint\")\n```\n\n## License\n\nMIT License\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faspadax%2Fdim","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faspadax%2Fdim","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faspadax%2Fdim/lists"}