{"id":19756748,"url":"https://github.com/thebracket/llm_rust_article_1","last_synced_at":"2026-06-20T04:03:38.093Z","repository":{"id":250038658,"uuid":"833291486","full_name":"thebracket/llm_rust_article_1","owner":"thebracket","description":null,"archived":false,"fork":false,"pushed_at":"2024-07-25T15:38:19.000Z","size":7737,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-28T09:19:14.995Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thebracket.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-24T18:29:01.000Z","updated_at":"2024-07-24T20:32:31.000Z","dependencies_parsed_at":"2025-01-10T22:50:01.463Z","dependency_job_id":null,"html_url":"https://github.com/thebracket/llm_rust_article_1","commit_stats":null,"previous_names":["thebracket/llm_rust_article_1"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thebracket/llm_rust_article_1","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thebracket%2Fllm_rust_article_1","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thebracket%2Fllm_rust_article_1/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thebracket%2Fllm_rust_article_1/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thebracket%2Fllm_rust_article_1/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thebracket","download_url":"https://codeload.github.com/thebracket/llm_rust_article_1/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thebracket%2Fllm_rust_article_1/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34556497,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-20T02:00:06.407Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T03:16:49.167Z","updated_at":"2026-06-20T04:03:38.072Z","avatar_url":"https://github.com/thebracket.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Categorizing Data with Large Language Models in Rust\n\nLibreQoS is an open source project for monitoring and providing quality-of-experience for Internet Service Providers\n(ISPs) and large networks. It runs as a \"middle-box\", monitoring traffic that passes through it. It recently\ngained the ability to track individual *data flows* - connections between two endpoints.\n\nPublic Internet IP addresses belong to an ASN - an Autonomous System Number. An ASN is a unique number that identifies\na network on the Internet. Tracking flows allows you to see which ASNs your users are connecting to, how much data\nis flowing, and monitor the quality of the connection.\n\nJust providing ASN names isn't very useful. Most people who see that Joe is connecting to \"SSI-AS\" won't realize that \nthis means they are watching Netflix! To make this data useful, we need to categorize the ASNs. There are a *lot* of \nASN IP blocks - over 40,000! - so we need to automate this process.\n\n## Obtaining the ASN Data\n\n[ipinfo.io](https://ipinfo.io) provides downloadable CSV files containing information about IP addresses and ASNs. \n(An outdated copy is included in [data/asn.csv](data/asn.csv).) The data looks like this:\n\n```csv\nstart_ip,end_ip,asn,name,domain\n1.0.0.0,1.0.0.255,AS13335,\"Cloudflare, Inc.\",cloudflare.com\n1.0.4.0,1.0.7.255,AS38803,Wirefreebroadband Pty Ltd,gtelecom.com.au\n1.0.16.0,1.0.16.255,AS2519,ARTERIA Networks Corporation,arteria-net.com\n```\n\n\u003e There are 420,772 lines in the file - we're not going to list them all here!\n\n## Loading the Data\n\n\u003e The code for this is in [load_data/](./load_data/).\n\nRust makes reading CSV files easy. We'll use a couple of crates to help us out: Serde and CSV. \nWe'll also include `anyhow` for easy error handling. You can add them as follows:\n\n```bash\ncargo add serde -F derive\ncargo add csv\ncargo add anyhow\n```\n\nNow we create a structure to define the data. We won't be using most of the fields, so we'll add a]\n`#[allow(dead_code)]` to suppress warnings about unused fields. We also add `#[derive(Deserialize)]`\nto automatically generate the code to read the data from the CSV file.\n\n```rust\n#[derive(Deserialize)]\n#[allow(dead_code)] // Ignore unused fields. They have to be here to match the CSV file.\nstruct AsnRow {\n    start_ip: String,\n    end_ip: String,\n    asn: String,\n    name: String,\n    domain: String,\n}\n```\n\nNext, let's read the data from the CSV file. We only care about the `domain` field, so we'll\nwrite some code to load the data and return just that field:\n\n```rust\npub fn load_asn_domains() -\u003e Result\u003cVec\u003cString\u003e\u003e {\n    let data = include_str!(\"../../data/asn.csv\");\n    let mut reader = csv::Reader::from_reader(data.as_bytes());\n    let rows: Vec\u003c_\u003e = reader\n        .deserialize::\u003cAsnRow\u003e() // Deserialize - returns a result\n        .into_iter() // Consume the iterator\n        .flatten()// Keep only Ok records\n        .map(|r| r.domain.to_lowercase().trim().to_string()) // Extract just the domain\n        .filter(|d| !d.is_empty()) // Remove empty domains\n        .collect(); // Move the results into a vector\n\n    println!(\"Loaded {} domains\", rows.len());\n\n    Ok(rows)\n}\n```\n\nCalling this function returns 412,795 domains. A scan through the data shows a *lot* of duplicates!\nWe don't want to categorize the same domain multiple times, so we need to de-duplicate the data.\nFortunately, a crate named `Itertools` makes this very easy.\n\n\u003e If you're tempted to just add it all to a `HashSet` and let that do the job---it'll work, but\n\u003e it will be substantially slower. Generating a hash for every string is expensive. It's *much*\n\u003e faster to sort the strings, iterate forward and only retain unique items!\n\nAdd a dependency on `Itertools`:\n\n```bash\ncargo add itertools\n```\n\nWe can then add two lines to our function, and we're done!\n\n```rust\npub fn load_asn_domains() -\u003e Result\u003cVec\u003cString\u003e\u003e {\n    let data = include_str!(\"../../data/asn.csv\");\n    let mut reader = csv::Reader::from_reader(data.as_bytes());\n    let rows: Vec\u003c_\u003e = reader\n        .deserialize::\u003cAsnRow\u003e() // Deserialize - returns a result\n        .into_iter() // Consume the iterator\n        .flatten()// Keep only Ok records\n        .map(|r| r.domain.to_lowercase().trim().to_string()) // Extract just the domain\n        .filter(|d| !d.is_empty()) // Remove empty domains\n        .sorted() // Sort the results\n        .dedup() // Remove duplicates\n        .collect(); // Move the results into a vector\n\n    println!(\"Loaded {} domains\", rows.len());\n\n    Ok(rows)\n}\n```\n\nThat runs very quickly - and we're down to 63,519 domains. That's still a lot---but at least we aren't\ndoing the same work over and over again.\n\n## Setting Up a Local LLM\n\nYou probably don't want to pay for 63,519 API calls (assuming everything is one shot, works perfectly\nfirst time, and you never need to run a second test!). So let's set up a local LLM. I used\n`Ollama` on my Linux box: it neatly wraps the complexities of `llama-cpp`, supports my AMD GPU\nout of the box, and is easy to install. Your setup will vary by platform. Visit \n[https://ollama.com/](https://ollama.com/) and follow the instructions there. I'm using the\n`llama3.1` model. I installed it with `ollama pull llama3.1`.\n\nOnce you have `Ollama` installed, you can test it with `ollama run llama3.1` and had a little\nchat:\n\n```\n\u003e\u003e\u003e Is Rust a great language?\nRust is a highly-regarded programming language that has gained popularity in recent years, and opinions about it vary depending on \none's background, experience, and goals. Here are some aspects where Rust excels:\n\n**Great features:**\n\n1. **Memory Safety**: Rust's ownership model ensures memory safety without relying on garbage collection. This makes it an attractive \nchoice for systems programming and applications that require high performance.\n2. **Concurrency**: Rust provides built-in support for concurrency through its `async/await` syntax, making it easy to write \nasynchronous code.\n3. **Performance**: Rust is designed with performance in mind. It can compete with C++ in terms of execution speed and memory usage.\n4. **Type System**: Rust's type system is both expressive and flexible. It allows for static checking of types, ensuring that your \ncode is correct at compile-time rather than runtime.\n\n(and on, and on, and on - this is a chatty LLM!)\n```\n\nSo now that we have a working local LLM, let's talk to it via the API from Rust.\n\n## Talking to the LLM\n\nIn our Rust code, we're going to add three dependencies: `Tokio` (an async runtime) and \n`Reqwest` (an HTTP client) and `serde_json` (to make JSON easy to work with).\n\n```bash\ncargo add tokio -F full\ncargo add reqwest\ncargo add serde_json -F json\n```\n\nTalking to Ollama uses a relatively simple [Rest API](https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion). If you've\ntaken our *Rust Foundations* or *Rust as a Service* classes, you'll know this one!\n\nLet's start by setting a constant to the local LLM's API endpoint URL:\n\n```rust\nconst LLM_API: \u0026str = \"http://localhost:11434/api/generate\";\n```\n\nNow, we'll define a structure to receive data from the API:\n\n```rust\n#[derive(Deserialize)]\nstruct Response {\n    response: String,\n}\n```\n\nAnd finally, we can write a function that talks to the LLM:\n\n```rust\nasync fn llm_completion(prompt: \u0026str) -\u003e Result\u003cString\u003e {\n    // Use serde_json to quickly make a JSON object\n    let request = json!({\n        \"model\": \"llama3.1\",\n        \"prompt\": prompt,\n    });\n\n    // Start the Reqwest client\n    let client = reqwest::Client::new();\n    // Create a POST request, add the request JSON, and send it\n    let mut res = client.post(LLM_API)\n        .json(\u0026request)\n        .send()\n        .await?;\n\n    // Empty string to assemble the response\n    let mut response = String::new();\n    \n    // While res.chunk() returns Some(data), the stream\n    // holds data we want. So we can grab each chunk,\n    // and add it to the response string.\n    while let Some(chunk) = res.chunk().await? {\n        let chunk: Response = serde_json::from_slice(\u0026chunk)?;\n        response.push_str(\u0026chunk.response);\n    }\n\n    // Return the response\n    Ok(response)\n}\n```\n\nThe only tricky part here is that we are *streaming* the response, rather than\nprocessing it all at once. LLMs return one response at a time. Fortunately,\nstreaming is built into Reqwest.\n\nLet's give this a try:\n\n```rust\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    let response = llm_completion(\"Good Morning!\").await?;\n    println!(\"{}\", response);\n    Ok(())\n}\n```\n\nOn my machine, this returns:\n\n```\nGood morning! Hope you're having a great start to the day! How can I help or chat with you today?\n```\n\nNow that we have a working LLM, we can start categorizing the data.\n\n## Categorizing the Data: First Try - Oneshot!\n\n\u003e \"Oneshot\" is a term used in the LLM world to describe a single request with no\nhelper data, no introspection, chain-of-thought or anything else. In a perfect\nworld, this would be all you need. (Note: this isn't a perfect world).\n\nIt will take a while to categorize 63,519 domains, so we'll start with a small\nsample. Let's add the `rand` crate (`cargo add rand`) to help us pick \nrandom test data.\n\nWe've already loaded the domain list, and we have a function to talk to the LLM---so \nlet's put this together:\n\n```rust\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    let domains = load_asn_domains()?;\n\n    // Pick a small random sample\n    let mut rng = rand::thread_rng();\n    let sample = domains.choose_multiple(\u0026mut rng, 2);\n\n    // Let's do some categorizing\n    for domain in sample {\n        println!(\"Domain: {}\", domain);\n        let prompt = format!(\"Please categorize this domain with a single keyword. \\\n        Do not elaborate, do not explain or otherwise \\\n        enhance the answer. The domain is: {domain}\");\n        let response = llm_completion(\u0026prompt).await?;\n        println!(\"Response: {}\", response);\n    }\n    Ok(())\n}\n```\n\nThings to note:\n\n* We're using a very simple prompt. Adding \"do not elaborate\" and \"do not explain\" is a\n  common technique to get a single-word answer. LLMs like to talk.\n* We're not providing *any* additional information or context---we're relying on the LLM's baked-in knowledge.\n* I like to say \"please\" to LLMs, so when the inevitable *Super Intelligence Apocalypse* happens, hopefully I'll be spared.\n\nRunning this gave me:\n\n```\nDomain: r-tk.net\nResponse: Radio\nDomain: 365it.fr\nResponse: Software\n```\n\nIs the answer any good? I'd not heard of either of those domains, so---unlike the LLM---I fired up\na browser to try and determine if the LLM was hallucinating (I was definitely expecting \nsome hallucination!).\n\nUnsurprisingly, the LLM was wrong. `r-tk.net` is a Russian company that provides Internet and\nstreaming television services. `365it.fr`is an IT services company in France.\n\nI ran it a few more times, and the results weren't great. Sometimes, the LLM was\nspot on---and most of the time, it was hallucinating an effectively random answer.\n\n\u003e Large Language Models are \"next token predictors\". While there is some evidence that emergent\n\u003e behavior develops in large models, they aren't a font of all knowledge. If a domain wasn't\n\u003e part of their training data, they don't know what it is---and will \"hallucinate\" a likely-\n\u003e sounding answer.\n\nSo let's try and give the LLM some context to work with.\n\n## Adding Context\n\nThe vast majority of the listed domains have a website associated. Maybe we could\nscrape text from the website and use that as context?\n\n\u003e Many recent LLMs can use tools. For example, ChatGPT can fire up a web browser\n\u003e to search for the answer to your question. This article isn't going to try to\n\u003e incorporate tool usage---instead, we'll scrape the website data and provide\n\u003e context directly.\n\nLet's make use of the `reqwest` crate to fetch the website data, and the `scraper`\ncrate to extract some text. LLMs have a pretty short context window, so we\ndon't want to overwhelm the poor AI with too much data.\n\n```bash\ncargo add scraper\n```\n\nWe'll add a function to fetch the website data:\n\n```rust\nasync fn website_text(domain: \u0026str) -\u003e Result\u003cString\u003e {\n  let url = format!(\"http://{}/\", domain);\n\n  // Build a header with a Firefox user agent\n  let mut headers = header::HeaderMap::new();\n  headers.insert(\n    header::USER_AGENT,\n    header::HeaderValue::from_static(\"Mozilla/5.0 (platform; rv:geckoversion) Gecko/geckotrail Firefox/firefoxversion\")\n  );\n\n  // Setup Reqwest with the header\n  let client = reqwest::Client::builder()\n          .default_headers(headers)\n          .timeout(Duration::from_secs(30))\n          .build()?;\n\n  // Fetch the website\n  let body = client\n          .get(\u0026url).send().await?\n          .text().await?;\n\n  // Parse the HTML\n  let doc = scraper::Html::parse_document(\u0026body);\n  // Search for parts of the site with text in likely places\n  let mut content = Vec::new();\n  for items in [\"title\", \"meta\", \"ul,li\", \"h1\", \"p\"] {\n    content.extend(find_content(items, \u0026doc));\n  }\n  // We now have a big list of words (hopefully) from the website\n  let result = content\n          .into_iter() // Consuming iterator\n          .sorted() // Sort alphabetically\n          .dedup_with_count()// De-duplicate, and return a tuple (count, word)\n          .sorted_by(|a, b| b.0.cmp(\u0026a.0)) // Sort by count, descending\n          .map(|(count, word)| word)// Take only the word\n          .take(100)// Take the top 100 words\n          .join(\" \"); // Join them into a string\n\n  Ok(result)\n}\n```\n\nThat's quite a mouthful, but it does a lot:\n\n1. The function creates a header mimicking Firefox. Many domains won't reply without a valid `USER-AGENT` string.\n2. It creates a Reqwest instance featuring the header, and a 30-second timeout window for obtaining data from the remote website.\n3. It fetches the website, and extracts the result's body as text.\n4. It uses `scraper` to parse the HTML.\n5. It searches for text in likely places: the title, meta tags, unordered lists, list items, headers, and paragraphs. The `find_content` function (below) gathers the word list from each section of the website.\n6. It sorts the words.\n7. It de-duplicates the word list, keeping count of how many times each word appeared.\n8. It retains the 100 most used words on the website.\n\nThe helper function uses the `scraper` crate to extract text from the HTML,\nand convert it into lowercase words as a vector of strings:\n\n```rust\nfn find_content(selector: \u0026str, document: \u0026Html) -\u003e Vec\u003cString\u003e {\n  let selector = scraper::Selector::parse(selector).unwrap();\n  let mut content = Vec::new();\n  for element in document.select(\u0026selector) {\n    // Get all text elements matching the selector\n    let e: String = element.text().collect::\u003cString\u003e();\n\n    // Split at whitespace, and filter out words shorter than 3 characters and\n    // convert to lowercase.\n    let e: Vec\u003cString\u003e = e.split_whitespace()\n            .filter(|s| s.len() \u003e 3)\n            .map(|s| s.trim().to_lowercase())\n            .collect();\n\n    if !e.is_empty() {\n      content.extend(e);\n    }\n  }\n\n  content\n}\n```\n\n\u003e Why `trim` and `to_lowercase`? Websites are often full of whitespace, and often have\n\u003e pretty strange formatting. We only care about the word *content* - we don't want the\n\u003e gaps in-between. Normalizing to lower-case ensures that \"Provision\" and \"provision\"\n\u003e will be counted as the same word.\n\nHere's the resulting context from `provision.ro` (which my system picked at random):\n\n```\nsecurity management data application threat protection risk firewall access endpoint \ndigital privacy e-mail encryption gateway response secure testing detection hunting \nidentity infrastructure network assessment training vulnerability advance \nanalytics/ anti-phishing asset attacks authentication automated automation awareness \nbots browser casb centric classification client collaboration container cspm database \nddos deception detection/protection discovery ediscovery governance human incident \nintelligence isolation malware mast penetration privilege rights runtime sase \nself-protection side siem soar third-party tools ueba visibility wireless media \noperations analysis cloud compliance generation masking messaging mobile next social \ntrust zero about services solutions provision technologies partners contact find home \nmore cyber experience expertise help information provision’s\n```\n\nCombining these functions together yields a list of at-most 100 words from the website. We\nhave to keep the list small, to not overwhelm the LLM's context window---and keep processing\nrelatively performant.\n\nSo now we slightly change our `main` function to include the context in the prompt:\n\n```rust\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n    let domains = load_asn_domains()?;\n\n    // Pick a small random sample\n    let mut rng = rand::thread_rng();\n    let sample = domains.choose_multiple(\u0026mut rng, 5);\n\n    // Let's do some categorizing\n    for domain in sample {\n        println!(\"Domain: {}\", domain);\n        if let Ok(text) = website_text(domain).await {\n            let prompt = format!(\"Please categorize this domain with a single keyword. \\\n            Do not elaborate, do not explain or otherwise enhance the answer. \\\n            The domain is: {domain}. Here are some items from the website: {text}\");\n\n            let response = llm_completion(\u0026prompt).await?;\n            println!(\"Response: {}\", response);\n        } else {\n            println!(\"unable to scrape: {domain}\");\n        }\n    }\n    Ok(())\n}\n```\n\nSo let's see how we're doing with some context included:\n\n```\nDomain: eternet.cc\nResponse: Internet\nDomain: wilken-rz.de\nunable to scrape: wilken-rz.de\nDomain: orovalleyaz.gov\nResponse: Government\nDomain: embl.de\nResponse: Biotechnology\nDomain: baikonur.net\nResponse: Internet\n```\n\nManually visiting these sites:\n\n* `eternet.cc` is indeed an Internet provider.\n* `orovalleyaz.gov` is the official site for the town of Oro Valley, Arizona. So \"Government\" is right.\n* `embl.de` is the European Molecular Biology Laboratory. So \"Biotechnology\" is right.\n* `baikonur.net` is a Russian site that provides Internet services. So \"Internet\" is right.\n\nOne scraping failure, and 4/4 on categorization! I repeated this a few times, and it was\nconsistently good!\n\nNow let's turn this into a program that can complete our intended task.\n\n## Let's Add Some Performance!\n\n\u003e Running the LLM queries is by far the slowest part of this process. We aren't going to\n\u003e optimize `Ollama` itself. Sadly, buying a better GPU---or using a faster LLM- is the best \n\u003e way to speed that up.\n\nRunning through each site one-at-a-time is going to take a *really long time*. Let's speed things up,\nand make use of Tokio's async capabilities (one thread per core, work stealing). We'll add the\n`futures` crate to provide `join_all`---which I find easier than Tokio's `JoinSet` system.\n\n```bash\ncargo add futures\n```\n\n### Appending Results to a File\n\nLet's add a helper function that appends a line to a file:\n\n```rust\nasync fn append_to_file(filename: \u0026str, line: \u0026str) -\u003e Result\u003c()\u003e {\n    let mut file = tokio::fs::OpenOptions::new()\n        .append(true)\n        .create(true)\n        .open(filename)\n        .await?;\n    tokio::io::AsyncWriteExt::write_all(\u0026mut file, format!(\"{}\\n\", line).as_bytes()).await?;\n    Ok(())\n}\n```\n\nThis is *not* thread-safe---but it's designed to be called from a channel, which\nwill serialize the calls to it.\n\nNext, we'll build a function that receives a report of failures, and uses the `append_to_file`\nfunction to append errors as they occur:\n\n```rust\nasync fn failures() -\u003e Sender\u003cString\u003e {\n    let (tx, mut rx) = tokio::sync::mpsc::channel::\u003cString\u003e(32);\n    tokio::spawn(async move {\n        while let Some(domain) = rx.recv().await {\n            println!(\"Failed to scrape: {}\", domain);\n            // Append to \"failures.txt\"\n            if let Err(e) = append_to_file(\"failures.txt\", \u0026domain).await {\n                eprintln!(\"Failed to write to file: {}\", e);\n            }\n        }\n    });\n    return tx;\n}\n```\n\nWe'll do the same for success, but we'll use a struct to store both the domain and the\ncategory (I think a struct is nicer than a `(String, String)` tuple):\n\n```rust\nstruct Domain {\n    domain: String,\n    category: String,\n}\n\nasync fn success() -\u003e Sender\u003cDomain\u003e {\n    let (tx, mut rx) = tokio::sync::mpsc::channel::\u003cDomain\u003e(32);\n    tokio::spawn(async move {\n        while let Some(domain) = rx.recv().await {\n            println!(\"Domain: {}, Category: {}\", domain.domain, domain.category);\n            // Append to \"categories.csv\"\n            if let Err(e) = append_to_file(\"categories.csv\", \u0026format!(\"{},{}\", domain.domain, domain.category)).await {\n                eprintln!(\"Failed to write to file: {}\", e);\n            }\n        }\n    });\n    return tx;\n}\n```\n\n### Categorization as a Function\n\nLet's move our prompt generation and calling into a function as well:\n\n```rust\nasync fn categorize_domain(domain: \u0026str, text: \u0026str) -\u003e Result\u003cDomain\u003e {\n    let prompt = format!(\"Please categorize this domain with a single keyword in English. \\\n            Do not elaborate, do not explain or otherwise enhance the answer. \\\n            The domain is: {domain}. Here are some items from the website: {text}\");\n\n    let response = llm_completion(\u0026prompt).await?;\n    Ok(Domain {\n        domain: domain.to_string(),\n        category: response,\n    })\n}\n```\n\nFinally, let's tie this together to process the entire list.\n\n### And Let's Call It!\n\nWe're probably going to melt some CPU and GPU chips here! Let's do it!\n\n```rust\n#[tokio::main]\nasync fn main() -\u003e Result\u003c()\u003e {\n  // Load the domains\n  let mut domains = load_asn_domains()?;\n\n  // Shuffle the domains (so in test runs we aren't always hitting the same ones)\n  domains.shuffle(\u0026mut rand::thread_rng());\n\n  // Create the channels for results\n  let report_success = success().await;\n  let report_failures = failures().await;\n\n  // Create a big set of tasks\n  let already_done = std::fs::read_to_string(\"categories.csv\").unwrap_or_default();\n  let mut futures = Vec::new();\n  for domain in domains.into_iter() {\n    // Skip domains we've already done - in case we have to run it more than once\n    if already_done.contains(\u0026domain) {\n      continue;\n    }\n    // Clone the channels - they are designed for this.\n    let my_success = report_success.clone();\n    let my_failure = report_failures.clone();\n    let future = tokio::spawn(async move {\n      match website_text(\u0026domain).await {\n        Ok(text) =\u003e {\n          match categorize_domain(\u0026domain, \u0026text).await {\n            Ok(domain) =\u003e { let _ = my_success.send(domain).await; },\n            Err(_) =\u003e { let _ = my_failure.send(domain).await; },\n          }\n        }\n        Err(_) =\u003e {\n          let _ = my_failure.send(domain).await;\n        }\n      }\n    });\n    futures.push(future);\n\n    // Limit the number of concurrent tasks\n    if futures.len() \u003e= 32 {\n      let the_future: Vec\u003c_\u003e = futures.drain(..).collect();\n      let _ = join_all(the_future).await;\n    }\n  }\n\n  // Call any leftover items\n  join_all(futures).await;\n\n  Ok(())\n}\n```\n\nI ran this with `take(10)` to test it. There were no failures! The category list looks good, too:\n\n```csv\nphatriasulung.net.id,Webhosting\ncornellcollege.edu,Education\nconnexcs.com,Communications\nprovedorsupply.net.br,Internet\nthornburg.com,Investment\nusp.org,Pharmaceuticals\nvalassis.com,Marketing\nagen-rs.si,Energy\nbalasai.com,Hosting\nlima.co.uk,Technology\n```\n\nWe have the basis of a working categorization engine!\n\n# Conclusion\n\nLLMs provide a powerful tool for categorizing data, and Rust makes it easy to\nwork with them. Rust has excellent tools for scraping web data and massaging\nthe results, allowing you to provide context to your LLM calls. With Tokio, Reqwest\nand futures, you can easily parallelize your work to make the most of your\nhardware.\n\nThere's quite a few possible improvements to this code:\n\n* Ask the LLM to use one of a provided list of keywords, rather than coming up with them for you.\n* You could use more than one LLM and take the majority answer.\n* You should try different LLM models. Llama 3 is a fun, open source model---but there's a *lot* of models out there!\n* You could definitely improve the word selection algorithm. Remove stop words, prioritize the title, etc.\n\nThe final program categorizes about 1,300 domains per hour on my MacBook Air M1. It's not\namazingly fast, but it's not bad either. It's definitely faster than doing it by hand! \nSpot-checking the results shows that it's pretty accurate, too. It's not perfect, but it's\nmore accurate than I would be if I read all 63,519 domains myself!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthebracket%2Fllm_rust_article_1","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthebracket%2Fllm_rust_article_1","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthebracket%2Fllm_rust_article_1/lists"}