{"id":21999126,"url":"https://github.com/frizensami/plagiarism-basic","last_synced_at":"2025-04-30T21:48:41.160Z","repository":{"id":36789667,"uuid":"230042889","full_name":"frizensami/plagiarism-basic","owner":"frizensami","description":"Offline quick-and-dirty text plagiarism checker written in Rust","archived":false,"fork":false,"pushed_at":"2024-04-05T09:15:42.000Z","size":1269,"stargazers_count":19,"open_issues_count":2,"forks_count":3,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-16T04:44:12.855Z","etag":null,"topics":["plagiarism","rust","string-similarity"],"latest_commit_sha":null,"homepage":"https://sriramsami.com","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frizensami.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-12-25T04:46:39.000Z","updated_at":"2024-04-05T08:28:26.000Z","dependencies_parsed_at":"2022-08-25T06:50:57.149Z","dependency_job_id":null,"html_url":"https://github.com/frizensami/plagiarism-basic","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frizensami%2Fplagiarism-basic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frizensami%2Fplagiarism-basic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frizensami%2Fplagiarism-basic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frizensami%2Fplagiarism-basic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frizensami","download_url":"https://codeload.github.com/frizensami/plagiarism-basic/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227225786,"owners_count":17750747,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["plagiarism","rust","string-similarity"],"created_at":"2024-11-29T22:27:44.528Z","updated_at":"2024-11-29T22:27:45.158Z","avatar_url":"https://github.com/frizensami.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Basic Plagiarism Detection\n## 1. Motivation\nOnline plagiarism detection tools usually come with a few constraints. It could be a paid-only service, the number of characters to check could be artificially limited, etc. This tool aims to fill a gap where:\n- Plagiarism cases are usually simple copy-paste jobs of a few text phrases with minor edits,\n- Paying for an online tool is unpalatable,\n- The source texts that might be copied from can be put together manually by the user into a few files (i.e. the Internet is not automatically searched by the tool), or the only concern is people copying from each other, and\n- Running a command-line tool is simple enough for the user\n\n## 2. Table of Contents\n\u003c!-- TOC depthFrom:2 --\u003e\n\n- [Basic Plagiarism Detection](#basic-plagiarism-detection)\n  - [1. Motivation](#1-motivation)\n  - [2. Table of Contents](#2-table-of-contents)\n  - [3. Philosophy](#3-philosophy)\n  - [4. Definitions](#4-definitions)\n  - [5. Project Objectives](#5-project-objectives)\n    - [5.1. Hard Objectives](#51-hard-objectives)\n    - [5.2. Soft (Optimization) Objectives](#52-soft-optimization-objectives)\n    - [5.3. Anti-Objectives](#53-anti-objectives)\n  - [6. Project Status](#6-project-status)\n  - [7. Installation Options](#7-installation-options)\n    - [7.1. Binary Release](#71-binary-release)\n    - [7.2. Cargo Install](#72-cargo-install)\n    - [7.3. Building from source](#73-building-from-source)\n  - [8. Usage](#8-usage)\n  - [9. Examples](#9-examples)\n  - [10. Technical Details](#10-technical-details)\n    - [10.1. Defining Plagiarism](#101-defining-plagiarism)\n    - [10.2. Choosing n, s and M](#102-choosing-n-s-and-m)\n\n\u003c!-- /TOC --\u003e\n## 3. Philosophy\nThis tool is really only to catch amateur attempts at plagiarism. Whatever your opinion is on plagiarism tools, we can probably agree that lazy copy-paste jobs of chunks of text should probably not be respected. Therefore, this tool is designed to make detecting these situations easier.\n\n## 4. Definitions\n- Untrusted: Something that might be plagiarised\n- Trusted: Something that is definitely not plagiarised, but might be a source used in plagiarism\n\n## 5. Project Objectives\n### 5.1. Hard Objectives\n- Detect potential cases of plagiarism between multiple untrusted strings (intra-source plagiarism)\n- Detect potential cases of plagiarism between trusted source strings and untrusted strings (external-source plagiarism)\n\n### 5.2. Soft (Optimization) Objectives\n1. Minimizing false positive and false negative detection rates\n2. Fastest possible detection speed without compromising objective 1.\n\n### 5.3. Anti-Objectives\n- The project **does not aim to search the Internet automatically** for potential plagiarism sources. These sources have to be assembled by users manually into text files in the `trusted` folder.\n- This is **not a tool for code plagiarism detection!**. Please refer to [Stanford's MOSS](https://theory.stanford.edu/~aiken/moss/) instead.\n\n## 6. Project Status\n- All options are usable in the executable, and the `equal` metric is quite fast at detecting copy-paste plagiarism of a few words.\n- The `lev` metric is faster now by parallelizing using `rayon` and promises more fine-grained control over how different two phrases can be. However, it might be slow for very large datasets.\n- The current output format is both in HTML and in the terminal. In the HTML output, both texts are displayed side-by-side, with all detected plagiarized phrases being highlighted in bold. The percent of plagiarism detected (# plagiarized words / all words * 100%) is also indicated for each source pair. The goal is for all of this to be color-coded, which is a project priority.\n\n## 7. Installation Options\n### 7.1. Binary Release\nDownload a binary from the [Releases](https://github.com/frizensami/plagiarism-basic/releases/) page. Currently only `x86_64-unknown-linux-gnu` targets are supported. \n\n### 7.2. Cargo Install\n1. Install the `rust` language toolchain (https://www.rust-lang.org/tools/install).\n1. Run `cargo install plagiarism-basic`\n1. The executable should be installed from `crates.io` automatically by `cargo`. \n\n### 7.3. Building from source\n1. The previous two options will likely distribute an outdated version of this tool. For the most up-to-date code, building from source is recommended.\n1. Install the `rust` language toolchain (https://www.rust-lang.org/tools/install).\n1. `git clone` this repository to a folder of your choice.\n1. Run `cargo build --release` in that folder.\n1. The `target/release` folder will contain the `plagiarism-basic` executable to be used.\n\n## 8. Usage\nSome setup is required to use this tool.\n- There are **three folders** where source text can be kept. These folders can be **named anything you'd like**, but the names must be specified in the options to the program later.\n    1. **Mandatory**: an **\"untrusted\" folder** to hold source texts that might be guilty of plagiarism. This could be where student submissions are placed.\n    1.  Not mandatory: A **\"trusted\" folder** where texts that are highly likely to be copied from can be placed. This could contain text files with the content of the top ten Google search results for a particular question, or reference text material that is likely to be copied.\n    1. Not mandatory: An **\"ignore\" folder** where strings of text that should be ignored are placed. If students are answering a particular essay question, you might want to put that question itself as a source in the \"ignore\" folder so that those strings in students answers will not trigger a plagiarism warning.\n- An example can be found in the `plagiarismbasic_lib/testfiles/cs-corpus` directory of the GitHub repository.\n- Each folder may contain any number of files in its top-level directory (i.e. subdirectories inside those folders will be ignored). Each file will be treated as a separate source of text (whether trusted, untrusted, or content to be ignored). \n- The files in both folders must only contain UTF-8 interpretable text (which should be the case barring any really special characters). \n- The name of the file will be used in the output of the program, so naming the files appropriately is a good idea. \n- After these steps are done, the `plagiarism-basic` executable can be run and the path to these folders can be specified in the arguments to the executable.\n```\n$ ./plagiarism-basic -h\nBasic Plagiarism Checker \nSriram Sami (@frizensami on GitHub)\nChecks for plagiarism using very basic metrics between different text files\n\nUSAGE:\n    plagiarism-basic [FLAGS] [OPTIONS] --metric \u003cmetric\u003e --sensitivity \u003csensitivity\u003e --similarity \u003csimilarity\u003e --untrusted \u003cuntrusted-directory\u003e\n\nFLAGS:\n    -h, --help        Prints help information\n        --openhtml    If the HTML file should be opened automatically after writing\n        --cli         If the results should be printed to the command line\n        --html        If the results should be printed to a HTML file\n    -V, --version     Prints version information\n\nOPTIONS:\n    -i, --ignore \u003cignore-directory\u003e          Sets the directory containing text files with content to be ignored from\n                                             plagiarism checks.\n    -m, --metric \u003cmetric\u003e                    Sets the metric (function) used for similarity testing. Equal checks that\n                                             both strings are equal, and lev uses the Levenshtein distance [possible\n                                             values: equal, lev]\n    -n, --sensitivity \u003csensitivity\u003e          Sets the number of words required to form a unit of plagiarism checking\n    -s, --similarity \u003csimilarity\u003e            Sets the threshold value for plagiarism to be detected by a chosen metric\n    -t, --trusted \u003ctrusted-directory\u003e        Sets the directory containing trusted text files. Each file will be treated\n                                             as a separate possible plagiarism source text.\n    -u, --untrusted \u003cuntrusted-directory\u003e    Sets the directory containing untrusted text files. Each file will be\n                                             treated as a separate submission by a separate person.\n```\n## 9. Examples\n**Exact string matching:**\n```\n./plagiarism-basic -t testfiles/cs-corpus/t/ -u ./testfiles/cs-corpus/ut/ -m equal -n 10 -s 0 --openhtml --cli --html\n```\n\n**Levenshtein distance matching:**\n```\n./plagiarism-basic -t testfiles/cs-corpus/t/ -u ./testfiles/cs-corpus/ut/ -m lev -n 5 -s 1 --openhtml --cli --html\n```\n\n**With Cargo from this folder:**\n```\ncargo run --release -- -t plagiarismbasic_lib/testfiles/cs-corpus/t/ -u plagiarismbasic_lib/testfiles/cs-corpus/ut/ -m equal -n 10 -s 0 --openhtml --cli --html\n```\n\n\n\n**Output (Basic HTML)**\n\n![HTML output example](./readme-assets/plag_v8_html.png)\n\n## 10. Technical Details\n### 10.1. Defining Plagiarism\nInformally, two strings that are long enough and with the same number of words that are \"similar enough\" by a chosen metric are considered to be plagiarised. \n\nFormally:\n- Two separate strings (`s1` and `s2`) consisting of words (a sequence of characters without a space) are considered plagiarised if: \n- Both have `l` words\n    - Where `l` \u003c some user-chosen *sensitivity value* `n`\n- Where a metric `M` and *similarity value* `s` produces `M(s1, s2) \u003c= s`\n- Subject to pre-processing of\n    - Removing CR + LF\n    - Removing extra spaces (only one space between words)\n    - Converting all letters to lowercase\n    - Removing all non alphanumeric characters\n\n### 10.2. Choosing n, s and M\n- `n` is a user-chosen value to indicate **how many words** a string needs to be before being considered for plagiarism. If the value is too low, the false positive rate will be very high (imagine matching the phrase \"I am\" for `n = 2`). If the value is too high, correspondingly, the false negative rate will be too high.\n- `s` is a user-chosen value to indicate **how similar** the strings have to be before being considered for plagiarism. This follows the opposite false positive/negative trend as `n` (too high = too many false positive and vice versa), but only affects results when a non `equal` metric is used.\n- `M` is the **metric** used to evaluate the strings for similarity. They can be one of the following\n    - `equal`: checks if the strings are equal, ignores `s` value. Uses hashed set intersections, very fast.\n    - `lev`: uses the Levenshtein distance between the words, uses the `s` value. Compares between all combinations of string fragments.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrizensami%2Fplagiarism-basic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrizensami%2Fplagiarism-basic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrizensami%2Fplagiarism-basic/lists"}