{"id":23920440,"url":"https://github.com/sunny-chung/bigtext","last_synced_at":"2025-04-18T16:24:49.133Z","repository":{"id":271066720,"uuid":"912299648","full_name":"sunny-chung/bigtext","owner":"sunny-chung","description":"A data structure for JVM and `@Composable` text field and label for Jetpack Compose to edit big text (up to 100 MB) efficiently.","archived":false,"fork":false,"pushed_at":"2025-04-10T16:43:24.000Z","size":1938,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-10T16:58:57.209Z","etag":null,"topics":["jetpack-compose","jvm","large-text","text-editing","text-field"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sunny-chung.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-05T07:03:07.000Z","updated_at":"2025-04-10T16:43:27.000Z","dependencies_parsed_at":"2025-01-05T08:27:30.857Z","dependency_job_id":"280937a0-8f13-4342-9174-0e4a96b6bc55","html_url":"https://github.com/sunny-chung/bigtext","commit_stats":null,"previous_names":["sunny-chung/bigtext"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny-chung%2Fbigtext","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny-chung%2Fbigtext/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny-chung%2Fbigtext/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunny-chung%2Fbigtext/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sunny-chung","download_url":"https://codeload.github.com/sunny-chung/bigtext/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249517887,"owners_count":21284854,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jetpack-compose","jvm","large-text","text-editing","text-field"],"created_at":"2025-01-05T15:36:04.306Z","updated_at":"2025-04-18T16:24:49.095Z","avatar_url":"https://github.com/sunny-chung.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BigText, BigTextField, BigTextLabel\n\n![BigText Data Structure](https://img.shields.io/maven-central/v/io.github.sunny-chung/bigtext-datastructure?label=bigtext-datastructure)\n![BigText UI Composable](https://img.shields.io/maven-central/v/io.github.sunny-chung/bigtext-ui-composable?label=bigtext-ui-composable)\n\n![JVM](https://img.shields.io/badge/JVM-blue)\n\n[![As Seen In - jetc.dev Newsletter Issue #249](https://img.shields.io/badge/As_Seen_In-jetc.dev_Newsletter_Issue_%23249-blue?logo=Jetpack+Compose\u0026amp;logoColor=white)](https://jetc.dev/issues/249.html)\n\n**BigText** is an in-memory data structure capable of manipulating large string (tested up to 100 MB) specialized for text editing applications. It is independent of UI frameworks, only supports JVM currently but is planned to go Kotlin Multiplatform. It is possible to implement your own text buffer for BigText to support other CharSequence.\n\n**BigTextField** is a Jetpack Compose text field component utilizing BigText to provide the capability of editing large styleable text, and is designed to fix the known issues and limitations of the existing `TextField`, `BasicTextField` and `BasicTextField2`. It aims to replace `BasicTextField`. It is efficient enough to be used for syntax highlighting. It is available for desktop platforms only, but is planned to support all platforms that supported by Jetpack Compose Multiplatform.\n\n**BigTextLabel**, a Jetpack Compose text component, is the read-only version of BigTextField.\n\n![Demo](doc/media/demo.png)\n![Demo](doc/media/demo2.png)\n\n\n## Comparing with `TextField` / `BasicTextField` / `BasicTextField2`\n\nThe goods:\n\n- It does not freeze when a 300K text, styled with text transformation, is rendered -- it renders instantly.\n- It does not crash when a 10 MB text is feed in.\n- The text processing can be customized to be in background (see demo), so theoretical it can support even larger size.\n- You don't have to calculate transformation offset anymore. I calculate it for you, using balanced binary trees, efficiently.\n- Text field random bouncing? Unexpected selection? Selected text out of view could not be copied? Input eaten because typing too fast? No more weird bugs.\n- You can feed in `AnnotatedString` as the initial value.\n- Soft wrap can be turned off.\n- `AnnotatedString` can react to mouse events.\n- When real bugs are reported, it will be fixed in some days, not unfixed after 200 days.\n- Help me to append to this list.\n\nThe bads:\n\n- It does not embrace immutability.\n- You cannot always access the full string, or it would be as slow as `BasicTextField`.\n- You write incremental transformation instead, or it would be as slow as `BasicTextField`.\n- You take care the time complexity of your callback handlers, incremental transformations and decorators, or it would be as slow as `BasicTextField`.\n- You use `WeakReference` to wrap BigText instances as cache keys to prevent memory leaks.\n- Lots of fancy stuffs are not yet supported. See [limitations](#limitations).\n\n\n## Performance\n\n### Time\nIn the demo, typing in a 100 MB text reacts fast enough to every keystroke.\n\nThe recomposition time of a 10 MB text field was measured to be within 2ms.\n\n### Memory\nA 100 MB BigText has an overhead of around 20 MB memory, so it consumes around 120 MB (Latin) or 220 MB (Unicode) memory in JVM initially. The overhead grows according to user inputs in order to support undo/redo and transformations, and is configurable. Note that use of UI framework may bring additional overheads.\n\nMemory usage complexity: O(BigText buffer size) + O(transformed buffer size) + O(undo capacity) + O(redo capacity) + O(node count)\n\n\n## Limitations\n- BigText does not support Compose Multiplatform 1.7\n- BigText is entirely living in memory, not supporting lazy I/O loading. It may be difficult to load gigabytes of text.\n- UI Integration with Jetpack Compose is currently only available in JVM platforms.\n- Multiple line heights are not yet supported. All the rows are assumed to have same height as the given text style.\n- Emoji modifiers and sequences are not yet supported (Contribution is welcome!)\n- Breaking by word in soft wrap according to Unicode algorithm are not yet available (Contribution is welcome!)\n- `requestFocus` does not work properly (see [workaround](#requesting-focus-a-temporary-workaround-available-since-v201))\n- Styles (e.g. underline) on space characters do not work properly\n- Following parameters in Jetpack Compose BasicTextField/BasicText have no equivalent in this library or easy workaround:\n  - Text directions\n  - Paragraphs\n  - Font features\n  - Letter spacing\n  - MutableInteractionSource\n\n\n## Demo App\n\n`./gradlew :demo-ui-composable:run`\n\n\n## Getting Started\n\n![BigText Data Structure](https://img.shields.io/maven-central/v/io.github.sunny-chung/bigtext-datastructure?label=bigtext-datastructure)\n![BigText UI Composable](https://img.shields.io/maven-central/v/io.github.sunny-chung/bigtext-ui-composable?label=bigtext-ui-composable)\n\nTo use without Jetpack Compose,\n\n```kotlin\nimplementation(\"io.github.sunny-chung:bigtext-datastructure:\u003cversion\u003e\")\n```\n\nTo use with Jetpack Compose,\n\n```kotlin\nimplementation(\"io.github.sunny-chung:bigtext-ui-composable:\u003cversion\u003e\")\n```\n\n\n## Usage Examples\n\n### Simplest\n\n```kotlin\nval bigTextFieldState by rememberConcurrentLargeAnnotatedBigTextFieldState(\"initial super big string\", cacheKey)\nval scrollState = rememberScrollState()\n\nBox {\n    BigTextField(\n        textFieldState = bigTextFieldState,\n        color = Color.Black,\n        cursorColor = Color.Blue,\n        isSoftWrapEnabled = true,\n        scrollState = scrollState,\n        modifier = Modifier.fillMaxSize()\n    )\n    VerticalScrollbar(\n        adapter = rememberScrollbarAdapter(scrollState),\n        modifier = Modifier.align(Alignment.TopEnd).fillMaxHeight()\n    )\n}\n```\n\n### Asynchronous Loading with a Loading Spin\n\n```kotlin\nvar bigTextFieldState by remember {\n    mutableStateOf(\n        BigTextFieldState(\n            text = ConcurrentBigText(BigText.createFromLargeAnnotatedString(AnnotatedString(\"\"))),\n            viewState = BigTextViewState()\n        )\n    )\n}\nval scrollState = rememberScrollState()\nval horizontalScrollState = rememberScrollState()\nvar numOfComputations by remember { mutableStateOf(0) }\nval coroutineScope = rememberCoroutineScope()\n\nfun loadBigTextInBackground() {\n    ++numOfComputations\n    coroutineScope.launch {\n        val textState = withContext(Dispatchers.IO) {\n            val initialText = \"A\".repeat(100000000)\n            BigTextFieldState(\n                ConcurrentBigText(BigText.createFromLargeAnnotatedString(AnnotatedString(initialText))),\n                BigTextViewState()\n            )\n        }\n        bigTextFieldState = textState\n        scrollState.scrollTo(0)\n        --numOfComputations\n    }\n}\n\nBox {\n    BigTextField(\n        textFieldState = bigTextFieldState,\n        color = Color.Black,\n        cursorColor = Color.Black,\n        fontFamily = FontFamily.Serif,\n        isSoftWrapEnabled = false,\n        onHeavyComputation = { computation -\u003e // compute in background and display a \"loading\" spinner\n            withContext(coroutineScope.coroutineContext) {\n                ++numOfComputations\n            }\n            withContext(Dispatchers.IO) {\n                computation()\n            }\n            withContext(coroutineScope.coroutineContext) {\n                --numOfComputations\n            }\n        },\n        scrollState = scrollState,\n        horizontalScrollState = horizontalScrollState, // only required for soft wrap disabled\n        modifier = Modifier.background(Color(224, 224, 224))\n            .fillMaxSize()\n    )\n    if (numOfComputations \u003e 0) {\n        CircularProgressIndicator(Modifier.align(Alignment.Center))\n    } else {\n        VerticalScrollbar(\n            adapter = rememberScrollbarAdapter(scrollState),\n            modifier = Modifier.align(Alignment.TopEnd).fillMaxHeight()\n        )\n        HorizontalScrollbar(\n            adapter = rememberScrollbarAdapter(horizontalScrollState),\n            modifier = Modifier.align(Alignment.BottomStart).fillMaxWidth()\n        )\n    }\n}\n\nLaunchedEffect(Unit) {\n    loadBigTextInBackground()\n}\n```\n\n### Transformation and More\n\nhttps://github.com/user-attachments/assets/5733ccfc-8c4c-4678-a893-fa22994a1c7b\n\n```kotlin\n/**\n * Transform an input so that it is displayed in a format of \"(xxx) xxxx-xxxx\"\n */\nclass PhoneNumberIncrementalTransformation : IncrementalTextTransformation\u003cUnit\u003e {\n    override fun initialize(text: BigText, transformer: BigTextTransformer) {\n        transform(text, transformer)\n    }\n\n    override fun afterTextChange(change: BigTextChangeEvent, transformer: BigTextTransformer, context: Unit) {\n        transform(change.bigText, transformer)\n    }\n\n    private fun transform(text: BigText, transformer: BigTextTransformer) {\n        transformer.restoreToOriginal(0 .. text.length)\n        if (text.isNotEmpty) {\n            transformer.insertAt(0, \"(\")\n            if (text.length \u003e= 3) transformer.insertAt(3, \") \")\n            if (text.length \u003e= 3 + 4) transformer.insertAt(7, \"-\")\n        }\n    }\n}\n```\n\n```kotlin\nBigTextField(\n    textTransformation = remember { PhoneNumberIncrementalTransformation() }, // `remember` is needed to avoid recomputation!\n    isSingleLineInput = true,\n    maxInputLength = 3 + 4 + 4,\n    inputFilter = remember { { it.replace(\"[^0-9]\".toRegex(), \"\") } },\n    // ...\n)\n```\n\n### Incremental Transformation \u0026 Decorator\n\nIncremental Transformation means when the text is changed, the transformation only processes and updates the changed portions.\n\nFor an example of incremental transformation, see [VariableIncrementalTransformation](demo-ui-composable/src/jvmMain/kotlin/com/sunnychung/lib/multiplatform/bigtext/demo/VariableIncrementalTransformation.kt).\n\nIt can be a mess to implement incremental transformation. You may want to look at incremental parsers.\n\nThe time complexity of implementations of the following functions should be significantly lower than `O(BigText's length)`. Ideally, `O(lg(BigText's length))`.\n- `IncrementalTextTransformation\u003c*\u003e.beforeTextChange`\n- `IncrementalTextTransformation\u003c*\u003e.afterTextChange`\n- `BigTextDecorator.beforeTextChange`\n- `BigTextDecorator.afterTextChange`\n- `BigTextDecorator.onApplyDecorationOnOriginal`\n- `BigTextDecorator.onApplyDecorationOnTransformation`\n\nDecorator allows large amount of changing dense styles that do not change the text layout or character width. It is especially designed for syntax highlighting. Different from incremental transformation, decorator transforms styles just before they are rendered, only transforms text that within the viewport, and the transformation result is not persisted.\n\nSee [GraphqlSyntaxHighlightDecorator](https://www.github.com/sunny-chung/hello-http/tree/main/src/jvmMain/kotlin/com/sunnychung/application/multiplatform/hellohttp/ux/transformation/incremental/GraphqlSyntaxHighlightDecorator.kt) is an example to use an incremental parser to handle input events, and use `BigTextDecorator` to apply syntax highlighting styles.\n\n### Transformation Offset Mapping\n\nBigText offers two types of offset mapping:\n- Block -- the cursor never goes _into_ the transformation\n- Incremental (it is a different thing to Incremental Transformation) -- the cursor can navigate through the transformation as if it is a normal text, until `min(length of original subsequence, length of transformed subsequence)` has been reached\n\nDifferent offset mapping can be mixed and applied to the same text. It depends on the transformation operations.\n\nFor inserts, it is always block-offset transformations. For each replacement, you specify which offset mapping type to take:\n\n```kotlin\noverride fun afterTextChange(change: BigTextChangeEvent, transformer: BigTextTransformer, context: Unit) {\n    // ...\n    transformer.replace(\n        range = matchText.range,\n        text = AnnotatedString(matchText.value, spanStyle),\n        offsetMapping = BigTextTransformOffsetMapping.Incremental\n    )\n    transformer.replace(\n        range = matchText.range.endInclusive + 1 .. matchText.range.endInclusive + 5,\n        text = AnnotatedString(\"0123456789\"),\n        offsetMapping = BigTextTransformOffsetMapping.Block\n    )\n}\n```\n\n### Modify the BigText value\n\nModify BigText directly:\n\n```kotlin\nval bigTextFieldState: BigTextFieldState by rememberConcurrentLargeAnnotatedBigTextFieldState(\"\")\nval text = bigTextFieldState.text\n\nButton(\n    onClick = {\n        val selection = bigTextFieldState.viewState.selection\n        text.insertAt(selection.last + 1, \"}}\")\n        text.insertAt(selection.first, \"\\${{\")\n        text.recordCurrentChangeSequenceIntoUndoHistory() // take a snapshot for undo/redo\n    }\n) {\n    Text(\"Transform\")\n}\n\nBigTextField(\n    textFieldState = bigTextFieldState,\n    // ...\n)\n```\n\nTo move the cursor along with the input:\n\n```kotlin\nfun onPressEnterAddIndent(textState: BigTextFieldState) {\n    val newSpaces = \"\\n    \"\n    textState.replaceTextAtCursor(newSpaces)\n}\n```\n\n### Keep Using String for Compatibility\n\nIf you don't need to deal with big text and want to keep using String, you must use a unique cache key for each content with BigTextField. For example:\n\n```kotlin\n@Composable\nfun MyTextField(initialValue: String, onValueChange: (String) -\u003e Unit, recordType: RecordType, recordId: Long) {\n    val textState by rememberConcurrentLargeAnnotatedBigTextFieldState(initialValue, recordType, recordId /*, any other cache keys*/)\n\n    BigTextField(\n        textFieldState = textState,\n        onTextChange = {\n            onValueChange(it.bigText.buildString())\n        },\n        // ...\n    )\n}\n```\n\nThe initialValue is used only in the first recomposition where cache keys are changed. It is then ignored until cache keys change.\n\n### Requesting focus (a temporary workaround available since v2.0.1)\n\nTo request focus before BigTextField is ready,\n\n```kotlin\nval focusRequester = remember { FocusRequester() }\n\nCoreBigTextField(\n    // ...\n    onFinishInit = { focusRequester.requestFocus() }\n)\n```\n\nIf `requestFocus()` is invoked after BigTextField is ready, it would work without this workaround.\n\n### Reducing Latency while Using Many BigTexts or BigTextFields\n\nIt may experience significant delays (0.6+ seconds) while rendering 10+ BigTextField. It is possible to improve (not eliminate) the latency by configuring BigTextField to process synchronously.\n\n```kotlin\nBigTextField(\n    // ...\n    onHeavyComputation = AsyncOperation.Synchronous\n)\n```\n\n\n## FAQ\n\nQ: Can it be used for general-purpose small text?\n\nA: Sure, and use small or tiny text buffer to save memory.\n\n```kotlin\nval textState = rememberSaveable(cacheKeys) {\n    mutableStateOf(\n        BigTextFieldState(\n            createFromTinyAnnotatedString(AnnotatedString(initialValue)),\n            BigTextViewState()\n        )\n    )\n}\n```\n\n\nQ: My text is small, and my application use String everywhere. Can BigTextField be used with String instead of BigText?\n\nA: Yes. Refer to [Keep Using String for Compatibility](#keep-using-string-for-compatibility).\n\n\nQ: The cursor goes into middle of a character when multiple fonts are used for the same text field! Is it a bug?\n\nA: Only text value and text transformation support multiple fonts, not decorators. If it is a must to use decorator, use together with transformation -- transform it to a desired font first, then decorate it.\n\n\n## More References\n\nThe previous version of `BigTextField`, `BigMonospaceTextField`, is already in production use by the [Hello HTTP](https://github.com/sunny-chung/hello-http) software. There are also some incremental transformation/decorator examples. This project is a good production example for reference.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunny-chung%2Fbigtext","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsunny-chung%2Fbigtext","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunny-chung%2Fbigtext/lists"}