https://github.com/sunny-chung/bigtext
A data structure for JVM and `@Composable` text field and label for Jetpack Compose to edit big text (up to 100 MB) efficiently.
https://github.com/sunny-chung/bigtext
jetpack-compose jvm large-text text-editing text-field
Last synced: 6 months ago
JSON representation
A data structure for JVM and `@Composable` text field and label for Jetpack Compose to edit big text (up to 100 MB) efficiently.
- Host: GitHub
- URL: https://github.com/sunny-chung/bigtext
- Owner: sunny-chung
- License: mit
- Created: 2025-01-05T07:03:07.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-04-10T16:43:24.000Z (7 months ago)
- Last Synced: 2025-04-10T16:58:57.209Z (7 months ago)
- Topics: jetpack-compose, jvm, large-text, text-editing, text-field
- Language: Kotlin
- Homepage:
- Size: 1.85 MB
- Stars: 8
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# BigText, BigTextField, BigTextLabel
[](https://jetc.dev/issues/249.html)
**BigText** is an in-memory data structure capable of manipulating large string (tested up to 100 MB) specialized for text editing applications. It is independent of UI frameworks, only supports JVM currently but is planned to go Kotlin Multiplatform. It is possible to implement your own text buffer for BigText to support other CharSequence.
**BigTextField** is a Jetpack Compose text field component utilizing BigText to provide the capability of editing large styleable text, and is designed to fix the known issues and limitations of the existing `TextField`, `BasicTextField` and `BasicTextField2`. It aims to replace `BasicTextField`. It is efficient enough to be used for syntax highlighting. It is available for desktop platforms only, but is planned to support all platforms that supported by Jetpack Compose Multiplatform.
**BigTextLabel**, a Jetpack Compose text component, is the read-only version of BigTextField.
## Comparing with `TextField` / `BasicTextField` / `BasicTextField2`
The goods:
- It does not freeze when a 300K text, styled with text transformation, is rendered -- it renders instantly.
- It does not crash when a 10 MB text is feed in.
- The text processing can be customized to be in background (see demo), so theoretical it can support even larger size.
- You don't have to calculate transformation offset anymore. I calculate it for you, using balanced binary trees, efficiently.
- Text field random bouncing? Unexpected selection? Selected text out of view could not be copied? Input eaten because typing too fast? No more weird bugs.
- You can feed in `AnnotatedString` as the initial value.
- Soft wrap can be turned off.
- `AnnotatedString` can react to mouse events.
- When real bugs are reported, it will be fixed in some days, not unfixed after 200 days.
- Help me to append to this list.
The bads:
- It does not embrace immutability.
- You cannot always access the full string, or it would be as slow as `BasicTextField`.
- You write incremental transformation instead, or it would be as slow as `BasicTextField`.
- You take care the time complexity of your callback handlers, incremental transformations and decorators, or it would be as slow as `BasicTextField`.
- You use `WeakReference` to wrap BigText instances as cache keys to prevent memory leaks.
- Lots of fancy stuffs are not yet supported. See [limitations](#limitations).
## Performance
### Time
In the demo, typing in a 100 MB text reacts fast enough to every keystroke.
The recomposition time of a 10 MB text field was measured to be within 2ms.
### Memory
A 100 MB BigText has an overhead of around 20 MB memory, so it consumes around 120 MB (Latin) or 220 MB (Unicode) memory in JVM initially. The overhead grows according to user inputs in order to support undo/redo and transformations, and is configurable. Note that use of UI framework may bring additional overheads.
Memory usage complexity: O(BigText buffer size) + O(transformed buffer size) + O(undo capacity) + O(redo capacity) + O(node count)
## Limitations
- BigText does not support Compose Multiplatform 1.7
- BigText is entirely living in memory, not supporting lazy I/O loading. It may be difficult to load gigabytes of text.
- UI Integration with Jetpack Compose is currently only available in JVM platforms.
- Multiple line heights are not yet supported. All the rows are assumed to have same height as the given text style.
- Emoji modifiers and sequences are not yet supported (Contribution is welcome!)
- Breaking by word in soft wrap according to Unicode algorithm are not yet available (Contribution is welcome!)
- `requestFocus` does not work properly (see [workaround](#requesting-focus-a-temporary-workaround-available-since-v201))
- Styles (e.g. underline) on space characters do not work properly
- Following parameters in Jetpack Compose BasicTextField/BasicText have no equivalent in this library or easy workaround:
- Text directions
- Paragraphs
- Font features
- Letter spacing
- MutableInteractionSource
## Demo App
`./gradlew :demo-ui-composable:run`
## Getting Started
To use without Jetpack Compose,
```kotlin
implementation("io.github.sunny-chung:bigtext-datastructure:")
```
To use with Jetpack Compose,
```kotlin
implementation("io.github.sunny-chung:bigtext-ui-composable:")
```
## Usage Examples
### Simplest
```kotlin
val bigTextFieldState by rememberConcurrentLargeAnnotatedBigTextFieldState("initial super big string", cacheKey)
val scrollState = rememberScrollState()
Box {
BigTextField(
textFieldState = bigTextFieldState,
color = Color.Black,
cursorColor = Color.Blue,
isSoftWrapEnabled = true,
scrollState = scrollState,
modifier = Modifier.fillMaxSize()
)
VerticalScrollbar(
adapter = rememberScrollbarAdapter(scrollState),
modifier = Modifier.align(Alignment.TopEnd).fillMaxHeight()
)
}
```
### Asynchronous Loading with a Loading Spin
```kotlin
var bigTextFieldState by remember {
mutableStateOf(
BigTextFieldState(
text = ConcurrentBigText(BigText.createFromLargeAnnotatedString(AnnotatedString(""))),
viewState = BigTextViewState()
)
)
}
val scrollState = rememberScrollState()
val horizontalScrollState = rememberScrollState()
var numOfComputations by remember { mutableStateOf(0) }
val coroutineScope = rememberCoroutineScope()
fun loadBigTextInBackground() {
++numOfComputations
coroutineScope.launch {
val textState = withContext(Dispatchers.IO) {
val initialText = "A".repeat(100000000)
BigTextFieldState(
ConcurrentBigText(BigText.createFromLargeAnnotatedString(AnnotatedString(initialText))),
BigTextViewState()
)
}
bigTextFieldState = textState
scrollState.scrollTo(0)
--numOfComputations
}
}
Box {
BigTextField(
textFieldState = bigTextFieldState,
color = Color.Black,
cursorColor = Color.Black,
fontFamily = FontFamily.Serif,
isSoftWrapEnabled = false,
onHeavyComputation = { computation -> // compute in background and display a "loading" spinner
withContext(coroutineScope.coroutineContext) {
++numOfComputations
}
withContext(Dispatchers.IO) {
computation()
}
withContext(coroutineScope.coroutineContext) {
--numOfComputations
}
},
scrollState = scrollState,
horizontalScrollState = horizontalScrollState, // only required for soft wrap disabled
modifier = Modifier.background(Color(224, 224, 224))
.fillMaxSize()
)
if (numOfComputations > 0) {
CircularProgressIndicator(Modifier.align(Alignment.Center))
} else {
VerticalScrollbar(
adapter = rememberScrollbarAdapter(scrollState),
modifier = Modifier.align(Alignment.TopEnd).fillMaxHeight()
)
HorizontalScrollbar(
adapter = rememberScrollbarAdapter(horizontalScrollState),
modifier = Modifier.align(Alignment.BottomStart).fillMaxWidth()
)
}
}
LaunchedEffect(Unit) {
loadBigTextInBackground()
}
```
### Transformation and More
https://github.com/user-attachments/assets/5733ccfc-8c4c-4678-a893-fa22994a1c7b
```kotlin
/**
* Transform an input so that it is displayed in a format of "(xxx) xxxx-xxxx"
*/
class PhoneNumberIncrementalTransformation : IncrementalTextTransformation {
override fun initialize(text: BigText, transformer: BigTextTransformer) {
transform(text, transformer)
}
override fun afterTextChange(change: BigTextChangeEvent, transformer: BigTextTransformer, context: Unit) {
transform(change.bigText, transformer)
}
private fun transform(text: BigText, transformer: BigTextTransformer) {
transformer.restoreToOriginal(0 .. text.length)
if (text.isNotEmpty) {
transformer.insertAt(0, "(")
if (text.length >= 3) transformer.insertAt(3, ") ")
if (text.length >= 3 + 4) transformer.insertAt(7, "-")
}
}
}
```
```kotlin
BigTextField(
textTransformation = remember { PhoneNumberIncrementalTransformation() }, // `remember` is needed to avoid recomputation!
isSingleLineInput = true,
maxInputLength = 3 + 4 + 4,
inputFilter = remember { { it.replace("[^0-9]".toRegex(), "") } },
// ...
)
```
### Incremental Transformation & Decorator
Incremental Transformation means when the text is changed, the transformation only processes and updates the changed portions.
For an example of incremental transformation, see [VariableIncrementalTransformation](demo-ui-composable/src/jvmMain/kotlin/com/sunnychung/lib/multiplatform/bigtext/demo/VariableIncrementalTransformation.kt).
It can be a mess to implement incremental transformation. You may want to look at incremental parsers.
The time complexity of implementations of the following functions should be significantly lower than `O(BigText's length)`. Ideally, `O(lg(BigText's length))`.
- `IncrementalTextTransformation<*>.beforeTextChange`
- `IncrementalTextTransformation<*>.afterTextChange`
- `BigTextDecorator.beforeTextChange`
- `BigTextDecorator.afterTextChange`
- `BigTextDecorator.onApplyDecorationOnOriginal`
- `BigTextDecorator.onApplyDecorationOnTransformation`
Decorator allows large amount of changing dense styles that do not change the text layout or character width. It is especially designed for syntax highlighting. Different from incremental transformation, decorator transforms styles just before they are rendered, only transforms text that within the viewport, and the transformation result is not persisted.
See [GraphqlSyntaxHighlightDecorator](https://www.github.com/sunny-chung/hello-http/tree/main/src/jvmMain/kotlin/com/sunnychung/application/multiplatform/hellohttp/ux/transformation/incremental/GraphqlSyntaxHighlightDecorator.kt) is an example to use an incremental parser to handle input events, and use `BigTextDecorator` to apply syntax highlighting styles.
### Transformation Offset Mapping
BigText offers two types of offset mapping:
- Block -- the cursor never goes _into_ the transformation
- Incremental (it is a different thing to Incremental Transformation) -- the cursor can navigate through the transformation as if it is a normal text, until `min(length of original subsequence, length of transformed subsequence)` has been reached
Different offset mapping can be mixed and applied to the same text. It depends on the transformation operations.
For inserts, it is always block-offset transformations. For each replacement, you specify which offset mapping type to take:
```kotlin
override fun afterTextChange(change: BigTextChangeEvent, transformer: BigTextTransformer, context: Unit) {
// ...
transformer.replace(
range = matchText.range,
text = AnnotatedString(matchText.value, spanStyle),
offsetMapping = BigTextTransformOffsetMapping.Incremental
)
transformer.replace(
range = matchText.range.endInclusive + 1 .. matchText.range.endInclusive + 5,
text = AnnotatedString("0123456789"),
offsetMapping = BigTextTransformOffsetMapping.Block
)
}
```
### Modify the BigText value
Modify BigText directly:
```kotlin
val bigTextFieldState: BigTextFieldState by rememberConcurrentLargeAnnotatedBigTextFieldState("")
val text = bigTextFieldState.text
Button(
onClick = {
val selection = bigTextFieldState.viewState.selection
text.insertAt(selection.last + 1, "}}")
text.insertAt(selection.first, "\${{")
text.recordCurrentChangeSequenceIntoUndoHistory() // take a snapshot for undo/redo
}
) {
Text("Transform")
}
BigTextField(
textFieldState = bigTextFieldState,
// ...
)
```
To move the cursor along with the input:
```kotlin
fun onPressEnterAddIndent(textState: BigTextFieldState) {
val newSpaces = "\n "
textState.replaceTextAtCursor(newSpaces)
}
```
### Keep Using String for Compatibility
If you don't need to deal with big text and want to keep using String, you must use a unique cache key for each content with BigTextField. For example:
```kotlin
@Composable
fun MyTextField(initialValue: String, onValueChange: (String) -> Unit, recordType: RecordType, recordId: Long) {
val textState by rememberConcurrentLargeAnnotatedBigTextFieldState(initialValue, recordType, recordId /*, any other cache keys*/)
BigTextField(
textFieldState = textState,
onTextChange = {
onValueChange(it.bigText.buildString())
},
// ...
)
}
```
The initialValue is used only in the first recomposition where cache keys are changed. It is then ignored until cache keys change.
### Requesting focus (a temporary workaround available since v2.0.1)
To request focus before BigTextField is ready,
```kotlin
val focusRequester = remember { FocusRequester() }
CoreBigTextField(
// ...
onFinishInit = { focusRequester.requestFocus() }
)
```
If `requestFocus()` is invoked after BigTextField is ready, it would work without this workaround.
### Reducing Latency while Using Many BigTexts or BigTextFields
It may experience significant delays (0.6+ seconds) while rendering 10+ BigTextField. It is possible to improve (not eliminate) the latency by configuring BigTextField to process synchronously.
```kotlin
BigTextField(
// ...
onHeavyComputation = AsyncOperation.Synchronous
)
```
## FAQ
Q: Can it be used for general-purpose small text?
A: Sure, and use small or tiny text buffer to save memory.
```kotlin
val textState = rememberSaveable(cacheKeys) {
mutableStateOf(
BigTextFieldState(
createFromTinyAnnotatedString(AnnotatedString(initialValue)),
BigTextViewState()
)
)
}
```
Q: My text is small, and my application use String everywhere. Can BigTextField be used with String instead of BigText?
A: Yes. Refer to [Keep Using String for Compatibility](#keep-using-string-for-compatibility).
Q: The cursor goes into middle of a character when multiple fonts are used for the same text field! Is it a bug?
A: Only text value and text transformation support multiple fonts, not decorators. If it is a must to use decorator, use together with transformation -- transform it to a desired font first, then decorate it.
## More References
The previous version of `BigTextField`, `BigMonospaceTextField`, is already in production use by the [Hello HTTP](https://github.com/sunny-chung/hello-http) software. There are also some incremental transformation/decorator examples. This project is a good production example for reference.