{"id":13803736,"url":"https://github.com/davidmoten/rxjava-extras","last_synced_at":"2025-05-14T19:09:14.582Z","repository":{"id":23905458,"uuid":"27285466","full_name":"davidmoten/rxjava-extras","owner":"davidmoten","description":"Utilities for use with rxjava ","archived":false,"fork":false,"pushed_at":"2025-04-03T16:40:26.000Z","size":2581,"stargazers_count":268,"open_issues_count":12,"forks_count":29,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-13T15:07:45.395Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/davidmoten.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-11-28T23:25:14.000Z","updated_at":"2025-04-03T16:40:24.000Z","dependencies_parsed_at":"2023-11-21T17:31:12.829Z","dependency_job_id":"171448f5-0313-44c2-ad03-02afa958f038","html_url":"https://github.com/davidmoten/rxjava-extras","commit_stats":{"total_commits":1470,"total_committers":10,"mean_commits":147.0,"dds":"0.055102040816326525","last_synced_commit":"df168ed47ece0c202547169856b31fb7b57d11e2"},"previous_names":[],"tags_count":94,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Frxjava-extras","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Frxjava-extras/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Frxjava-extras/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/davidmoten%2Frxjava-extras/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/davidmoten","download_url":"https://codeload.github.com/davidmoten/rxjava-extras/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254209859,"owners_count":22032897,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T01:00:37.414Z","updated_at":"2025-05-14T19:09:12.764Z","avatar_url":"https://github.com/davidmoten.png","language":"Java","readme":"rxjava-extras\n=============\n\n\u003ca href=\"https://travis-ci.org/davidmoten/rxjava-extras\"\u003e\u003cimg src=\"https://travis-ci.org/davidmoten/rxjava-extras.svg\"/\u003e\u003c/a\u003e\u003cbr/\u003e\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/com.github.davidmoten/rxjava-extras/badge.svg?style=flat)](https://maven-badges.herokuapp.com/maven-central/com.github.davidmoten/rxjava-extras)\u003cbr/\u003e\n[![codecov](https://codecov.io/gh/davidmoten/rxjava-extras/branch/master/graph/badge.svg)](https://codecov.io/gh/davidmoten/rxjava-extras)\n\nUtilities for use with RxJava 1 (see [rxjava2-extras](https://github.com/davidmoten/rxjava2-extras) for RxJava 2):\n\n* `Functions.identity, alwaysTrue, alwaysFalse, constant, not`\n* `Actions.setAtomic, doNothing, unsubscribe, increment, decrement, addTo, println, setToTrue, countDown, printStackTrace`\n* [`Checked`](#checked) provides lambda helpers for dealing with checked exceptions in functions and actions\n* [`TestingHelper.test`](#testinghelpertest)\n* [`RetryWhen`](#retrywhen) builder for use with `.retryWhen(Func1)` operator\n* [`Transformers.ignoreElementsThen`](#transformersignoreelementsthen)\n* [`Transformers.mapWithIndex`](#transformersmapwithindex)\n* [`Transformers.matchWith`](#transformersmatchwith)\n* [`Transformers.orderedMergeWith`](#transformersorderedmergewith)\n* [`Transformers.stateMachine`](#transformersstatemachine)\n* [`Transformers.collectWhile`](#transformerscollectwhile)\n* [`Transformers.toListWhile`](#transformerstolistwhile)\n* [`Transformers.toListUntilChanged`](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#toListUntilChanged--)\n* [`Transformers.toListUntil`](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#toListUntil-rx.functions.Func1-)\n* [`Transformers.collectStats`](#transformerscollectstats) \n* [`Transformers.doOnFirst`](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#doOnFirst-rx.functions.Action1-)\n* `Transformers.doOnNth`\n* [`Transformers.onBackpressureBufferToFile`](#transformersonbackpressurebuffertofile) - buffer items to disk \n* [`Transformers.toOperator`](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#toOperator-rx.functions.Func1-)\n* `Transformers.windowMin`, `.windowMax`\n* `Transformers.sampleFirst`\n* `Transformers.decode`\n* `Transformers.delayFinalUnsubscribe` - to keep a source active for a period after last unsubscribe (useful with `refCount`/`share`)\n* [`Transformers.repeatLast`](#transformersrepeatlast)\n* [`Transformers.doOnEmpty`](#transformersdoonempty) \n* [`Serialized.read/write`](#serialized)\n* [`Bytes.from`](#bytesfrom) - read bytes from resources (`InputStream`, `File`)\n* [`Bytes.unzip`](#bytesunzip) - unzips zip archives\n* [`Bytes.collect`](#bytescollect) - collect bytes into single byte array\n* `Strings.from`\n* `Strings.lines` - supports backpressure (not available in rxjava-string 1.0.1)\n* `Strings.split` - supports backpressure (not available in rxjava-string 1.0.1)\n* `PublishSubjectSingleSubscriber`\n* `OperatorUnsubscribeEagerly`\n* [`TestingHelper`](#testinghelper)\n\n\nStatus: *released to Maven Central*\n\nMaven site reports are [here](http://davidmoten.github.io/rxjava-extras/index.html) including [javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/index.html).\n\nGetting started\n-----------------\nAdd this to your pom.xml:\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.github.davidmoten\u003c/groupId\u003e\n  \u003cartifactId\u003erxjava-extras\u003c/artifactId\u003e\n  \u003cversion\u003eVERSION_HERE\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nOr add this to your build.gradle:\n```groovy\nrepositories {\n    mavenCentral()\n}\n\ndependencies {\n    compile 'com.github.davidmoten:rxjava-extras:VERSION_HERE'\n}\n```\n\nTestingHelper.test\n----------------------\nUse method chaining in your tests (inspired by rxjava 2.x functionality):\n\n```java\nObservable.range(1, 1000)\n  .count()\n  .to(TestingHelper.test())\n  .assertValue(1000)\n  .assertCompleted();\n```\n\nSet the initial request like this:\n\n```java\nObservable.range(1, 1000)\n  .to(TestingHelper.testWithRequest(2))\n  .assertValues(1, 2)\n  .assertNoTerminalEvent();\n```\n\nTransformers.ignoreElementsThen\n------------------------------------\nTo ignore the elements of an observable (but wait for completion) and then emit a second observable, use `Transformers.ignoreElementsThen`:\n\n\u003cimg src=\"src/docs/ignoreElementsThen.png?raw=true\"/\u003e\n\nTransformers.mapWithIndex\n-------------------------\nMaps each item to an item wrapped with a zero-based index:\n\n\u003cimg src=\"src/docs/mapWithIndex.png?raw=true\" /\u003e\n\n[javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#mapWithIndex--)\n\nTransformers.matchWith\n-------------------------\nFinds out-of-order matches in two streams.\n\n\u003cimg src=\"src/docs/match.png?raw=true\" /\u003e\n\n[javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#matchWith--)\n\nYou can use `Tranformers.matchWith` or `Obs.match`:\n\n```java\nObservable\u003cInteger\u003e a = Observable.just(1, 2, 4, 3);\nObservable\u003cInteger\u003e b = Observable.just(1, 2, 3, 5, 6, 4);\nObs.match(a, b,\n     x -\u003e x, // key to match on for a\n     x -\u003e x, // key to match on for b\n     (x, y) -\u003e x // combiner\n    )\n   .forEach(System.out::println);\n```\ngives\n```\n1\n2\n3\n4\n```\nDon't rely on the output order!\n\nUnder the covers elements are requested from `a` and `b` in alternating batches of 128 by default. The batch size is configurable in another overload.\n\nTransformers.orderedMergeWith\n------------------------------\nTo merge two (or more) streams in order (according to a `Comparator`):\n\n```java\nsource1.compose(Transformers.orderedMergeWith(source2, comparator));\n```\n\n\u003cimg src=\"src/docs/orderedMerge.png?raw=true\" /\u003e\n\nTo merge with many:\n\n```java\nsource1.compose(Transformers.orderedMergeWith(Arrays.asList(source2, source3), comparator));\n```\n\n[javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#orderedMergeWith-rx.Observable-rx.functions.Func2-)\n\nTransformers.toListWhile\n---------------------------\nYou may want to group emissions from an Observable into lists of variable size. This can be achieved safely using `toListWhile`.\n\n\u003cimg src=\"src/docs/toListWhile.png?raw=true\" /\u003e\n\n[javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#toListWhile-rx.functions.Func2-)\n\nAs an example from a sequence of temperatures lets group the sub-zero and zero or above temperatures into contiguous lists:\n\n```java\nObservable.just(10, 5, 2, -1, -2, -5, -1, 2, 5, 6)\n    .compose(Transformers.toListWhile( \n        (list, t) -\u003e list.isEmpty() \n            || Math.signum(list.get(0)) \u003c 0 \u0026\u0026 Math.signum(t) \u003c 0\n            || Math.signum(list.get(0)) \u003e= 0 \u0026\u0026 Math.signum(t) \u003e= 0)\n    .forEach(System.out::println);\n```\nproduces\n```\n[10, 5, 2]\n[-1, -2, -5, -1]\n[2, 5, 6]\n```\n\nTransformers.collectWhile\n-------------------------\nBehaves as per `toListWhile` but allows control over the data structure used. \n\n\u003cimg src=\"src/docs/collectWhile.png?raw=true\" /\u003e\n\n[javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#collectWhile-rx.functions.Func0-rx.functions.Action2-rx.functions.Func2-)\n\nTransformers.collectStats\n---------------------------\nAccumulate statistics, emitting the accumulated results with each item.\n\n\u003cimg src=\"src/docs/collectStats.png?raw=true\" /\u003e\n\nTransformers.repeatLast\n------------------------\nIf a stream has elements and completes then the last element is repeated.\n\n\u003cimg src=\"src/docs/repeatLast.png?raw=true\" /\u003e \n\n```java\nsource.compose(Transformers.repeatLast());\n```\n\nTransformers.doOnEmpty\n-------------------------\nPerforms an action only if a stream completes without emitting an item.\n\n```java\nsource.compose(Transformers.doOnEmpty(action));\n```\n\nTransformers.stateMachine\n--------------------------\nCustom operators are difficult things to get right in RxJava mainly because of the complexity of supporting backpressure efficiently. `Transformers.stateMachine` enables a custom operator implementation when:\n\n* each source emission is mapped to 0 to many emissions (of a different type perhaps) to downstream but those emissions are calculated based on accumulated state\n\n\u003cimg src=\"src/docs/stateMachine.png?raw=true\" /\u003e\n\n[javadoc](http://davidmoten.github.io/rxjava-extras/apidocs/com/github/davidmoten/rx/Transformers.html#stateMachine-rx.functions.Func0-rx.functions.Func3-rx.functions.Action2-)\n\nAn example of such a transformation might be from a list of temperatures you only want to emit sequential values that are less than zero but are part of a sub-zero sequence at least 1 hour in duration. You could use `toListWhile` above but `Transformers.stateMachine` offers the additional efficiency that it will immediately emit temperatures as soon as the duration criterion is met. \n\nTo implement this example, suppose the source is half-hourly temperature measurements:\n\n```java\nstatic class State {\n     final List\u003cDouble\u003e list;\n     final boolean reachedThreshold;\n     State(List\u003cDouble\u003e list, boolean reachedThreshold) {\n         this.list = list; \n         this.reachedThreshold = reachedThreshold;\n     }\n}\n\nint MIN_SEQUENCE_LENGTH = 2;\n\nTransformer\u003cDouble, Double\u003e trans = Transformers \n    .stateMachine() \n    .initialStateFactory(() -\u003e new State(new ArrayList\u003c\u003e(), false))\n    .\u003cDouble, Double\u003e transition((state, t, subscriber) -\u003e {\n        if (t \u003c 0) {\n            if (state.reachedThreshold) {\n                if (subscriber.isUnsubscribed()) {\n                    return null;\n                }\n                subscriber.onNext(t);\n                return state;\n            } else if (state.list.size() == MIN_SEQUENCE_LENGTH - 1) {\n                for (Double temperature : state.list) {\n                    if (subscriber.isUnsubscribed()) {\n                        return null;\n                    }\n                    subscriber.onNext(temperature);\n                }\n                return new State(null, true);\n            } else {\n                List\u003cDouble\u003e list = new ArrayList\u003c\u003e(state.list);\n                list.add(t);\n                return new State(list, false);\n            }\n        } else {\n            return new State(new ArrayList\u003c\u003e(), false);\n        }\n    }).build();\nObservable\n    .just(10.4, 5.0, 2.0, -1.0, -2.0, -5.0, -1.0, 2.0, 5.0, 6.0)\n    .compose(trans)\n    .forEach(System.out::println);\n```\n\nRetryWhen\n----------------------\nA common use case for `.retry()` is some sequence of actions that are attempted and then after a delay a retry is attempted. RxJava does not provide \nfirst class support for this use case but the building blocks are there with the `.retryWhen()` method. `RetryWhen` offers static methods that build a `Func1` for use with `Observable.retryWhen()`.\n\n\u003cimg src=\"http://reactivex.io/documentation/operators/images/retry.C.png\" width=\"500\"/\u003e\n\n### Retry after a constant delay\n\n```java\nobservable.retryWhen(\n    RetryWhen.delay(10, TimeUnit.SECONDS).build());\n```\n\n### Retry after a constant delay with a maximum number of retries\n\n```java\nobservable.retryWhen(\n    RetryWhen.delay(10, TimeUnit.SECONDS)\n        .maxRetries(10).build());\n```\n\n### Retry after custom delays\n\n```java\n//the length of waits determines number of retries\nObservable\u003cLong\u003e delays = Observable.just(10L,20L,30L,30L,30L);\nobservable.retryWhen(\n    RetryWhen.delays(delays, TimeUnit.SECONDS).build());\n```\n\n### Retry with exponential backoff\n\n```java\n//the length of waits determines number of retries\nobservable.retryWhen(\n    RetryWhen.exponentialBackoff(delay, TimeUnit.SECONDS).build());\n```\n\nYou can cap the delay:\n```java\n//the length of waits determines number of retries\nobservable.retryWhen(\n    RetryWhen.exponentialBackoff(delay, maxDelay, TimeUnit.SECONDS).build());\n```\n\nThe default backoff factor is 2 (the delay doubles on each consecutive failure). You can customize this value with another overload:\n```java\n//the length of waits determines number of retries\nobservable.retryWhen(\n    RetryWhen.exponentialBackoff(delay, maxDelay, TimeUnit.SECONDS, 1.5).build());\n```\n\n### Retry only for a particular exception\n\n```java\nobservable.retryWhen(\n    RetryWhen.retryWhenInstanceOf(IOException.class)\n        .build());\n```\n\nTransformers.onBackpressureBufferToFile\n----------------------------------------\nAs of 0.7.2, you can offload an observable's emissions to disk to reduce memory pressure when you have a fast producer + slow consumer (or just to minimize memory usage).\n\n\u003cimg src=\"src/docs/onBackpressureBufferToFile.png?raw=true\" /\u003e\n\nIf you use the `onBackpressureBuffer` operator you'll know that when a stream is producing faster than the downstream operators can process (perhaps the producer cannot respond meaningfully to a *slow down* request from downstream) then `onBackpressureBuffer` buffers the items to an in-memory queue until they can be processed. Of course if memory is limited then some streams might eventually cause an `OutOfMemoryError`. One solution to this problem is to increase the effectively available memory for buffering by using disk instead (and small in-memory read/write buffers). That's why `Transformers.onBackpressureBufferToFile` was created. \n\nInternally one queue corresponds to a sequence of files (each with \u003c=M entries). A read position and a write position for the file are maintained and relate to byte positions in files in the sequence. Naturally enough the write position will always be \u003e= the read position. As the read position advances past the end of a file to the next file in the sequence the previous file is closed and deleted. Any files between the read position and the write position are closed (not referenced by open file descriptors) and are opened when the read position advances to it. The read position can advance beyond the last position in the last file into the write buffer (but will always be before or at the write position).\n\n\u003cimg src=\"src/docs/file-queue.png?raw=true\" /\u003e\n\nNote that new files for a file buffered observable are created for each subscription and thoses files are in normal circumstances deleted on unsubscription (triggered by `onCompleted`/`onError` termination or manual unsubscription). \n\nHere's an example:\n\n```java\n// write the source strings to a \n// disk-backed queue on the subscription\n// thread and emit the items read from \n// the queue on the computation() scheduler.\nObservable\u003cString\u003e source = \n  Observable\n    .just(\"a\", \"b\", \"c\")\n    .compose(\n      Transformers.onBackpressureBufferToFile(\n          DataSerializers.string(),\n          Schedulers.computation()));\n```\n\nThis example does the same as above but more concisely and uses standard java IO serialization (normally it will be more efficient to write your own `DataSerializer`):\n\n```java\nObservable\u003cString\u003e source = \n  Observable\n    .just(\"a\", \"b\", \"c\")\n    .compose(Transformers.\u003cString\u003eonBackpressureBufferToFile());\n```\n\nAn example with a custom serializer:\n\n```java\n// define how the items in the source stream would be serialized\nDataSerializer\u003cString\u003e serializer = new DataSerializer\u003cString\u003e() {\n\n    @Override\n    public void serialize(DataOutput output, String s) throws IOException {\n        output.writeUTF(s);\n    }\n\n    @Override\n    public String deserialize(DataInput input, int availableBytes) throws IOException {\n        return input.readUTF();\n    }\n};\nObservable\n  .just(\"a\", \"b\", \"c\")\n  .compose(\n    Transformers.onBackpressureBufferToFile(\n        serializer, Schedulers.computation()))\n  ...\n```\nYou can configure various options:\n\n```java\nObservable\n  .just(\"a\", \"b\", \"c\")\n  .compose(\n    Transformers.onBackpressureBufferToFile(\n        serializer, \n        Schedulers.computation(), \n        Options\n          .fileFactory(fileFactory)\n          .bufferSizeBytes(1024)\n          .rolloverEvery(10000)\n          .rolloverSizeBytes(10000000)\n          .delayError(false)\n          .build()))\n  ...\n```\n`Options.fileFactory(Func0\u003cFile\u003e)` specifies the method used to create the root temporary file used by the queue storage mechanism (MapDB). The default is a factory that calls `Files.createTempFile(\"bufferToFileDB\", \"\")`.\n\nRollover (via `Options.rolloverEvery(long)` and/or `Options.rolloverSizeBytes(long)`) is an important option for long running/infinite streams.  The strategy used to reclaim disk space is to create a new file based queue every N emissions and/or on the file size reaching a threshold. Writing will occur to the latest created queue and reading will be occuring on the earliest non-closed queue. Once a queue instance is read fully and it is not the last queue it is closed and its file resources deleted. The abstraction used internally to handle these operations is [`RollingQueue`](src/main/java/com/github/davidmoten/rx/internal/operators/RollingSPSCQueue.java). \n\n* If you have a long running stream (or just a lot of data going through in terms of MB) then **be sure to specify a value for `rolloverEvery` or `rolloverSizeBytes`**\n\nThere are some inbuilt `DataSerializer` implementations:\n\n* `DataSerializers.string()`\n* `DataSerializers.integer()`\n* `DataSerializers.byteArray()`\n* `DataSerializers.javaIO()` - uses standard java serialization (`ObjectOutputStream` and such)\n\nUsing default java serialization you can buffer array lists of integers to a file like so:\n\n```java\nObservable.just(1, 2, 3, 4)\n    //accumulate into sublists of length 2\n    .buffer(2)\n    .compose(\n      Transformers.\u003cList\u003cInteger\u003e\u003eonBackpressureBufferToFile())\n```\n\nIn the above example it's fortunate that `.buffer` emits `ArrayList\u003cInteger\u003e` instances which are serializable. To be strict you might want to `.map` the returned list to a data type you know is serializable:\n\n```java\nObservable.just(1, 2, 3, 4)\n    .buffer(2)\n    .map(list -\u003e new ArrayList\u003cInteger\u003e(list))\n    .compose(\n      Transformers.\u003cList\u003cInteger\u003e\u003eonBackpressureBufferToFile())\n```\n\n### Performance\nThroughput writing to spinning disk (and reading straight away with little downstream processing cost) on an i7 with `Options.bufferSizeBytes=1024`:\n\n```\nrate = 42.7MB/s (1K messages, no rollover, write only)\nrate = 42.2MB/s (1K messages, rollover, write only)\nrate = 37.4MB/s (1K messages, no rollover)\nrate = 37.3MB/s (1K messages, rollover)\nrate = 15.2MB/s (4B messages, no rollover)\nrate = 9.3MB/s (4B messages, rollover)\n```\n\nI wouldn't be surprised to see significant improvement on some of these benchmarks in the medium term (perhaps using memory-mapped files). There are at least a couple of java file based implementations out there that have impressive throughput using memory-mapped files.\n\nChecked\n------------------\n\nChecked exceptions can be annoying. If you are happy to wrap a checked exception with a ```RuntimeException``` then the function and action helpers in ```Checked``` are great:\n\nInstead of \n```java\nOutputStream os =  ...;\nObservable\u003cString\u003e source = ...;\nsource.doOnNext(s -\u003e {\n\t    try {\n\t        os.write(s.getBytes());\n\t    } catch (IOException e) {\n\t        throw new RuntimeException(e);\n\t    }\n    })\n    .subscribe();\n```\n\nyou can write:\n```java\nsource.doOnNext(Checked.a1(s -\u003e os.write(s.getBytes())))\n      .subscribe();\n```\n\nSerialized\n------------------\nTo read serialized objects from a file:\n\n```java\nObservable\u003cItem\u003e items = Serialized.read(file);\n```\n\nTo write an Observable to a file:\n\n```java\nSerialized.write(observable, file).subscribe();\n```\n\n### Kryo\n`Serialized` also has support for the very fast serialization library [kryo](https://github.com/EsotericSoftware/kryo). Unlike standard Java serialization *Kryo* can also serialize/deserialize objects that don't implement `Serializable`. \n\nAdd this to your pom.xml:\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.esotericsoftware\u003c/groupId\u003e\n    \u003cartifactId\u003ekryo\u003c/artifactId\u003e\n    \u003cversion\u003e3.0.3\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\nFor example,\n\nTo read:\n```java\nObservable\u003cItem\u003e items = Serialized.kryo().read(file);\n```\n\nTo write:\n```java\nSerialized.write(observable, file).subscribe();\n```\n\nYou can also call `Serialized.kryo(kryo)` to use an instance of `Kryo` that you have configured specially. \n\nBytes.from\n-------------------\nTo read bytes from an `InputStream` in chunks:\n\n```java\nObservable\u003cbyte[]\u003e chunks = Bytes.from(inputStream, chunkSize);\n```\n\nTo read bytes from a `File` in chunks:\n\n```java\nObservable\u003cbyte[]\u003e chunks = Bytes.from(file, chunkSize);\n``` \nBytes.unzip\n-----------------------\nSuppose you have a a zip file `file.zip` and you want to stream the lines of the file `doc.txt` extracted from the archive:\n\n```java\nObservable\u003cString\u003e lines = \n    Bytes.unzip(new File(\"file.zip\"))\n       .filter(entry -\u003e entry.getName().equals(\"doc.txt\"))\n       .concatMap(entry -\u003e Strings.from(entry.getInputStream()))\n       .compose(o-\u003e Strings.split(o, \"\\n\"));\n```\nNote that above you don't need to worry about closing `entry.getInputStream()` because it is handled in the unsubscribe of the `Bytes.unzip` source.\n\nYou must process the emissions of `ZippedEntry` synchronously (don't replace the `concatMap()` with a `flatMap(...  .subscribeOn(Schedulers.computation())` for instance. This is because the `InputStream` of each `ZippedEntry` must be processed fullly (which could mean ignoring it of course) before moving on to the next one.\n\nBytes.collect\n---------------------------\nGiven a stream of byte arrays this is an easy way of collecting those bytes into one byte array:\n\n```java\nObservable\u003cbyte[]\u003e chunks = ...\nbyte[] allBytes = chunks.compose(Bytes::collect).toBlocking().single();\n```\n\nTestingHelper\n-----------------\nFor a given named test the following variations  are tested:\n\n* without backpressure\n* intiial request maximum, no further request \n* initial request maximum, keep requesting single \n* backpressure, initial request 1, then by 0 and 1 \n* backpressure, initial request 1, then by 1 \n* backpressure, initial request 2, then by 2 \n* backpressure, initial request 5, then by 5 \n* backpressure, initial request 100, then by 100 \n* backpressure, initial request 1000, then by 1000 \n* backpressure, initial request 2, then Long.MAX_VALUE-1 (checks for request overflow)\n\nNote that the above list no longer contains a check for negative request because that situation is covered by ```Subscriber.request``` throwing an ```IllegalArgumentException```.\n\nFor each variation the following aspects are tested:\n\n* expected *onNext* items received\n* unsubscribe from source occurs (for completion, error or explicit downstream unsubscription (optional))\n* unsubscribe from downstream subscriber occurs\n* ```onCompleted``` called (if unsubscribe not requested before completion and no errors expected)\n* if ```onCompleted``` expected is only called once\n* ```onError``` not called unless error expected\n* if error expected ```onCompleted``` not called after ```onError```\n* should not deliver more than requested\n\nAn example that tests all of the above variations and aspects for the ```Observable.count()``` method:\n\n```java\nimport junit.framework.TestCase;\nimport junit.framework.TestSuite;\nimport rx.Observable;\n\nimport com.github.davidmoten.rx.testing.TestingHelper;\n\npublic class CountTest extends TestCase {\n\n    public static TestSuite suite() {\n\n        return TestingHelper\n                .function(o -\u003e o.count())\n                // test empty\n                .name(\"testCountOfEmptyReturnsZero\")\n                .fromEmpty()\n                .expect(0)\n                // test error\n                .name(\"testCountErrorReturnsError\")\n                .fromError()\n                .expectError()\n                // test error after some emission\n                .name(\"testCountErrorAfterTwoEmissionsReturnsError\")\n                .fromErrorAfter(5, 6)\n                .expectError()\n                // test non-empty count\n                .name(\"testCountOfTwoReturnsTwo\")\n                .from(5, 6)\n                .expect(2)\n                // test single input\n                .name(\"testCountOfOneReturnsOne\")\n                .from(5)\n                .expect(1)\n                // count many\n                .name(\"testCountOfManyDoesNotGiveStackOverflow\")\n                .from(Observable.range(1, 1000000))\n                .expect(1000000)\n                // get test suites\n                .testSuite(TestingHelperCountTest.class);\n    }\n\n    public void testDummy() {\n        // just here to fool eclipse\n    }\n\n}\n```\n\nWhen you run ```CountTest``` as a JUnit test in Eclipse you see the test variations described as below:\n\n\u003cimg src=\"src/docs/eclipse-junit.png?raw=true\" /\u003e\n\nAn asynchronous example with ```OperatorMerge``` is below. Note the specific control of the wait times. For synchronous transformations the wait times can be left at their defaults:\n\n```java\npublic class TestingHelperMergeTest extends TestCase {\n\n    private static final Observable\u003cInteger\u003e MERGE_WITH = \n        Observable.from(asList(7, 8, 9)).subscribeOn(Schedulers.computation());\n\n    public static TestSuite suite() {\n\n        return TestingHelper\n                .function(o-\u003eo.mergeWith(MERGE_WITH).subscribeOn(Schedulers.computation())\n                .waitForUnsubscribe(100, TimeUnit.MILLISECONDS)\n                .waitForTerminalEvent(10, TimeUnit.SECONDS)\n                .waitForMoreTerminalEvents(100, TimeUnit.MILLISECONDS)\n                // test empty\n                .name(\"testEmptyWithOtherReturnsOther\")\n                .fromEmpty()\n                .expect(7, 8, 9)\n                // test error\n                .name(\"testMergeErrorReturnsError\")\n                .fromError()\n                .expectError()\n                // test error after items\n                .name(\"testMergeErrorAfter2ReturnsError\")\n                .fromErrorAfter(1, 2)\n                .expectError()\n                // test non-empty count\n                .name(\"testTwoWithOtherReturnsTwoAndOtherInAnyOrder\")\n                .from(1, 2)\n                .expectAnyOrder(1, 7, 8, 9, 2)\n                // test single input\n                .name(\"testOneWithOtherReturnsOneAndOtherInAnyOrder\").from(1)\n                .expectAnyOrder(7, 1, 8, 9)\n                // unsub before completion\n                .name(\"testTwoWithOtherUnsubscribedAfterOneReturnsOneItemOnly\").from(1, 2)\n                .unsubscribeAfter(1).expectSize(1)\n                // get test suites\n                .testSuite(TestingHelperMergeTest.class);\n    }\n\n    public void testDummy() {\n        // just here to fool eclipse\n    }\n}\n```\n\nHow to use OperatorUnsubscribeEagerly\n---------------------------------------\n\n```java \nObservable\u003cT\u003e o;\n\nObservable\u003cT\u003e stream = \n      o.lift(OperatorUnsubscribeEagerly.\u003cT\u003einstance());\n```\n\n","funding_links":[],"categories":["Utilities"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidmoten%2Frxjava-extras","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdavidmoten%2Frxjava-extras","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdavidmoten%2Frxjava-extras/lists"}