https://github.com/dkuku/ex_text_splitter

Elixir bindings for text_splitter crate
https://github.com/dkuku/ex_text_splitter

Last synced: 17 days ago
JSON representation

Elixir bindings for text_splitter crate

• Host: GitHub
• URL: https://github.com/dkuku/ex_text_splitter
• Owner: dkuku
• Created: 2024-04-15T21:46:12.000Z (9 months ago)
• Default Branch: master
• Last Pushed: 2024-04-16T20:49:32.000Z (9 months ago)
• Last Synced: 2024-12-06T19:39:36.755Z (about 1 month ago)
• Language: Elixir
• Size: 926 KB
• Stars: 1
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# ExTextSplitter

This package provides bindings to [text-splitter](https://github.com/benbrandt/text-splitter) crate

Check the [example livebook](examples/ex_text_splitter.livemd)

By default only the text_splitter function is available but you can configure the available features:

```elixir

# this will enable all features

config :ex_text_splitter,

  features: ["markdown", "tiktoken-rs"]

```

This can be also configured using Mix.installed

```elixir

Mix.install(

    [:ex_text_splitter],

    config: [ex_text_splitter: [features: ["markdown"]]]

)

```

## Installation

If [available in Hex](https://hex.pm/docs/publish), the package can be installed

by adding `ex_text_splitter` to your list of dependencies in `mix.exs`:

```elixir

def deps do

  [

    {:ex_text_splitter, "~> 0.1.0"}

  ]

end

```

Documentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)

and published on [HexDocs](https://hexdocs.pm). Once published, the docs can

be found at .