https://github.com/luismedel/tsid-python
Time-Sorted Unique Identifiers (TSID) for Python
https://github.com/luismedel/tsid-python
id snowflake snowflake-id snowflake-twitter tsid twitter-snowflake uuid uuidv7
Last synced: 7 days ago
JSON representation
Time-Sorted Unique Identifiers (TSID) for Python
- Host: GitHub
- URL: https://github.com/luismedel/tsid-python
- Owner: luismedel
- License: mit
- Created: 2023-04-07T23:40:12.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-07-14T17:07:55.000Z (10 months ago)
- Last Synced: 2025-05-09T05:36:50.762Z (13 days ago)
- Topics: id, snowflake, snowflake-id, snowflake-twitter, tsid, twitter-snowflake, uuid, uuidv7
- Language: Python
- Homepage:
- Size: 55.7 KB
- Stars: 32
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# tsid-python
A Python library for generating Time-Sorted Unique Identifiers (TSID) as defined in .
This library is a port of the original Java code by [Fabio Lima](https://github.com/fabiolimace).
## Installation
```bash
pip install tsidpy
```
## What is a TSID?
The term TSID stands for (roughly) Time-Sorted ID. A TSID is a value that is formed by its creation time along with a random value.
It brings together ideas from [Twitter's Snowflake](https://github.com/twitter-archive/snowflake/tree/snowflake-2010) and [ULID Spec](https://github.com/ulid/spec).
In summary:
- Sorted by generation time.
- Can be stored as a 64-bit integer.
- Can be stored as a 13-char len string.
- String format is encoded to [Crockford's base32](https://www.crockford.com/base32.html).
- String format is URL safe, case insensitive and has no hyphens.
- Shorter than other unique identifiers, like [UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier), [ULID](https://github.com/ulid/spec) and [KSUID](https://github.com/segmentio/ksuid).
## TSID Structure
A TSID has 2 components:
1. A time component (42 bits), consisting in the elapsed milliseconds since `2020-01-01 00:00:00 UTC` (this epoch can be configured)
2. A _random_ component (22 bits), containing 2 sub-parts:
- A node identifier (can use 0 to 20 bits)
- A counter (can use 2 to 22 bits)
> Note: The counter length depends on the node identifier length.
>
> For example, if we use 10 bits for the node representation:
>
> - The counter is limited to 12 bits.
> - The maximum node value is `2^10-1 = 1023`
> - The maximum counter value is `2^12-1 = 4095`, so the maximum TSIDs that can be generated _per millisecond_ is `4096`.
This is the default TSID structure:
```text
adjustable
<---------->
|------------------------------------------|----------|------------|
time (msecs since 2020-01-01) node counter
42 bits 10 bits 12 bits
- time: 2^42 = ~69 to ~139 years with adjustable epoch (see notes below)
- node: up to 2^20 values with adjustable bits.
- counter: 2^2..2^22 with adjustable bits and randomized values every millisecond.
```
> Notes:
>
> - The time component can be used for ~69 years if stored in a `SIGNED 64-bit` integer field (41 usable bits) or ~139 years if stored in a `UNSIGNED 64-bit` integer field (42 usable bits).
> - By default, new TSID generators use 10 bits for the node identifier and 12 bits to the counter. It's possible to adjust the node identifier length to a value between 0 and 20.
> - The time component can be 1 ms or more ahead of the system time when necessary to maintain monotonicity and generation speed.
### Node identifier
The simplest way to avoid collisions is to make sure that each generator has an exclusive node ID.
The node ID can be passed to the `TSIDGenerator` constructor. If no node ID is passed, the generator will use a random value.
### Recommended readings
- [The best UUID type for a database Primary Key](https://vladmihalcea.com/uuid-database-primary-key/)
- [The primary key dilemma: ID vs UUID and some practical solutions](https://fillumina.wordpress.com/2023/02/06/the-primary-key-dilemma-id-vs-uuid-and-some-practical-solutions/)
- [Primary keys in the DB - what to use? ID vs UUID or is there something else?](https://www.linkedin.com/pulse/primary-keys-db-what-use-id-vs-uuid-something-else-lucas-persson)
Related with the [original library](https://github.com/f4b6a3/tsid-creator):
- [FAQ wiki page](https://github.com/f4b6a3/tsid-creator/wiki)
- [Javadocs](https://javadoc.io/doc/com.github.f4b6a3/tsid-creator)
- [How to not use TSID factories](https://fillumina.wordpress.com/2023/01/19/how-to-not-use-tsid-factories/)
- [The best way to generate a TSID entity identifier with JPA and Hibernate](https://vladmihalcea.com/tsid-identifier-jpa-hibernate/)
## Basic usage
Create a TSID:
```python
from tsidpy import TSID
tsid: TSID = TSID.create()
```
Create a TSID as an `int`:
```python
>>> TSID.create().number
432511671823499267
```
Create a TSID as a `str`:
```python
>>> str(TSID.create())
'0C04Q2BR40003'
```
Create a TSID as an hexadecimal `str`:
```python
>>> TSID.create().to_string('x')
'06009712f0400003'
```
> Note: TSID generators are [thread-safe](https://en.wikipedia.org/wiki/Thread_safety).
### TSID as int
The `TSID::number` property simply unwraps the internal `int` value of a TSID.
```python
>>> from tsidpy import TSID
>>> TSID.create(432511671823499267).number
432511671823499267
```
Sequence of TSIDs:
```text
38352658567418867
38352658567418868
38352658567418869
38352658567418870
38352658567418871
38352658567418872
38352658567418873
38352658567418874
38352658573940759 < millisecond changed
38352658573940760
38352658573940761
38352658573940762
38352658573940763
38352658573940764
38352658573940765
38352658573940766
^ ^ look
|--------|------|
time random
```
### TSID as str
The `TSID::to_string()` method encodes a TSID as a [Crockford's base 32](https://www.crockford.com/base32.html) string. The returned string is 13 characters long.
```python
>>> from tsidpy import TSID
>>> tsid: str = TSID.create().to_string()
'0C04Q2BR40004'
```
Or, alternatively:
```python
>>> tsid: str = str(TSID.create())
'0C04Q2BR40004'
```
Sequence of TSID strings:
```text
01226N0640J7K
01226N0640J7M
01226N0640J7N
01226N0640J7P
01226N0640J7Q
01226N0640J7R
01226N0640J7S
01226N0640J7T
01226N0693HDA < millisecond changed
01226N0693HDB
01226N0693HDC
01226N0693HDD
01226N0693HDE
01226N0693HDF
01226N0693HDG
01226N0693HDH
^ ^ look
|-------|---|
time random
```
The string format can be useful for languages that store numbers in [IEEE 754 double-precision binary floating-point format](https://en.wikipedia.org/wiki/Double-precision_floating-point_format), such as [Javascript](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number).
### More Examples
Create a TSID using the default generator:
```python
from tsidpy import TSID
tsid: TSID = TSID.create()
```
---
Create a TSID from a canonical string (13 chars):
```python
from tsidpy import TSID
tsid: TSID = TSID.from_string('0123456789ABC')
```
---
Convert a TSID into a canonical string in lower case:
```python
>>> tsid.to_string('s')
'0123456789abc'
```
---
Get the creation `timestamp` of a TSID:
```python
>>> tsid.timestamp
1680948418241.0 # datetime.datetime(2023, 4, 8, 12, 6, 58, 241000)
```
---
Encode a TSID to base-62:
```python
>>> tsid.to_string('z')
'0T5jFDIkmmy'
```
---
A `TSIDGenerator` that creates TSIDs similar to [Twitter Snowflakes](https://github.com/twitter-archive/snowflake):
- Twitter snowflakes use 10 bits for node id: 5 bits for datacenter ID (max 31) and 5 bits for worker ID (max 31)
- Epoch starts on `2010-11-04T01:42:54.657Z`
- Counter uses 12 bits and starts at `0` (max: 4095 values per millisecond)
```python
from tsidpy import TSID, TSIDGenerator
datacenter: int = 1
worker: int = 1
node: int = datacenter << 5 | worker
epoch: datetime = datetime.fromisoformat('2010-11-04T01:42:54.657Z')
twitter_generator: TSIDGenerator = TSIDGenerator(node=node, node_bits=10,
epoch=epoch.timestamp() * 1000,
random_fn=lambda n: 0)
# use the generator
tsid: TSID = twitter_generator.create()
```
---
A `TSIDGenerator` that creates TSIDs similar to [Discord Snowflakes](https://discord.com/developers/docs/reference#snowflakes):
- Discord snowflakes use 10 bits for node id: 5 bits for worker ID (max 31) and 5 bits for process ID (max 31)
- Epoch starts on `2015-01-01T00:00:00.000Z`
- Counter uses 12 bits and starts at a random value.
```python
from tsidpy import TSID, TSIDGenerator
worker: int = 1
process: int = 1
node: int = worker << 5 | process
epoch: datetime = datetime.fromisoformat("2015-01-01T00:00:00.000Z")
discord_generator: TSIDGenerator = TSIDGenerator(node=node, node_bits=10,
epoch=epoch.timestamp() * 1000)
# use the generator
tsid: TSID = discord_generator.create()
```
---
Make `TSID.create()` to use the previous Discord generator:
```python
TSID.set_default_generator(discord_generator)
# at this point, you can use the default TSID.create()
tsid: TSID = TSID.create()
# or the generator
tsid: TSID = discord_generator.create()
```
---
### A note about node id and node bits
When creating a `TSIDGenerator`, remember you can't use a node id greater than `2^node_bits - 1`. For example, if you need to use a node id greater than 7, you need to use more than 3 bits for the node id:
```python
from tsidpy import TSIDGenerator
gen0 = TSIDGenerator(node=0, node_bis=3) # ok
gen1 = TSIDGenerator(node=1, node_bis=3) # ok
...
gen7 = TSIDGenerator(node=7, node_bis=3) # ok
# error: can't represent 8 with 3 bits
gen8 = TSIDGenerator(node=8, node_bis=3)
```
## Other ports, forks and OSS
Ports, forks, implementations and other OSS
------------------------------------------------------
Ports, forks and implementations:
| Language | Name |
| -------- | ---- |
| Go | [vishal-bihani/go-tsid](https://github.com/vishal-bihani/go-tsid) |
| Java | [vladmihalcea/hypersistence-tsid](https://github.com/vladmihalcea/hypersistence-tsid) |
| Java | [vincentdaogithub/tsid](https://github.com/vincentdaogithub/tsid) |
| .NET | [kgkoutis/TSID.Creator.NET](https://github.com/kgkoutis/TSID.Creator.NET) |
| PHP | [odan/tsid](https://github.com/odan/tsid) |
| Python | [luismedel/tsid-python](https://github.com/luismedel/tsid-python) |
| Rust | [jakudlaty/tsid](https://github.com/jakudlaty/tsid)
| TypeScript | [yubintw/tsid-ts](https://github.com/yubinTW/tsid-ts) |
Other OSS:
| Language | Name |
| -------- | ---- |
| Java | [fillumina/id-encryptor](https://github.com/fillumina/id-encryptor) |
| .NET | [ullmark/hashids.net](https://github.com/ullmark/hashids.net) |
## License
This library is Open Source software released under the [MIT license](https://opensource.org/licenses/MIT).