{"id":30781381,"url":"https://github.com/andezion/tans-integration","last_synced_at":"2025-09-08T23:03:11.769Z","repository":{"id":312936861,"uuid":"1049364261","full_name":"Andezion/tANS-Integration","owner":"Andezion","description":"My attempt to integrate a simplified tANS algorithm into the ContikiNG environment","archived":false,"fork":false,"pushed_at":"2025-09-02T23:09:18.000Z","size":759,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-03T00:09:50.452Z","etag":null,"topics":["algorithms","asymetric-numeral-systems","c","contiki-ng","embedded-c","udp-protocol"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Andezion.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-02T21:43:40.000Z","updated_at":"2025-09-02T23:54:13.000Z","dependencies_parsed_at":"2025-09-03T00:09:51.812Z","dependency_job_id":"8ff612e8-bb9c-4396-8915-6ee48d17d1c3","html_url":"https://github.com/Andezion/tANS-Integration","commit_stats":null,"previous_names":["andezion/tans-integration"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/Andezion/tANS-Integration","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andezion%2FtANS-Integration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andezion%2FtANS-Integration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andezion%2FtANS-Integration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andezion%2FtANS-Integration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Andezion","download_url":"https://codeload.github.com/Andezion/tANS-Integration/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andezion%2FtANS-Integration/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273733314,"owners_count":25158259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-05T02:00:09.113Z","response_time":402,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","asymetric-numeral-systems","c","contiki-ng","embedded-c","udp-protocol"],"created_at":"2025-09-05T08:39:18.694Z","updated_at":"2025-09-06T10:07:18.322Z","avatar_url":"https://github.com/Andezion.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tANS-Integration\nMy attempt to integrate a simplified tANS algorithm into the ContikiNG environment\n\n## Table of contents\n* [General info](#general-info)  \n* [Final result](#final-result)  \n* [Features](#features)  \n* [Technologies](#technologies)  \n* [Setup](#setup)  \n\n---\n\n## General info  \nFirst, let's take a look at what tANS is and why it is important to be able to integrate it!\ntANS (Tabled Asymmetric Numeral Systems) is a modern entropy compression method that combines the efficiency of arithmetic coding with the simplicity of Huffman implementation. It allows for high compression ratios with minimal computational and memory overhead, making it particularly valuable for embedded systems and IoT devices.\n\nThe integration of tANS into Contiki-NG, an operating system for the Internet of Things, opens up the possibility of reducing the amount of data transmitted over wireless channels and lowering energy consumption. This is critically important in the context of limited sensor node resources, where every byte transmitted and every processor cycle counts.\n\n---\n\n## Final result  \nAs a result of this work, the tANS algorithm was integrated into the Contiki-NG operating system!\n\nOn the local device, I successfully implemented a single process that integrates the following components:\n\nInitialization\n - Encoder and decoder are created based on a predefined alphabet with weighted symbol frequencies (spaces, vowels, consonants).\n\nCompression\n - String messages are encoded into a compact bitstream using the tANS method.\n\n![Compression](./photo/photo2.png)\n\nTransmission\n - Encoded packets are transmitted via UDP between nodes (in the test setup — loopback to the same node).\n   \n![Transmission](./photo/photo.png)\n\nDecoding\n - Messages are successfully reconstructed on the receiving side without loss of information.\n\nExperimental evaluation\n - Average compression ratio: ~0.6.\n - Confirms the efficiency of tANS under constrained memory, bandwidth, and energy conditions.\n\nConclusion\n- The integration demonstrates that tANS can be applied in Contiki-NG-based IoT systems to reduce network traffic and power consumption.\n\n---\n\nNext, I decided to check the network interaction.\nTwo nodes in the network:\n\n - Terminal 1 (Receiver) raised the tun0 interface and received the global IPv6 address fd00::302:304:506:708.\n - Terminal 2 (Sender) raised tun1 and received fd00::302:304:506:709.\n - They are in the same fd00::/64 subnet, which confirms the correctness of the network configuration.\n\nInitialisation of tANS on both nodes:\n\nThe algorithm works with the same alphabet (64 characters, 28 unique, with bias for spaces and vowels).\n - Both the encoder (Sender) and decoder (Receiver) start successfully and output the same frequency table.\n\nExchange process:\n\n - Every 10 seconds, the Sender takes a test string, compresses it using tANS, forms a UDP packet and sends it to the Receiver.\n - The Receiver receives the packet, restores the original message, checks the state and bitstream.\n\n```\nopened tun device ``/dev/tun0''\ntun0: flags=4305\u003cUP,POINTOPOINT,RUNNING,NOARP,MULTICAST\u003e  mtu 1500\n        inet 127.0.1.1  netmask 255.255.255.255  destination 127.0.1.1\n        inet6 fe80::d95a:1d10:efe2:918d  prefixlen 64  scopeid 0x20\u003clink\u003e\n        inet6 fd00::1  prefixlen 64  scopeid 0x0\u003cglobal\u003e\n        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)\n[INFO: Main      ] Starting Contiki-NG-develop/v5.1-dirty\n[INFO: Main      ] - Routing: RPL Lite\n[INFO: Main      ] - Net: tun6\n[INFO: Main      ] - MAC: nullmac\n[INFO: Main      ] - 802.15.4 PANID: 0xabcd\n[INFO: Main      ] - 802.15.4 Default channel: 26\n[INFO: Main      ] Node ID: 1800\n[INFO: Main      ] Link-layer address: 0102.0304.0506.0708\n[INFO: Main      ] Tentative link-local IPv6 address: fe80::302:304:506:708\n[INFO: Native    ] Added global IPv6 address fd00::302:304:506:708\n[INFO: tANS Receiver] tANS Receiver starting\n[INFO: tANS Receiver] tANS decoder initialized successfully\nBlock size: 64\nUnique symbols: 28\nSymbol frequencies: ' ':20 'a':3 'e':3 'i':3 'o':3 'u':3 'n':1 'r':1 't':1 'l':1 's':1 'd':1 'h':1 'c':1 'f':1 'm':1 'p':1 'g':1 'w':1 'y':1 'b':1 'v':1 'k':1 'j':1 'x':1 'q':1 'z':1 '.':8 \n[INFO: tANS Receiver] Waiting for tANS packets on UDP port 5678\n\n[INFO: tANS Receiver] Received UDP packet from fd00::302:304:506:709 port 8765, length 518\n[INFO: tANS Receiver] Received tANS packet: state=105, bitstream_size=52 bits\n[INFO: tANS Receiver] Successfully decoded message: 'hello world' (11 bytes)\n[INFO: tANS Receiver] Compression ratio: 0.59\n[INFO: tANS Receiver] Final state: 64\n\n[INFO: tANS Receiver] Received UDP packet from fd00::302:304:506:709 port 8765, length 518\n[INFO: tANS Receiver] Received tANS packet: state=101, bitstream_size=103 bits\n[INFO: tANS Receiver] Successfully decoded message: 'this is a test message' (22 bytes)\n[INFO: tANS Receiver] Compression ratio: 0.59\n[INFO: tANS Receiver] Final state: 64\n```\n\nEfficiency:\n\n - The message ‘hello world’ (11 bytes = 88 bits) was reduced to 52 bits → 0.59 of the original.\n - The message ‘this is a test message’ (22 bytes = 176 bits) was reduced to 103 bits → also 0.59.\n - The receiver confirms that the messages are decoded without loss and match the original ones.\n\n```\nopened tun device ``/dev/tun0''\ntun0: flags=4305\u003cUP,POINTOPOINT,RUNNING,NOARP,MULTICAST\u003e  mtu 1500\n        inet 127.0.1.1  netmask 255.255.255.255  destination 127.0.1.1\n        inet6 fe80::d95a:1d10:efe2:918d  prefixlen 64  scopeid 0x20\u003clink\u003e\n        inet6 fd00::1  prefixlen 64  scopeid 0x0\u003cglobal\u003e\n        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 500  (UNSPEC)\n[INFO: Main      ] Starting Contiki-NG-develop/v5.1-dirty\n[INFO: Main      ] - Routing: RPL Lite\n[INFO: Main      ] - Net: tun6\n[INFO: Main      ] - MAC: nullmac\n[INFO: Main      ] - 802.15.4 PANID: 0xabcd\n[INFO: Main      ] - 802.15.4 Default channel: 26\n[INFO: Main      ] Node ID: 1800\n[INFO: Main      ] Link-layer address: 0102.0304.0506.0708\n[INFO: Main      ] Tentative link-local IPv6 address: fe80::302:304:506:708\n[INFO: Native    ] Added global IPv6 address fd00::302:304:506:708\n[INFO: tANS Receiver] tANS Receiver starting\n[INFO: tANS Receiver] tANS decoder initialized successfully\nBlock size: 64\nUnique symbols: 28\nSymbol frequencies: ' ':20 'a':3 'e':3 'i':3 'o':3 'u':3 'n':1 'r':1 't':1 'l':1 's':1 'd':1 'h':1 'c':1 'f':1 'm':1 'p':1 'g':1 'w':1 'y':1 'b':1 'v':1 'k':1 'j':1 'x':1 'q':1 'z':1 '.':8 \n[INFO: tANS Receiver] Waiting for tANS packets on UDP port 5678\n\n[INFO: tANS Receiver] Received UDP packet from fd00::302:304:506:709 port 8765, length 518\n[INFO: tANS Receiver] Received tANS packet: state=105, bitstream_size=52 bits\n[INFO: tANS Receiver] Successfully decoded message: 'hello world' (11 bytes)\n[INFO: tANS Receiver] Compression ratio: 0.59\n[INFO: tANS Receiver] Final state: 64\n\n[INFO: tANS Receiver] Received UDP packet from fd00::302:304:506:709 port 8765, length 518\n[INFO: tANS Receiver] Received tANS packet: state=101, bitstream_size=103 bits\n[INFO: tANS Receiver] Successfully decoded message: 'this is a test message' (22 bytes)\n[INFO: tANS Receiver] Compression ratio: 0.59\n[INFO: tANS Receiver] Final state: 64\n```\n---\n\ntANS Compression over TCP in C++\n\nImplemented tANS (table-based Asymmetric Numeral Systems) compression over TCP in C++ as part of the networking experiments for the Contiki project. This allowed me to test real-time compressed data transmission in a network environment.\n\nHighlights:\n- Full sender/receiver implementation using TCP sockets.\n- Encoding/decoding with tANS, handling arbitrary byte streams.\n- Collected detailed statistics: symbol frequencies, block sizes, compression ratio, and bitstream size.\n- Tested with messages of several KB to verify reliability and efficiency.\n- Gained hands-on experience with network programming and compression integration.\n\nExample output:\n![Transmission](./photo/photo3.png)\n\nKey conclusion:\nI don't just have a working codec in simulation, but two real nodes that exchange compressed data via UDP and correctly restore the original text. This proves that tANS can be integrated into the Contiki-NG stack to reduce network traffic in IoT.\n\n---\n\n## Features  \n- tANS Compression Algorithm: \nImplementation of table-based Asymmetric Numeral Systems for efficient text compression\n\n- Frequency-Based Alphabet: \nCustom 64-symbol alphabet optimized for English text with weighted character frequencies (20 spaces, 3 vowels each, consonants by usage frequency)\n\n- UDP Network Communication: \nIPv6 UDP-based packet transmission between sender and receiver nodes\n\n- Dual Operation Modes:\nSeparate sender/receiver applications for distributed testing\nCombined application for local loopback testing\n\n- Real-time Compression Metrics: \nLive compression ratio calculation and performance logging\n\n- Embedded-Friendly Design: \nMemory-efficient implementation suitable for IoT devices with limited resources\n\n- Custom Packet Structure: \nBinary packet format with state preservation for reliable decompression\n\n- Contiki-NG Integration: \nNative support for Contiki-NG RTOS with proper process management\n\n---\n\n## Technologies  \n- Operating System: Contiki-NG (IoT-focused RTOS)\n- Programming Language: C (ANSI C compatible)\n- Network Protocol: IPv6 UDP with Simple UDP API\n- Compression Algorithm: table-based Asymmetric Numeral Systems (tANS)\n- Block Size: 64-symbol alphabet with configurable frequency distribution\n- Target Platform: Native target (Linux/Unix) with TUN interface support\n- Packet Size: 1024 bytes per compressed packet (including headers and metadata)\n- Compression Performance: ~0.59-0.62 compression ratio for typical English text\n- Network Architecture: Point-to-point communication using fd00::/64 IPv6 local addresses\n- Build System: Contiki-NG Makefile system with modular source file inclusion\n- Interface: TUN/TAP virtual network interfaces for testing and deployment\n\n---\n\n## Setup  \n1) Installing Contiki-NG\n``` \nsudo apt update \u0026\u0026 sudo apt upgrade -y\n\nsudo apt install -y build-essential git wget unzip python3-pip python3-setuptools\nsudo apt install -y gcc-arm-none-eabi doxygen ant default-jdk\nsudo apt install -y libcoap-3-dev libssl-dev\n\ncd ~\ngit clone https://github.com/contiki-ng/contiki-ng.git\ncd contiki-ng\n\necho 'export CONTIKI=~/contiki-ng' \u003e\u003e ~/.bashrc\nsource ~/.bashrc\n\ncd examples/hello-world\nmake TARGET=native\nsudo ./hello-world.native\n```\nYou should see something like this:\n![helper](./photo/photo1.png)\n\nIf something went wrong, I strongly recommend watching this video:\nhttps://www.youtube.com/watch?v=a4radETt04U\u0026list=LL\u0026index=21\n\n---\n\n2) Cloning your project:\n```\ncd ~\nmkdir tans-project \u0026\u0026 cd tans-project\n\ngit clone https://github.com/Andezion/tANS-Integration.git\ncd tans-contiki-demo\n```\n3) Installing additional tools:\n```\nsudo apt install -y socat tcpdump wireshark netcat-openbsd\n\nsudo modprobe tun\nls -la /dev/net/tun\n```\nIf you want to check the operation from different computers, then:\n4.1) Terminal 1 - Receiver:\n```\ncd ~/tans-project/tANS-Integration/receiver\nmake TARGET=native clean\nmake TARGET=native\nsudo ./build/native/receiver.native\n```\n5.1) Terminal 2 - Sender:\n```\ncd ~/tans-project/tANS-Integration/sender\nmake TARGET=native clean  \nmake TARGET=native\nsudo ./build/native/sender.native\n```\n6.1) Terminal 3 - Network settings:\n```\nip link show | grep tun\n\nsudo sysctl -w net.ipv6.conf.all.forwarding=1\n\nsudo ip link add name br0 type bridge\nsudo ip link set br0 up\n```\nError) \nProblem: Failed to open tun device\n```\nsudo chmod 666 /dev/net/tun\n```\nProblem: Cannot find device tun1\n```\nsudo ip tuntap add mode tun dev tun1\nsudo ip link set tun1 up\n```\nProblem: Packets are not reaching their destination\n```\nsudo tcpdump -i any udp port 5678 -v\nping6 fd00::302:304:506:708\n```\n\n---\n\nIf you want to check a simpler option:\n4.2) Testing locally (simple option)\n```\ncd ~/tans-project/tans-contiki-demo/combined\nmake TARGET=native\nsudo ./build/native/tans_combined.native\n```\nFinally) Cleaning after testing\n```\nsudo pkill -f \".native\"\n\nsudo ip link delete tun0 2\u003e/dev/null || true\nsudo ip link delete tun1 2\u003e/dev/null || true\nsudo ip link delete br0 2\u003e/dev/null || true\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandezion%2Ftans-integration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandezion%2Ftans-integration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandezion%2Ftans-integration/lists"}