{"id":13651725,"url":"https://github.com/krunch3r76/entropythief","last_synced_at":"2025-10-25T17:31:39.490Z","repository":{"id":46671083,"uuid":"382565574","full_name":"krunch3r76/entropythief","owner":"krunch3r76","description":"pilfer provider entropy stashes","archived":false,"fork":false,"pushed_at":"2024-05-27T23:21:36.000Z","size":1020,"stargazers_count":4,"open_issues_count":6,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-10-30T01:59:02.614Z","etag":null,"topics":["golem","golem-apps"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/krunch3r76.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-03T08:32:31.000Z","updated_at":"2024-05-27T23:21:40.000Z","dependencies_parsed_at":"2024-01-03T05:15:41.454Z","dependency_job_id":"d7d4172a-7b86-418e-b095-d19605eab228","html_url":"https://github.com/krunch3r76/entropythief","commit_stats":{"total_commits":440,"total_committers":4,"mean_commits":110.0,"dds":0.525,"last_synced_commit":"dc6a0e9b4f63886f503c4cbfb64b4846feaac571"},"previous_names":[],"tags_count":47,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krunch3r76%2Fentropythief","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krunch3r76%2Fentropythief/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krunch3r76%2Fentropythief/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krunch3r76%2Fentropythief/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/krunch3r76","download_url":"https://codeload.github.com/krunch3r76/entropythief/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238191436,"owners_count":19431419,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golem","golem-apps"],"created_at":"2024-08-02T02:00:51.835Z","updated_at":"2025-10-25T17:31:39.483Z","avatar_url":"https://github.com/krunch3r76.png","language":"Python","funding_links":[],"categories":["Archive","Apps"],"sub_categories":["Apps","RNG"],"readme":"# entropythief\n\n**A golem requestor installation is needed to run this app. Please visit https://www.golem.network for details**\nget high quality entropy (randomness) at a steal of a rate from multiple providers utilizing Intel's RDSEED cpu instruction. Requests are sent whenever the pipe falls below half the set threshold. \n\n# requirements\na requestor installation of golem (see https://handbook.golem.network/requestor-tutorials/flash-tutorial-of-requestor-development)\nyapapi module version 0.13.1 (satisfied by following usage directions below)\n\n# Video Demo\nSee all videos at [https://krunch3r76.github.io/entropythief](https://krunch3r76.github.io/entropythief). Note: your browser must support the ogg codec to view most of these videos as of this writing.\n\n[![Watch the video]()](https://krunch3r76.github.io/entropythief/video2.ogg)\n\n[![Watch the video](https://krunch3r76.github.io/entropythief/thumbnail2.png)](https://user-images.githubusercontent.com/46289600/148718743-b365840e-86c3-436e-91b6-4852615bbe66.mp4)\n\nThe inline video here demonstrates the readers API (see API section) by running the included example scripts: these leverage the entropy to get random results such as dice throws or 1's and 0's.\n\n# non technical description\ncomputers can provide random numbers; however, hardware designs make such number sequences somewhat predictable. in computers, numbers are made up of sequences of 1s and 0's stored as bytes. entropythief has several computers produce random sequences of bytes and chops and mixes them to make an unpredictable sequence of never before seen random bit stream, like snowflakes, and without any intrinsic periodicity. randomness in computers has many scientific and practical applications and may have applications in art. atm entropythief makes no guarantees on the quality of randomness but does facilitate reprogramming to refine or improve the results. at any rate, entropythief may be summed up as a faucet of random 1's and 0's (from which a true random number generator may be built).\n\n# usage\n```\n# Linux or WSL\ngit clone https://github.com/krunch3r76/entropythief.git\ncd entropythief\ngit checkout v1.3.2\npython3 -m venv entropythief-venv\nsource entropythief-venv/bin/activate\npip install -r entropythief/requirements.txt\nyagna payment init --sender\n./entropythief.py # --help # to change the network from the default testnet\n```\n\n```\n# in a separate window while entropythief is running\ncd readers/print_nonce\npython3 print_nonce.py # to read some nonces\n\npython3 burn.sh # simple interactive script to read continuously from the pipe to demonstrate pipe refills\n\n\n# optional: \ntail -f stderr # or the entropythief yapapi log file\n```\n\nthis requestor runs to pilfer as many bytes of random 1's and 0's from as many providers as the user specifies. these parameter(s) can be adjusted on the fly by the user with the following commands:\n```\nset buflim=\u003cnum\u003e      # the minimum threshold that entropythief should do its best to stay above (refills when it falls beneath half this value)\nset maxworkers=\u003cnum\u003e  # the most workers Golem executor can provision  # the more the more exotic!\nset budget=\u003cfloat\u003e    # the budget above which work should cease (unless this is used to increase the budget)\nrestart               # after so many payment failures or after budget is exceeded, budget is implied to be over the limit. after setting run this.\nstop                  # stop/exit\n```\n\ntry: **note the following could take awhile to complete depending on network conditions but bytes are asynchronously chunked so it is not a hard wait; remember, you may want to follow along by invoking tail -f stderr.**\n```\npause\nset maxworkers=13 #across 13 workers\nset buflim=250*2**20 #for 1/4 gigabyte of random bytes\nset budget=5\nstart\n```\n# API\nonce entropythief runs, it displays the random bytes produced from workers as they arrive and are fed to a named pipe, topping it off. the named pipe can be accessed via any programming language and a sample Python API is provided at `readers/pipe_reader.py`, and an example script is in `readers/print_nonce`. The script retrieves 8 bytes from the pool of /tmp/pilferedbits and prints the corresponding 64bit nonce value. \n\nadditionally, to generate a random 1 or 0, developers can import EntropyBitReader from readers/entropybitreader.py\n\nand to get a truly random number, developers can import DiceRoller from readers/roll_die/diceroller.py\n\nthis application may expose sparsely undocumented parts of Golem's Python API, yapapi, to handle specific events in a novel way and to filter providers. see the code for details.\n\n# UI components\n```\nw:\u003cnumber of workers started but unfinished\u003e/\u003cmaximum\u003e\ncost:\u003ctotal cost aggregated from paid invoices\u003e/\u003cbudget\u003e\nbuf:\u003cnumber of random bits in units of bytes\u003e/\u003cmaximum number of bytes \"buflim\"\u003e\n```\non the same line that the UI accepts input, status messages are presented.\nthe fields are as described above.\n\n\n\n# other components\n```\n./stderr                  # messages and optionally yapapi logger info messages are written to this file in place of stderr\n./main.log                # development messages that come from the controller are written to this file\n./entropythief-yapapi.log # INFO and DEBUG messages from yapapi\n./entropythief.py         # controller-view runnable script that daemonizes the model (Golem executor) and coordinates with the view\napplication.py            # main routine\nmodel.py                  # the Golem specific code (daemonized by controller.py)\nTaskResultWriter.py       # base and derived TaskResultWriter (including Interleaver)\npipe_writer.py            # buffered named pipe writer\nreaders/pipe_reader.py        # API to named pipe\nreaders/entropybitreader.py   # provides a EntropyBitReader generator class to generate random bits\nreaders/roll_die/diceroller.py        # provides the DiceRoller class to function as a TRNG\n/tmp/pilferedbits             # named pipe to which the buffered writes continually occur as needed to top off\nworker/worker_public.py       # public namespace for variables etc needed by requestor to interact with the provider/vm\nworker/Dockerfile             # for vm creation\nworker/worker.py              # for vm creation\nworker/randwriter.c           # for vm, compiled into worker executable\n```\n\n# applications\nhave fun with a unpredictable and exotic stream of 1's and 0's!\n\n# memory management\nstart entropythief with the argument option --conceal-view which will prevent bytes from backlogging in stdout. this can be a considerable backlog while streaming gigabytes of random bits.\n\n# discussion\nrandomness is different things to different people. john venn described it as raindrops falling on a surface, touching different spots until the surface has been completely, or uniformly, saturated [0]. but the randomness here is not so \"perfect,\" as randomness commonly connotes otherwise! randomness with any uniformity and (potential) periodicity similar to what venn described is considered pseudo randomness, and in the context of using computers for research purposes, this is often sufficient and preferred. however, some endeavor to obtain true randomness (say for cryptography, monte carlo simulations...). one obstacle to this is the intrinsic determinism in any algorithm tied to the computer generating the numbers. the linux kernel offers \"true\" random, power-of-2 ranged numbers by altering random numbers according to \"random\" inputs, such as mouse movements. the amount of unpredictability thus generated is called entropy. however, there can only be so much user interaction before such randomness is exhausted. to solve this problem, intel added a feature and corresponding instruction to its 64-bit processors called RDSEED [1], [2]. it is regarded as a hardware true random number generator (base-2 ranged, e.g., 16-bit, 32-bit, or 64-bit ranges).\n\nhowever, some have raised concerns about its black-box nature. while RDSEED does not expand a seed deterministically like—say—RDRAND or /dev/urandom output, it still relies on intel’s proprietary hardware conditioner, made to remove bias. thus, although its output should be truly random, its quality ultimately depends on the trustworthiness of intel’s design. ideally, entropythief would draw from truly raw, non-uniform, and non-conditioned noise sources, but such access isn’t practical. while RDSEED’s output is carefully conditioned to be statistically uniform and unpredictable, unlike software PRNGs (Linux kernel) or DRBGs (via RDRAND), each value is generated non-deterministically from fresh hardware noise. by mixing streams from multiple independent processors, entropythief further increases assurance that its output remains both robustly uniform and unpredictable, even if one hardware source were to become biased or fail. it is hoped that, in this way, something approximating true randomness can be made manifest.\n\n## tl;dr\nentropythief solves the scarcity problem of high-quality entropy by procuring random bits in considerable quantities from independent sources—i.e., provider computers on the golem network—and mixing them together to effectively capture entropy from the random appearance and disappearance of providers (random sampling), rather than from physical user activity (like mouse movements). this input is inherently random because no one provider is necessarily available at any given time on the golem network. furthermore, its modular nature allows for increased refinement depending on the quality and/or quantity desired. for additional details, please refer to the cited sources [1,2] and to the source code of entropythief itself.\n\n[0] __The Logic of Chance__, 3rd Edition, John Venn, Chapter V: \"The conception of Randomness and its scientific treatment\". 1834-1923. https://www.gutenberg.org/ebooks/57359\n\n[1] https://software.intel.com/content/www/us/en/develop/articles/intel-digital-random-number-generator-drng-software-implementation-guide.html\n\n[2]\thttps://en.wikipedia.org/wiki/RDSEED\n\n\n# comments/reflections\n\nthis project was inspired by gandom. however, gandom draws upon the system's standard deterministic entropy source (kernel/cpu). furthermore, gandom mixes bytes (stream cipher) to produce a _single_ value whereas entropythief provides a stream of values, which incidentally are mixed by default, and can be played with in a myriad of ways, including passing thru a stream cipher (XOR'ing). additionally, entropythief stores bits in raw format while presenting to the user a bird's eye view of them in the intelligible base 16 (cf. base64).\n\nRDSEED samples the chip’s analog noise source, runs those raw bits through an on-die AES-based ‘whitener’ to remove bias, and returns the resulting 256-bit block directly—no deterministic expansion, just fresh hardware entropy every call (though it’s slower and may ask you to retry when the noise FIFO is empty).\n\nscalability of TRNGs remains an issue but that is a component problem. entropythief primarily addresses the scarcity of true \"randomness,\" indirectly addressing scalability. it is noteworthy that entropythief randomness does not attempt to fit the traditional modern definition of randomness, which is expected to have \"statistical independence, uniform distribution, and unpredictability\" [1], i.e. no sequence bias, no overall bias, no pattern. entropythief does not attempt to fit the definition but rather the connotation. it is therefore not necessary, for example, to prove uniformity, but all these qualities may reasonably be assumed to be met if run indefinitely. however, the proof, imho, would negate the reality of true randomness. if the reader is interested in traditonal randomness, the reader is referred to Intel's RDSEED in combination with software CSPRNGs.\n\n## credits\nentropythief was inspired by its predecessor golem app: https://github.com/reza-hackathons/gandom. the splash screen ascii art was obtained from: https://asciiart.website/index.php?art=logos%20and%20insignias/smiley\n\n## closing thoughts\nRandomness (exotic) is expensive. To get enough of it I think you literally have to steal it! This is a problem entropythief is working on but as of yet it serves as a POC with components that can be implemented in other apps where randomness or other features need to be leveraged.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrunch3r76%2Fentropythief","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkrunch3r76%2Fentropythief","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrunch3r76%2Fentropythief/lists"}