{"id":13713534,"url":"https://github.com/OdyseeTeam/chainquery","last_synced_at":"2025-05-06T23:32:23.940Z","repository":{"id":38036772,"uuid":"121024702","full_name":"OdyseeTeam/chainquery","owner":"OdyseeTeam","description":"Chainquery parses and syncs the LBRY blockchain data into structured SQL","archived":false,"fork":false,"pushed_at":"2024-09-27T17:03:02.000Z","size":6842,"stargazers_count":2238,"open_issues_count":41,"forks_count":39,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-04-11T15:57:12.059Z","etag":null,"topics":["blockchain","go","lbry","sql"],"latest_commit_sha":null,"homepage":"https://lbry.tech","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OdyseeTeam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-02-10T14:57:47.000Z","updated_at":"2025-03-14T07:41:52.000Z","dependencies_parsed_at":"2024-11-14T00:31:01.411Z","dependency_job_id":"9c635f8d-03fb-4f06-b011-6eda070887cf","html_url":"https://github.com/OdyseeTeam/chainquery","commit_stats":{"total_commits":591,"total_committers":13,"mean_commits":45.46153846153846,"dds":0.1252115059221658,"last_synced_commit":"068c29e028650135e3ebdcc63d2f007da7816fe6"},"previous_names":[],"tags_count":87,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OdyseeTeam%2Fchainquery","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OdyseeTeam%2Fchainquery/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OdyseeTeam%2Fchainquery/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OdyseeTeam%2Fchainquery/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OdyseeTeam","download_url":"https://codeload.github.com/OdyseeTeam/chainquery/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252787533,"owners_count":21804277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blockchain","go","lbry","sql"],"created_at":"2024-08-02T23:01:38.883Z","updated_at":"2025-05-06T23:32:23.493Z","avatar_url":"https://github.com/OdyseeTeam.png","language":"Go","readme":"\n# LBRY Chainquery\n\n[![Build Status](https://travis-ci.org/lbryio/chainquery.svg?branch=master)](https://travis-ci.org/lbryio/chainquery)\n\n![Code Climate](https://img.shields.io/codeclimate/tech-debt/lbryio/chainquery.svg)\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/lbryio/chainquery)](https://goreportcard.com/report/github.com/lbryio/chainquery)\n\n[![Maintainability](https://api.codeclimate.com/v1/badges/3f60ac6b2d7e96f93992/maintainability)](https://codeclimate.com/github/lbryio/chainquery/maintainability)\n\n![GitHub release](https://img.shields.io/github/release/lbryio/chainquery.svg)\n\n![Github commits (since latest release)](https://img.shields.io/github/commits-since/lbryio/chainquery/latest.svg)\n\n[![Coverage Status](https://coveralls.io/repos/github/lbryio/chainquery/badge.svg?branch=master)](https://coveralls.io/github/lbryio/chainquery?branch=master)\n\n## Prerequisites\n\n### OS Specifics\n\n#### OSX\n\n- In order to use  `wget` you will need `brew install wget` (used in [build.sh](/scripts/build.sh))\n- Chainquery is built for Linux by default in  [build.sh](/scripts/build.sh), so you will need to modify the cross compilation for an OSX build.\n- Be sure to give execute privileges to the [scripts](/scripts) you plan to use.\n\n### Go\n\nMake sure you have Go 1.11+ (required for [go-releaser](https://goreleaser.com/))\n\n- Ubuntu: https://launchpad.net/~longsleep/+archive/ubuntu/golang-backports or https://github.com/golang/go/wiki/Ubuntu [Easier but maybe older] **(recommended)**\n- Linux: [GoLang official install guide](https://golang.org/doc/install#tarball) [Harder but up to date] \n- OSX: [GoLang official install guide](https://golang.org/doc/install#macos)\n- Windows: [GoLang official install guide](https://golang.org/doc/install#windows)\n\n### MySQL\n\n- Install and run mysql 8 (OSX: `brew install mysql`)\n- Create `chainquery` database.\n- Create user `lbry` with password `lbry` and grant it all permissions on `chainquery` db.\n\n### Lbrycrd\n\n- Install lbrycrdd (https://github.com/lbryio/lbrycrd/releases)\n- Ensure `~/.lbrycrd/lbrycrd.conf` file exists with username and password.\n  If you don't have one, run:\n\n  ```\n  mkdir -p ~/.lbrycrd\n  echo -e \"rpcuser=lbryrpc\\nrpcpassword=$(env LC_CTYPE=C LC_ALL=C tr -dc A-Za-z0-9 \u003c /dev/urandom | head -c 16 | xargs)\" \u003e\u003e ~/.lbrycrd/lbrycrd.conf\n  ```\n\n- Run `./lbrycrdd -server -daemon -txindex -conf=$HOME/.lbrycrd/lbrycrd.conf`. If you get an error about indexing, add the `-reindex` flag for one run. You will only need to\n  reindex once.\n\n## Configuration\n\nChainquery can be [configured](/config/default/chainqueryconfig.toml) via toml file.\n\n## Running from Source\n\n```\ngo get -u github.com/lbryio/chainquery\ncd \"$(go env GOPATH)/src/github.com/lbryio/chainquery\"\n./dev.sh\n```\n## Running from Release\n\nThis will likely eventually be the main supported method of running Chainquery in your environment but this sections documentation is a WIP so YMMV\n\nGet a download link for your operating system specific release from [the releases page](https://github.com/lbryio/chainquery/releases) then use the following command with your download link.\n\n```\n  wget -O ~/chainquery.zip https://example.com/path/to/your/release.zip\n  Example:\n  wget -O ~/chainquery.zip https://github.com/lbryio/chainquery/releases/download/v1.1.2/chainquery_1.1.2_Linux_x86_64.zip\n```\n\nUnzip the package you just downloaded with the following.\n\n```\ncd ~/\nunzip ~/chainquery.zip\n```\nYour console should show you something similar to the following.\n```\nroot@8fe4046b6d46:~# unzip chainquery.zip\nArchive:  chainquery.zip\n  inflating: LICENSE\n  inflating: README.md\n  inflating: chainquery\n  ```\nOf course you don't have to extract all of this stuff to your machines home directory `~/` you must use whatever paths you prefer.  One that could be beneficial is adding these executables into your systems `$PATH` this is out of the scope of this README.\n\nThe main Chainquery binary should be marked as Executable by default but if not you can run the following.\n```\nchmod +x ~/chainquery\n```\nFinally running chainquery should be as simple as.\n```\n~/chainquery serve\n```\n\nYou can obtain information on the flags in Chainqueries main binary by running the following.\n```\n~/chainquery -help\n```\n\n## The Model \n\nThe model of Chainquery at its foundation consists of the fundamental data types found in the block chain.\nThis information is then expounded on with additional columns and tables that make querying the data much easier.\n\n### [Latest Schema](/db/chainquery_schema.sql)\n\n## What does Chainquery consist of?\n\nChainquery consists of 4 main parts. The API Server, the Daemon, the Job Scheduler, and the upgrade manager. \n\n### API Server\n\nThe API Server services either structured queries via defined APIs or raw SQL against \nthe Chainquery MySQL database. The APIs are documented via [Chainquery APIs](https://lbryio.github.io/chainquery/),\na work in progress :) . \n\n### Daemon\n\nThe Daemon is responsible for updating the Chainquery database to keep it in sync with lbrycrd data. The daemon runs periodically to check if there are newly \ncreated blocks that need to be processed. The Daemon simply processes the block and its\ntransactions. It also handles blockchain reorganizations. It will remove the orphaned block data and processing the new blocks from that height it diverged.\nThe entry points are [daemon iterations](/daemon/daemon.go)(`func daemonIteration()`) [block processing](/daemon/processing/block.go)(`func RunBlockProcessing(height *uint64)`), \n[transaction processing](/daemon/processing/transaction.go)(`func ProcessTx(jsonTx *lbrycrd.TxRawResult, blockTime uint64)`).\n\n### Job Scheduler\n\nThe job scheduler schedules different types of jobs to update the Chainquery database [example](/daemon/jobs/claimtriesync.go).\nThese jobs synchronize different areas of the data either to make queries faster or ascertain information that is not\ndirectly part of the raw blockchain. The example provided is leveraged to handle the status of a claim which is actually\nstored in the ClaimTrie of LBRYcrd. So it runs periodically to make sure Chainquery has the most up to date status of \nclaims in the trie. The table `job_status` stores the current state of a particular job, like when it last synced.\n\n### Upgrade Manager\n\nThe upgrade manager handles data upgrades between versions. The table  `application_status` stores information about the\nstate of the application as it relates to the data, api and app versions. This is all leveraged by the upgrade manager so it \nknows what scripts might need to be run to keep the data in sync across deployments. The [scripts](/daemon/upgrademanager/script.go)\nare foundation of the [upgrade manager](/daemon/upgrademanager/upgrade.go).\n\n## Contributing\n\nContributions to this project are welcome, encouraged, and compensated. For more details, see [https://lbry.tech/contribute](https://lbry.tech/contribute)\n\nThe `master` branch is regularly built and tested, but is not guaranteed to be\ncompletely stable. [Releases](https://github.com/lbryio/chainquery/releases) are created\nregularly to indicate new official, stable release versions.\n\nDevelopers are strongly encouraged to write unit tests for new code, and to\nsubmit new unit tests for old code. Unit tests can be compiled and run\n with: `go test ./...` from the source directory which should be `$GOPATH/github.com/lbryio/chainquery`.\n \n All contributions should run the e2e test via `./e2e/ee2e.sh`. This requires Docker to be installed.\n\n## Updating the generated models\n\nWe use [sqlboiler](https://github.com/lbryio/sqlboiler) to generate our data models based on the db schema. If you make  schema changes, run `./gen_models.sh` to\nregenerate the models.\n\n**A note of caution:** the models are generated by connecting to the MySQL server and inspecting the current schema. If you made any db schema changes by hand, then the\nschema may be out of sync with the migrations. Here's the safe way to ensure that the models match the migrations:\n\n- Put all the schema changes you want to make into a migration.\n- In mysql, drop and recreate the db you're using, so that it's empty.\n- Run `./dev.sh`. This will run all the migrations on the empty db.\n- Run `./gen_models.sh` to update the models.\n\nThis process ensures that the generated models will match the updated schema exactly, so there are no surprises when the migrations are applied to the live db.\n\n## License\n\nThis project is MIT licensed. For the full license, see [LICENSE](LICENSE).\n\n## Security\n\nWe take security seriously. Please contact security@lbry.io regarding any security issues.\nOur PGP key is [here](https://lbry.com/faq/pgp-key) if you need it.\n\n## Contact\n\nThe primary contact for this project is [@tiger5226](https://github.com/tiger5226) (beamer -at- odysee.com)\n","funding_links":[],"categories":["Repositories","Go"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOdyseeTeam%2Fchainquery","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOdyseeTeam%2Fchainquery","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOdyseeTeam%2Fchainquery/lists"}