{"id":21496112,"url":"https://github.com/sovaslava/anydatarequester","last_synced_at":"2025-03-17T12:15:17.449Z","repository":{"id":62936341,"uuid":"551954557","full_name":"SovaSlava/AnydataRequester","owner":"SovaSlava","description":null,"archived":false,"fork":false,"pushed_at":"2022-11-15T13:44:31.000Z","size":319,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-09T02:32:27.668Z","etag":null,"topics":["chainlink","regular-expression"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SovaSlava.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-10-15T13:28:58.000Z","updated_at":"2022-11-20T23:55:33.000Z","dependencies_parsed_at":"2023-01-23T16:16:05.951Z","dependency_job_id":null,"html_url":"https://github.com/SovaSlava/AnydataRequester","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SovaSlava%2FAnydataRequester","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SovaSlava%2FAnydataRequester/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SovaSlava%2FAnydataRequester/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SovaSlava%2FAnydataRequester/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SovaSlava","download_url":"https://codeload.github.com/SovaSlava/AnydataRequester/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244031153,"owners_count":20386534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chainlink","regular-expression"],"created_at":"2024-11-23T16:14:51.357Z","updated_at":"2025-03-17T12:15:17.400Z","avatar_url":"https://github.com/SovaSlava.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Inspiration\nSometimes, smart contracts need to data from websites, but some of them don't have neccessary api. Or contract need to part of non-json ipfs file. I think Its hinders web3 development.\n## What it does\nAny data requester - is external adapter for chainlink node, which give you opportunity to get any data from web sites or ipfs file. Its possible, because you can use regular expressions for getting information. \nThere are 3 type of requests:\n1. extract one text - you have to pass matchIndex parameter for cases, if result of matches will have more than one match of regulax expression. \n2. extract all data(multiMatch), which match with regular expression.\n3. get true/false result for question-if text from website or ipfs contain data, which match to regular expression.\n\nAnd, if you make mistake in url or regexp, you will get response anyway. And can understand, where was mistake, because each response from external adapter hash status.\nIt is 0, if response complete without erros.\n1 - if url is unreachable\n2 - if regular express has error\n3 - no such index in array of matched text (only for multiMatch type)\n\n## How we built it\nI launch own chainlink node in goerli network on vps for testing external adapter. And use hardhat for writing smart contract. Also write external adapter's code.\n## Challenges we ran into\nWritting errors handlers for cornen cases, converting string array to bytes for passing in contract.\n## Accomplishments that we're proud of\nAccomplishments that we're proud of tool, which brings opportunity get any data from web2 sites and ipfs. \n## What we learned\nHow launch chainlink node, describe jobs, how external adapters work.\n## What's next for Any data requester\n* Next feature could be getting any data from tor. \n* Use own ipfs node instance \n* Add feature, which allow replace part of matched data, using external adapter. \n\n## How to use it\nContract in goerli - https://goerli.etherscan.io/address/0x49d042d1f5933C6322568DD140241B9730ab98c5\n#### Get second headline from chainlink press page\nUse function \"extractText\"\n* url - https://chain.link/press/\n* regexp - /h3-newstitle\\\"\u003e(.*?)\u003c/gim\n* matchIndex - 1 (because it starts from 0)\nWait about 1 minute.. and you can read extracted data using function extractTextResult \n\n#### Get all sections's names  from docs.openzeppelin.com\nUse function \"multiExtractText\"\n* url - https://docs.openzeppelin.com/\n* regexp - /card-title\\\"\u003e(.*?)\u003c/span/gim\nWait about 1 minute.. and you can read extracted data using function multipliExtractTextResult\n\n#### If ipfs file contains word with part \"EDU\" \nUse function isTextMatch\n* url - ipfs://QmSc6z1migCZKy9npZ4SjxQ5XPqeTqPL3qwgJweeJaYTwv\n* regexp - /EDU/gim\n\nFor clearing all results and status, call clearResults function\n\n## How to run adapter\n```\ncd externalAdapter\ndocker build . -t adapter\ndocker run -d --restart=on-failure -p 8080:8080 -it adapter:latest \n```\nJobs descriptions you can find in file jobsDescriptions.txt \nYou should change oracle address in job descriptions to you own oracle address.\n\nStatus description:\n0 - OK\n1 - error in regular expression\n2 - no such match-index\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsovaslava%2Fanydatarequester","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsovaslava%2Fanydatarequester","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsovaslava%2Fanydatarequester/lists"}