{"id":14988405,"url":"https://github.com/apache/rya","last_synced_at":"2025-06-16T16:04:42.697Z","repository":{"id":39825448,"uuid":"43734923","full_name":"apache/rya","owner":"apache","description":"Mirror of Apache Rya","archived":false,"fork":false,"pushed_at":"2023-04-16T19:16:26.000Z","size":8747,"stargazers_count":110,"open_issues_count":26,"forks_count":74,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-06-08T11:52:26.339Z","etag":null,"topics":["rya"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-10-06T07:00:05.000Z","updated_at":"2025-05-12T21:11:34.000Z","dependencies_parsed_at":"2025-01-16T18:24:32.498Z","dependency_job_id":"1ad504e7-1a18-489c-bc41-701367b616c2","html_url":"https://github.com/apache/rya","commit_stats":{"total_commits":446,"total_committers":39,"mean_commits":"11.435897435897436","dds":0.8094170403587444,"last_synced_commit":"ac44b3c00e2dc279bfb178eb94e56ba4384602ed"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/apache/rya","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Frya","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Frya/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Frya/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Frya/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/rya/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Frya/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259728481,"owners_count":22902406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rya"],"created_at":"2024-09-24T14:16:39.689Z","updated_at":"2025-06-16T16:04:42.580Z","avatar_url":"https://github.com/apache.png","language":"Java","funding_links":[],"categories":["数据库"],"sub_categories":[],"readme":"\u003c!-- Licensed to the Apache Software Foundation (ASF) under one\nor more contributor license agreements.  See the NOTICE file\ndistributed with this work for additional information\nregarding copyright ownership.  The ASF licenses this file\nto you under the Apache License, Version 2.0 (the\n\"License\"); you may not use this file except in compliance\nwith the License.  You may obtain a copy of the License at\n\n  http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing,\nsoftware distributed under the License is distributed on an\n\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\nKIND, either express or implied.  See the License for the\nspecific language governing permissions and limitations\nunder the License. --\u003e\n\n# Apache Rya\n\n## Overview\n\n[Apache Rya] is a scalable RDF Store that is built on top of a Columnar Index Store (such as Accumulo). It is implemented as an extension to RDF4J to provide easy query mechanisms (SPARQL, SERQL, etc) and Rdf data storage (RDF/XML, NTriples, etc).\n\nRya stands for RDF y(and) Accumulo.\n\n## Manual\n\nA copy of the Apache Rya Manual is located [here](extras/rya.manual/src/site/markdown/index.md).  The material in the manual and below may be out of sync.\n\n## Upgrade Path\n\nSince the data encodings changed in the 3.2.2 release, you will need to run the Upgrade322Tool MapReduce job to perform the upgrade.\n\n1. Build the project with -Pmr to build the mapreduce artifacts\n2. Make sure to clone the rya tables before doing the upgrade\n3. Run\n    \n```\nhadoop jar accumulo.rya-mr.jar org.apache.rya.accumulo.mr.upgrade.Upgrade322Tool -Dac.instance={} -Dac.username={} -Dac.pwd={}\n```\n\n## Quick Start VM\n\nA quickstart Vagrant VM is availible [here](extras/vagrantExample/src/main/vagrant)\n\n## Quick Start\n\nThis tutorial will outline the steps needed to get quickly started with the Rya store using the web based endpoint.\n\n### Prerequisites\n\n* Columnar Store (either Accumulo or MongoDB) The tutorial will go forward using Accumulo\n* Rya code (Git: https://gitbox.apache.org/repos/asf?p=rya.git or https://github.com/apache/rya.git)\n* Maven 3.0 +\n\n### Building from Source\n\nUsing Git, pull down the latest code from the url above.\n\nRun the command to build the code `mvn clean install`\n\nIf all goes well, the build should be successful and a war should be produced in `web/web.rya/target/web.rya.war`\n\nNote: The following profiles are available to tailor the build:\n\n\n| Profile ID | Purpose |\n| ---------- | ------- |\n| geoindexing | perform a build of the geomesa/lucene indexing |\n| mongodb | build with mongoDB configuration (defaults to accumulo) |\n\nTo run the build with the profile 'geoindexing' `mvn clean install -P geoindexing`.\n\nNote: If you are building on windows, you will need hadoop-common 2.6.0's `winutils.exe` and `hadoop.dll`.  You can download it from [here](https://github.com/amihalik/hadoop-common-2.6.0-bin/archive/master.zip).  This build requires the [Visual C++ Redistributable for Visual Studio 2015 (x64)](https://www.microsoft.com/en-us/download/details.aspx?id=48145).   Also you will need to set your path and Hadoop home using the commands below:\n\n```\nset HADOOP_HOME=c:\\hadoop-common-2.6.0-bin\nset PATH=%PATH%;c:\\hadoop-common-2.6.0-bin\\bin\n```\n\n### Deployment Using Tomcat\n\nUnwar the above war into the webapps directory.\n\nTo point the web.rya war to the appropriate database instance, make a properties file `environment.properties` and put it in the classpath. \n\nHere is an example for accumulo:\n\n```\n# Accumulo instance name\ninstance.name=accumulo\n# Accumulo Zookeepers\ninstance.zk=localhost:2181\n# Accumulo username\ninstance.username=root\n# Accumulo password\ninstance.password=secret\n\n# Rya Table Prefix\nrya.tableprefix=triplestore_\n# To display the query plan\nrya.displayqueryplan=true\n```\n\nPlease consult the [Accumulo], [ZooKeeper], and [Hadoop] documentation for help with setting up these prerequisites.\n\nHere is an example for mongoDB (populate user/userpassword if authentication to mongoDB required):\n```\nrya.displayqueryplan=true\nsc.useMongo=true\nsc.use_freetext=true\nsc.geo.predicates=http://www.opengis.net/ont/geosparql#asWKT\nsc.freetext.predicates=http://www.w3.org/2000/01/rdf-schema#label\nmongo.db.instance=localhost\nmongo.db.port=27017\nmongo.db.name=rya\nmongo.db.collectionprefix=rya_\nmongo.db.user=\nmongo.db.userpassword=\nmongo.geo.maxdist=1e-10\n```\nStart the Tomcat server. `./bin/startup.sh`\n\n## Usage\n\n### Load Data\n\n#### Direct Code\n\nHere is a code snippet for directly running against Accumulo with the code. You will need at least accumulo.rya.jar, rya.api, rya.sail.impl on the classpath and transitive dependencies. I find that Maven is the easiest way to get a project dependency tree set up.\n\n``` JAVA\nConnector connector = new ZooKeeperInstance(\"instance\", \"zoo1,zoo2,zoo3\").getConnector(\"user\", \"password\");\n\nfinal RdfCloudTripleStore store = new RdfCloudTripleStore();\nAccumuloRyaDAO crdfdao = new AccumuloRyaDAO();\ncrdfdao.setConnector(connector);\n\nAccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();\nconf.setTablePrefix(\"rya_\");\nconf.setDisplayQueryPlan(true);\ncrdfdao.setConf(conf);\nstore.setRyaDAO(crdfdao);\n\nInferenceEngine inferenceEngine = new InferenceEngine();\ninferenceEngine.setRyaDAO(crdfdao);\ninferenceEngine.setConf(conf);\nstore.setInferenceEngine(inferenceEngine);\n\nRepository myRepository = new RyaSailRepository(store);\nmyRepository.initialize();\n\nString query = \"select * where {\\n\" +\n                    \"\u003chttp://mynamespace/ProductType1\u003e ?p ?o.\\n\" +\n                    \"}\";\nRepositoryConnection conn = myRepository.getConnection();\nSystem.out.println(query);\nTupleQuery tupleQuery = conn.prepareTupleQuery(\n        QueryLanguage.SPARQL, query);\nValueFactory vf = SimpleValueFactory.getInstance();\n\nTupleQueryResultHandler writer = new SPARQLResultsXMLWriter(System.out);\ntupleQuery.evaluate(writer);\n\nconn.close();\nmyRepository.shutDown();\n```\n\n#### Web REST endpoint\n\nThe War sets up a Web REST endpoint at `http://server/web.rya/loadrdf` that allows POST data to get loaded into the Rdf Store. This short tutorial will use Java code to post data.\n\nFirst, you will need data to load and will need to figure out what format that data is in.\n\nFor this sample, we will use the following N-Triples:\n\n```\n\u003chttp://mynamespace/ProductType1\u003e \u003chttp://www.w3.org/1999/02/22-rdf-syntax-ns#type\u003e \u003chttp://mynamespace/ProductType\u003e .\n\u003chttp://mynamespace/ProductType1\u003e \u003chttp://www.w3.org/2000/01/rdf-schema#label\u003e \"Thing\" .\n\u003chttp://mynamespace/ProductType1\u003e \u003chttp://purl.org/dc/elements/1.1/publisher\u003e \u003chttp://mynamespace/Publisher1\u003e .\n```\n\nSave this file somewhere $RDF_DATA\n\nSecond, use the following Java code to load data to the REST endpoint:\n\n``` JAVA\nimport java.io.BufferedReader;\nimport java.io.InputStream;\nimport java.io.InputStreamReader;\nimport java.io.OutputStream;\nimport java.net.URL;\nimport java.net.URLConnection;\n\npublic class LoadDataServletRun {\n\n    public static void main(String[] args) {\n        try {\n            final InputStream resourceAsStream = Thread.currentThread().getContextClassLoader()\n                    .getResourceAsStream(\"$RDF_DATA\");\n            URL url = new URL(\"http://server/web.rya/loadrdf\" +\n                    \"?format=N-Triples\" +\n                    \"\");\n            URLConnection urlConnection = url.openConnection();\n            urlConnection.setRequestProperty(\"Content-Type\", \"text/plain\");\n            urlConnection.setDoOutput(true);\n\n            final OutputStream os = urlConnection.getOutputStream();\n\n            int read;\n            while((read = resourceAsStream.read()) \u003e= 0) {\n                os.write(read);\n            }\n            resourceAsStream.close();\n            os.flush();\n\n            BufferedReader rd = new BufferedReader(new InputStreamReader(\n                    urlConnection.getInputStream()));\n            String line;\n            while ((line = rd.readLine()) != null) {\n                System.out.println(line);\n            }\n            rd.close();\n            os.close();\n        } catch (Exception e) {\n            e.printStackTrace();\n        }\n    }\n}\n```\n\nCompile and run this code above, changing the references for $RDF_DATA and the url that your Rdf War is running at.\n\nThe default \"format\" is RDF/XML, but these formats are supported : RDFXML, NTRIPLES, TURTLE, N3, TRIX, TRIG.\n\n#### Bulk Loading data\n\nBulk loading data is done through Map Reduce jobs\n\n##### Bulk Load RDF data\n\n  This Map Reduce job will read a full file into memory and parse it into statements. The statements are saved into the store. Here is an example for storing in Accumulo:\n\n```\nhadoop jar target/accumulo.rya-3.0.4-SNAPSHOT-shaded.jar org.apache.rya.accumulo.mr.fileinput.BulkNtripsInputTool -Dac.zk=localhost:2181 -Dac.instance=accumulo -Dac.username=root -Dac.pwd=secret -Drdf.tablePrefix=triplestore_ -Dio.sort.mb=64 /tmp/temp.ntrips\n```\n\nOptions:\n\n* rdf.tablePrefix : The tables (spo, po, osp) are prefixed with this qualifier. The tables become: (rdf.tablePrefix)spo,(rdf.tablePrefix)po,(rdf.tablePrefix)osp\n* ac.* : Accumulo connection parameters\n* rdf.format : See RDFFormat from RDF4J, samples include (Trig, N-Triples, RDF/XML)\n* io.sort.mb : Higher the value, the faster the job goes. Just remember that you will need this much ram at least per mapper\n\nThe argument is the directory/file to load. This file needs to be loaded into HDFS before running.\n\n#### Direct RDF4J API\n\nHere is some sample code to load data directly through the RDF4J API. (Loading N-Triples data)\nYou will need at least accumulo.rya-\u003cversion\u003e, rya.api, rya.sail.impl on the classpath and transitive dependencies. I find that Maven is the easiest way to get a project dependency tree set up.\n\n``` JAVA\nfinal RdfCloudTripleStore store = new RdfCloudTripleStore();\nAccumuloRdfConfiguration conf = new AccumuloRdfConfiguration();\nAccumuloRyaDAO dao = new AccumuloRyaDAO();\nConnector connector = new ZooKeeperInstance(\"instance\", \"zoo1,zoo2,zoo3\").getConnector(\"user\", \"password\");\ndao.setConnector(connector);\nconf.setTablePrefix(\"rya_\");\ndao.setConf(conf);\nstore.setRyaDAO(dao);\n\nRepository myRepository = new RyaSailRepository(store);\nmyRepository.initialize();\nRepositoryConnection conn = myRepository.getConnection();\n\n//load data from file\nfinal File file = new File(\"ntriples.ntrips\");\nconn.add(new FileInputStream(file), file.getName(),\n        RDFFormat.NTRIPLES, new Resource[]{});\n\nconn.commit();\n\nconn.close();\nmyRepository.shutDown();\n```\n\n\n### Query Data\n\n#### Web JSP endpoint\n\nOpen a url to `http://server/web.rya/sparqlQuery.jsp`. This simple form can run Sparql.\n\n### Web REST endpoint\n\nThe War sets up a Web REST endpoint at `http://server/web.rya/queryrdf` that allows GET requests with queries.\n\nFor this sample, we will assume you already loaded data from the [loaddata.html] tutorial\n\nSave this file somewhere $RDF_DATA\n\nSecond, use the following Java code to load data to the REST endpoint:\n\n``` JAVA\nimport java.io.BufferedReader;\nimport java.io.InputStreamReader;\nimport java.net.URL;\nimport java.net.URLConnection;\nimport java.net.URLEncoder;\n\npublic class QueryDataServletRun {\n\n    public static void main(String[] args) {\n        try {\n            String query = \"select * where {\\n\" +\n                                \"\u003chttp://mynamespace/ProductType1\u003e ?p ?o.\\n\" +\n                                \"}\";\n\n            String queryenc = URLEncoder.encode(query, \"UTF-8\");\n\n            URL url = new URL(\"http://server/web.rya/queryrdf?query=\" + queryenc);\n            URLConnection urlConnection = url.openConnection();\n            urlConnection.setDoOutput(true);\n\n            BufferedReader rd = new BufferedReader(new InputStreamReader(\n                    urlConnection.getInputStream()));\n            String line;\n            while ((line = rd.readLine()) != null) {\n                System.out.println(line);\n            }\n            rd.close();\n        } catch (Exception e) {\n            e.printStackTrace();\n        }\n    }\n}\n```\n\nCompile and run this code above, changing the url that your Rdf War is running at.\n\n\n[Apache Rya]: http://rya.apache.org/ \n[Accumulo]: https://accumulo.apache.org/\n[ZooKeeper]: https://zookeeper.apache.org/\n[Hadoop]: http://hadoop.apache.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Frya","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Frya","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Frya/lists"}