Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/wikidata/queryanalysis
Code for analysing Wikidata SPARQL query logs
https://github.com/wikidata/queryanalysis
Last synced: about 1 month ago
JSON representation
Code for analysing Wikidata SPARQL query logs
- Host: GitHub
- URL: https://github.com/wikidata/queryanalysis
- Owner: Wikidata
- License: apache-2.0
- Created: 2016-10-15T12:24:08.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2022-12-08T00:02:06.000Z (about 2 years ago)
- Last Synced: 2024-04-17T08:12:35.586Z (8 months ago)
- Language: Java
- Homepage: https://meta.wikimedia.org/wiki/Research:Understanding_Wikidata_Queries
- Size: 6.87 MB
- Stars: 10
- Watchers: 18
- Forks: 0
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# QueryAnalysis
[](https://opensource.org/licenses/Apache-2.0)
## Getting Started
### Prerequisites
You need to have `Maven`, `OpenJDK 8` and `Python 2` installed.
### Installing
```shell
$ mvn clean package
```
### Running the main Java log analyser
```shell
# Processes the example SPARQL log files into exampleMonthsFolder/exampleMonth/processedLogData
$ mvn exec:java@QueryAnalysis -Dexec.args="-w exampleMonthsFolder/exampleMonth -logging"
# There are more (useful) CLI parameter available, you can list them with:
$ mvn exec:java@QueryAnalysis -Dexec.args="--help"
```
**Important:** In order to not flush the command line with error messages all uncatched Runtime Exceptions are being written to the log files, residing in the logs/ folder, so please have a look at those regularly. The logs are not generated by default, so you should enable them using the `-l` option.
### Running the QueryAnalysis-Script
The QueryAnalysis-Script handles both steps: Extraction using hive and processing using the java application. Extraction using hive only works on the server, but is ignored if the month exists in the months folder. To run the QueryAnalysis-Script locally, you need to provide the local months folder.
```shell
$ python QueryAnalysis exampleMonth -m ../exampleMonthsFolder
# You can also specify multiple months in the same directory by separating them using commas
$ python QueryAnalysis exampleMonth,otherMonth -m ../exampleMonthsFolder
```
## License
The code in this repository is released under the [Apache 2.0](LICENSE.txt) license. External libraries used may have their own licensing terms.