Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/SouthGreenPlatform/AgroLD_ETL
AgroLD is a RDF knowledge base that consists of data integrated from a variety of plant resources and ontologies. AgroLD ETL is the Python packages developed to transform plant datasets in RDF. Packages are developped for data standards such as GFF,GAF, VCF and specific plant databases.
https://github.com/SouthGreenPlatform/AgroLD_ETL
etl gaf gff ontologies rdf rdf-data vcf
Last synced: 4 months ago
JSON representation
AgroLD is a RDF knowledge base that consists of data integrated from a variety of plant resources and ontologies. AgroLD ETL is the Python packages developed to transform plant datasets in RDF. Packages are developped for data standards such as GFF,GAF, VCF and specific plant databases.
- Host: GitHub
- URL: https://github.com/SouthGreenPlatform/AgroLD_ETL
- Owner: SouthGreenPlatform
- License: gpl-3.0
- Created: 2018-11-22T06:00:49.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-04-22T12:28:28.000Z (10 months ago)
- Last Synced: 2024-04-22T12:30:26.675Z (10 months ago)
- Topics: etl, gaf, gff, ontologies, rdf, rdf-data, vcf
- Language: Python
- Homepage:
- Size: 1.81 MB
- Stars: 1
- Watchers: 11
- Forks: 1
- Open Issues: 40
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-starred - SouthGreenPlatform/AgroLD_ETL - AgroLD is a RDF knowledge base that consists of data integrated from a variety of plant resources and ontologies. AgroLD ETL is the Python packages developed to transform plant datasets in RDF. Packag (others)
README
# AgroLD ETL
AgroLD is a RDF knowledge base that consists of data integrated from a variety of plant resources and ontologies. AgroLD ETL is the Python packages developed to transform plant datasets in RDF. Packages are developped for data standards such as GFF,GAF, VCF and specific plant databases.
# Contact
* pierre.larmande_at_ird.fr
# Valorization and How to cite
* [https://doi.org/10.1371/journal.pone.0198270](https://doi.org/10.1371/journal.pone.0198270)
* [https://tel.archives-ouvertes.fr/IBC/hal-01176903v1](https://tel.archives-ouvertes.fr/IBC/hal-01176903v1)
* [http://ceur-ws.org/Vol-1546/poster_55.pdf](http://ceur-ws.org/Vol-1546/poster_55.pdf)
# Contributing
* Intellectual property belongs to IRD, CIRAD and SouthGreen development platform.
* Written by Aravind Venkatesan, Gildas Tagny Ngompe, Nordine El Hassouni, Manuel Ruiz, Pierre Larmande.
* Copyright 2014-2020
# The Architecture of project
AgroLD project is composed of two component: [AgroLD_ETL](/AgroLD_ETL) and [agrold_webapp](/agrold_webapp)
* The first component is a set of Parser and wrapper for translate a dataset. Follow this link for to know what data have been translated in RDF: [Documentation](http://volvestre.cirad.fr:8080/agrold/documentation.jsp)
* The second component is the web application who is connected at the triple store for to make queries.
***
```
AgroLD_ETL
-> model
-> rdf_ttl
-> riceKB
-> test_files
-> riceKBpipeline.py
```
***
### AgroLD_ETL
Contains parsers and model used to convert data considered for AgroLD to RDF.
* [model](/AgroLD_ETL/model): All documents who describe how data are transformed
* [rdf_ttl](/AgroLD_ETL/rdf_ttl): All output of transformation sort by dataset
* [riceKB](/AgroLD_ETL/riceKB): Contains scripts used for each data set
* [test_files](/AgroLD_ETL/test_files): All test files in input ( heterogeneous format: csv, tabbed files, gff3 )
* [riceKBpipeline.py](/AgroLD_ETL/riceKBpipeline.py): Script file where we have centralised all execution
The type of each dataset is different, GFF, HapMap, CSV and VCF. In first time we have developed parser for build a dictonary,
because is easy to browse a dictionary and create the RDF
# How to use AgroLD_ETL
For example if you want to execute a gff3 parser for to build a dictionary.
define a input file
```
AgroLD/AgroLD_ETL/riceKB/gffParser.py
path = '/os_file_gff3/file.gff3' # The input
ds = parseGFF3(path) # The parsing file
```
> **The dictionary :** The GFF3 Parser is a generic fonction who build a dictionary, it easy to browse this dictionary for build RDF data
```
{ 'attributes': { 'Dbxref': 'InterPro:IPR005333,InterPro:IPR017887',
'ID': 'BGIOSGA000770-TA',
'Name': 'BGIOSGA000770-TA',
'Parent': 'BGIOSGA000770'},
'end': 35414873,
'phase': None,
'score': None,
'seqid': 'Osi01',
'source': 'glean',
'start': 35413950,
'strand': '-',
'type': 'mRNA'},
{ 'attributes': { 'Derives_from': 'BGIOSGA000770-TA',
'ID': 'BGIOSGA000770-TA_protein',
'Name': 'BGIOSGA000770-TA'},
'end': 35414873,
'phase': None,
'score': None,
'seqid': 'Osi01',
'source': 'glean',
'start': 35413950,
'strand': '-',
'type': 'polypeptide'},
{ 'attributes': { 'Parent': 'BGIOSGA000770-TA'},
'end': 35414873,
'phase': '0',
'score': None,
'seqid': 'Osi01',
'source': 'glean',
'start': 35413950,
'strand': '-',
'type': 'CDS'},
{ 'attributes': { 'Parent': 'BGIOSGA000770-TA'},
'end': 35414873,
'phase': None,
'score': None,
'seqid': 'Osi01',
'source': 'glean',
'start': 35413950,
'strand': '-',
'type': 'exon'},
```
### Documentation
- AgroLD includes data on the following species on : [Species](http://volvestre.cirad.fr:8080/agrold/documentation.jsp#species)
- Ontologies in AgroLD : [Ontologies](http://volvestre.cirad.fr:8080/agrold/documentation.jsp#ontologies)
- Data sources in AgroLD : [Data](http://volvestre.cirad.fr:8080/agrold/documentation.jsp#sources)
- Species specific break down of the data sources : [Link](http://volvestre.cirad.fr:8080/agrold/documentation.jsp#break-down)
- Graph Names : [Link](http://volvestre.cirad.fr:8080/agrold/documentation.jsp#graphs)
- URIs : [Link](http://volvestre.cirad.fr:8080/agrold/documentation.jsp#uri)