https://github.com/seandavi/map_to_mesh_with_mti
Use the MTI tooling from NCBI to map UMLS and MESH terms to arbitrary text.
https://github.com/seandavi/map_to_mesh_with_mti
bioinformatics docker dockerfile machine-learning medical-subject-headings medline natural-language-processing nlp
Last synced: about 1 month ago
JSON representation
Use the MTI tooling from NCBI to map UMLS and MESH terms to arbitrary text.
- Host: GitHub
- URL: https://github.com/seandavi/map_to_mesh_with_mti
- Owner: seandavi
- Created: 2020-03-09T01:13:35.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2020-08-19T14:09:49.000Z (almost 6 years ago)
- Last Synced: 2025-04-09T06:16:16.805Z (about 1 year ago)
- Topics: bioinformatics, docker, dockerfile, machine-learning, medical-subject-headings, medline, natural-language-processing, nlp
- Language: Java
- Homepage:
- Size: 1.09 MB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.html
Awesome Lists containing this project
README
Semantic Knowledge Representation Web API Overview
<!--- hide script from old browsers
function displayMsg(text) {
msgWindow=window.open("", "message",
"screenX=300,screenY=300,outerWidth=500,outerHeight=500,scrollbars=yes,resizable=yes")
msgWindow.document.close();
msgWindow.document.open();
msgWindow.document.write();
msgWindow.document.write ("<head><title>Message Window</title></head>")
msgWindow.document.write ("<body bgcolor=white>")
msgWindow.document.write ("<form method=post>")
msgWindow.document.write ("<big>" + text + "<br><br>")
msgWindow.document.write ("<center>")
msgWindow.document.write ("<input type=button value=OK onClick=\"window.close();\">")
msgWindow.document.write ("</center></form></body>")
}
function toggleDisplay(options, which) {
if(options[0].selected) {
var block = document.getElementById(which);
block.style.display = "none";
}
else {
var block = document.getElementById(which);
block.style.display = "block";
}
}
// end hiding from old browsers -->
Contents:
Whenever you see
in the text below, it means that additional
information is available by selecting the symbol.
Introduction:
This Java-based API to the Semantic Knowledge Representation (SKR) Scheduler facility was created to provide users with the ability to programmatically submit jobs to the Scheduler Batch and Interactive facilities instead of using the web-based interface.
Background:
The SKR Project was initiated at NLM in order to develop programs to provide usable semantic representation of biomedical free text by building on resources currently available at the library.
The SKR project is concerned with reliable and effective management of the information encoded in natural language texts. The project develops programs that provide usable semantic representation of biomedical text by building on resources currently available at the Library, especially the UMLS knowledge sources and the natural language processing tools provided by the SPECIALIST system.
Two programs in particular, MetaMap and SemRep, are being evaluated and enhanced and applied to a variety of problems in the management of biomedical information. These include automatic indexing of MEDLINE citations, concept-based query expansion, analysis of complex Metathesaurus strings, accurate identification of the terminology and relationships in anatomical documents, and the extraction of chemical binding relations from biomedical text.
Third Party and Associated Licenses Used:
For specifics on the licensing and copyright information on each of these
packages, please visit their respective link. Notices/Attributions will
also be found in the source code when and where required.
- The http*.jar files are from the Apache
HttpComponents® project
(http://hc.apache.org/)
which is under the Apache License, Version 2.0
(http://www.apache.org/licenses/).
- The Json libraries are from JSON.org (http://json.org/) which uses the
JSON License (http://www.json.org/license.html)
- The Password Masking in the Java Programming Language software is from
http://java.sun.com/developer/technicalArticles/Security/pwordmask/.
Author: Qusay H. Mahmoud with contributions from Alan Sommerer
- Information on the CAS RESTful API Protocol used by this API can be found at:
https://wiki.jasig.org/display/CASUM/RESTful+API .
This software is produced by an agency of the U.S. Government, and
by statute is not subject to copyright in the United States. Recipients
of this software assume all responsibilities associated with its
operation, modification, and maintenance.
Users are responsible for compliance with the UMLS Metathesaurus
License Agreement.
To use the SKR Web API, you must have accepted the terms of the
UMLS Metathesaurus
License Agreement, which requires you to respect the copyrights
of the constituent vocabularies and to file a brief annual report on
your use of the UMLS. You also must have activated
a UMLS Terminology Services (UTS)
account. For information on how we use UTS authentication please select
the Info icon to the right:
For details of the licenses see the
UMLS Metathesaurus
License Agreement and
How to License
and Access the Unified Medical Language System� (UMLS�) Data.
- A running version of Java.
We have tested and confirmed that the SKR Web API
program compiles and runs under the following versions of Java:
- 1.6.0
- 1.6.0
- We have tested and confirmed that the SKR Web API program runs on the
following operating systems:
- Linux
- Microsoft Windows XP, 7
- Mac OS/X
- Linux
- Apache's Jakarta Project Ant
program is only required if you wish to modify
and/or compile the SKR Web API source code using the
existing build file:
- 1.6 or better
- 1.6 or better
- Your data in a recognized format. Please see the
Supported
File Formats section of the SKR Help Information web page for
detailed information and examples of valid input formats.
Installation:
To install the SKR Web API, do the following:
- Make a top level directory to store the SKR Web API program.
- Go to the SKR Web API page
from the SKR homepage.
- Download the latest version of the SKR Web API into the
directory created in #1.
- Change directory to the directory created in #1
- Extract the jar files from the downloaded jar file - SKR_Web_API_V2_0.jar.
-
On Windows:Run winzip, pkzip, or zip on the
downloaded file.
-
On Unix:Run the java jar program on the
jar file: java sun.tools.jar.Main xf SKR_Web_API_V2_0.jar
-
- This will create a directory called "SKR_Web_API_V2_0"
in the directory you created in #1 and
fill it with all of the program files needed.
- Now, change to this new directory: cd SKR_Web_API_V2_0
- You are now ready to run. Look at the example programs in the
"examples" subdirectory for ideas on what to do.
The SKR Web API has been designed to allow you to interact with our web-based
Scheduler using our Batch and Interactive facilities. Interactive access is currently only
available for the MetaMap and SemRep programs.
-
Batch Jobs have three main phases:
- Request is made for a new batch job and the SKR Web API submits the
job.
- Program waits for job to finish. This is done by monitoring the
Scheduler queue to see when the job exits the queue. The program
sleeps for 10 seconds after each check of the queue so that it
doesn't become a nuisance process.
- Once the job has left the Scheduler queue, the program then checks
to see if the "text.out.done" file exists in the job result
directory. If it does, then the results file "text.out" is returned
as a String to the calling method. If the "text.out.done" file
doesn't exist, an error has occurred and the program exits.
- Request is made for a new batch job and the SKR Web API submits the
-
Interactive Jobs only have two main phases:
- Request is made for a new interactive job and the SKR Web API submits the
job.
- Program waits for job to finish and returns the result as a String
to the calling method. Interactive jobs don't really enter the
Scheduler - they are run from the command line. The API simply
waits for the command to return a result without having to worry
about the Scheduler queue, etc., providing the fastest mechanism for
receiving results.
- Note: Interactive jobs will only accept a single
item up to 10,000 characters in length to process - anything
larger will generate an error.
- Request is made for a new interactive job and the SKR Web API submits the
There are six example programs included in the "examples" directory to illustrate how to incorporate the SKR Web API into your application. Two of the examples (GenericBatch.java and GenericBatchNew.java) show how to access the Scheduler Batch facility. MMInteractive and SRInteractive.java examples show how to access Interactive MetaMap and Interactive SemRep respectively. The last two examples GenericBatchUser.java and MMInteractiveUser.java illustrate how to
specify the username and password within the program to eliminate the need for
prompting when running.
For a list of the available options, please select the appropriate program name:
MetaMap and
MTI.
GenericBatch.java
Example program for submitting a new Generic Batch with Validation job
("GenericObject(true)" turns on validation) request to the Scheduler to
run. You will be prompted for your username and password and if they
are alright, the job is submitted to the Scheduler and the results are
returned in the String "results" below.
This example shows how to setup a basic Generic Batch with Validation job
with a small file (sample.txt) with ASCII MEDLINE formatted citations as
input data. You must set the Email_Address variable and use the
UpLoad_File
to specify the data to be processed. This example also shows the user
setting the SilentEmail option which tells the Scheduler to NOT send
email upon completing the job.
This example is set to run the MTI (Medical Text Indexer) program using
the -opt1L_DCMS and -E options. You can also setup any environment
variables
that will be needed by the program by setting the Batch_Env field.
NOTE:
The "-E" option/argument is very important and should be included
with whatever program (MetaMap, SemRep, MTI) you decided to run!
The reason is that this
version of the Generic Batch does validation and the -E option tells
the various programs to include a marker denoting when a successful
result has been found.
GenericBatchNew.java
Same as GenericBach.java above, except you can enter any of the
options and the inputfile on the command line. The usage of the
program and available options are listed below:
usage: GenericBatchNew [options] inputFilename
allowed options:
--email <address> : set email address. (required option)
--command <name> : batch command: metamap, semrep, etc. (default: MTI -opt1_DCMS -E)
--note <notes> : batch notes
--silent : don't send email after job completes.
--silent-errors : Silent on Errors
--singleLineInput : Single Line Delimited Input
--singleLinePMID : Single Line Delimited Input w/ID
--priority : request a Run Priority Level: 0, 1, or 2
NOTE:
The "-E" option/argument is very important and should be included
with whatever program you decided to run! The reason is that this
version of the Generic Batch does validation and the -E option tells
the various programs to include a marker denoting when a successful
result has been found.
MMInteractive.java
This example shows how to setup a basic Interactive MetaMap request.
This runs the latest version of MetaMap with 1011 (2010AB) version
of the UMLS Metathesaurus ("KSOURCE", "1011"), with "ignore_word_order" (-i)
and "all_derivational_variants" (D) set as arguments to MetaMap. The
default "Human Readable" output will be produced.
NOTE:
The "-E" option/argument is NOT required for Interactive use.
SRInteractive.java
This example shows how to setup a basic Interactive SemRep request.
This runs the latest version of SemRep with Full Fielded Output (-D).
NOTE:
The "-E" option/argument is NOT required for Interactive use.
GenericBatchUser.java
This example is the same as "GenericBatchNew.java" above, except that the
username and password are set in the program so that the user is not
prompted when the program runs. Please Note: Although we understand
the operational need for being able to specify the username and password
within a program, you must also be aware of the security risks when using
this setup. Please take care to secure the software you create using this
option.
MMInteractiveUser.java
This example is the same as "MMInteractive.java" above, except that the
username and password are set in the program so that the user is not
prompted when the program runs. Please Note: Although we understand
the operational need for being able to specify the username and password
within a program, you must also be aware of the security risks when using
this setup. Please take care to secure the software you create using this
option.
- Default MetaMap call (latest version of MetaMap, strict data model,
current UMLS Knowledge Source, human readable output,
and best mappings only):
metamap -E
- MetaMap formatted XML output call (latest version of MetaMap,
strict data model, current UMLS Knowledge Source, Formatted XML output,
and best mappings only) showing both short and long form of the
XML option:
metamap -% format -E
or
metamap --XML format -E
- Specify Semantic Types to Restrict MetaMap too (latest version of
MetaMap, strict data model, current UMLS Knowledge Source,
human readable output, best mappings only, only allow "Disease or
Syndrome", "Finding", and "Injury or Poisoning" Semantic Types):
metamap -J "dsyn,fndg,inpo" -E
- Default MTI call (latest version of MTI, use PubMed Related Articles
first and TexTool as backup for PRC, and default output):
MTI -opt1L_DCMS -E
- Default SemRep call (latest version of SemRep with Full Fielded Output):
semrep -D -E
- To compile the source jar file and/or any of the examples using Ant
- change to the top-level directory and run:
ant jar
- To compile any of the examples without Ant, change to
the examples directory first:
cd examples
Unix:
javac -cp ../classes:../lib/skrAPI.jar:../lib/commons-logging-1.1.1.jar:../lib/httpclient-cache-4.1.1.jar:../lib/httpcore-nio-4.1.jar:../lib/httpclient-4.1.1.jar:../lib/httpcore-4.1.jar:../lib/httpmime-4.1.1.jar -d ../classes GenericBatch.java
or, you can use the "compile.sh" script provided
../compile.sh GenericBatch.javaWindows:
javac -cp ..\classes;..\lib\skrAPI.jar;..\lib\commons-logging-1.1.1.jar;..\lib\httpclient-cache-4.1.1.jar;..\lib\httpcore-nio-4.1.jar;..\lib\httpclient-4.1.1.jar;..\lib\httpcore-4.1.jar;..\lib\httpmime-4.1.1.jar -d ..\classes GenericBatch.java
- To run any of the examples, change to the examples directory first:
cd examples
Unix:
java -cp ../classes:../lib/skrAPI.jar:../lib/commons-logging-1.1.1.jar:../lib/httpclient-cache-4.1.1.jar:../lib/httpcore-nio-4.1.jar:../lib/httpclient-4.1.1.jar:../lib/httpcore-4.1.jar:../lib/httpmime-4.1.1.jar GenericBatch
java -cp ../classes:../lib/skrAPI.jar:../lib/commons-logging-1.1.1.jar:../lib/httpclient-cache-4.1.1.jar:../lib/httpcore-nio-4.1.jar:../lib/httpclient-4.1.1.jar:../lib/httpcore-4.1.jar:../lib/httpmime-4.1.1.jar GenericBatchNew --email your@email inputFilename
or, you can use the "run.sh" script provided
../run.sh GenericBatch
../run.sh GenericBatchNew --email your@email inputFilenameWindows:
java -cp ..\classes;..\lib\skrAPI.jar;..\lib\commons-logging-1.1.1.jar;..\lib\httpclient-cache-4.1.1.jar;..\lib\httpcore-nio-4.1.jar;..\lib\httpclient-4.1.1.jar;..\lib\httpcore-4.1.jar;..\lib\httpmime-4.1.1.jar GenericBatchjava -cp ..\classes;..\lib\skrAPI.jar;..\lib\commons-logging-1.1.1.jar;..\lib\httpclient-cache-4.1.1.jar;..\lib\httpcore-nio-4.1.jar;..\lib\httpclient-4.1.1.jar;..\lib\httpcore-4.1.jar;..\lib\httpmime-4.1.1.jar GenericBatchNew --email your@email filename
Required Fields
General Description
Name
Type
Email Address
Email_Address
String
File to Upload
UpLoad_File
String
Optional Fields
General Description
Name
Type
Silent on Errors
ESilent
Boolean
Single Line Delimited Input
SingLine
Boolean
Single Line Delimited Input w/ID
SingLinePMID
Boolean
Requested Run Priority Level
RPriority
String
No Completion Email
SilentEmail
Boolean
- You "should" catch "RuntimeException" exceptions in your program
since this is what the SKR Web API throws internally when it encounters
an error. See example below for try/catch sample.
- You must specify an Email Address. This is used for logging as well
as notification when Batch jobs finish, or if there are problems with
a job.
- You must specify the UpLoad_File field.
- Valid Batch job priorities are "0", "1", or "2". Where "0" represents
Normal priority, "1" represents Medium, and "2" represents High
priority. Medium and High priorities are reserved and you must have the
proper permissions to use them.
- Explanation of setting up a basic SKR Web API program. The following
description details the steps to setting up a simple SKR Web API job to
send a small file to the Scheduler and receive a result back.
See the GenericBatch.java program file in the examples directory for the
full program.
- The first thing you need to do is create a job object for the type
of program you want to run.
GenericObject myGenericObj = new GenericObject();
This creates a Generic Batch job object with Validation.
- The next thing to do is setup any of the required or batch specific
fields.
myGenericObj.setField("Email_Address", "yourEmailAddress");
myGenericObj.setField("SilentEmail", true);
myGenericObj.setField("Batch_Command", "MTI -opt1L_DCMS -E");
myGenericObj.setField("UpLoad_File", "./sample.txt");
This tells the job object what your email address is (required),
tells the Scheduler that we don't want to be notified after the
job completes (SilentEmail), we want to run the MTI command with
"-opt1L_DCMS -E" as options, and that we want to use the local
file "sample.txt" as the input data to be processed (UpLoad_File).
- Now we are finally ready to actually submit the Batch job and
receive our results back. We surround this submission code with
a try/catch for any "RuntimeException" exceptions that might be
thrown back to us from the SKR Web API program.
try
{
String results = myGenericObj.handleSubmission();
System.out.print(results);
} catch (RuntimeException ex) {
System.err.println("");
System.err.print("An ERROR has occurred while processing your");
System.err.println(" request, please review any");
System.err.print("lines beginning with \"Error:\" above and the");
System.err.println(" trace below for indications of");
System.err.println("what may have gone wrong.");
System.err.println("");
System.err.println("Trace:");
ex.printStackTrace();
} // catch
This calls the job object routine that takes care of the job from
submission, monitoring, cleaning up, and returning a result.
- The first thing you need to do is create a job object for the type