Masking metabolite identifiers - A user manual for MetMask

• Incorporate an arbitrary list with any kind of identifier and keep track of which identifier is associated with which by assigning them to internal mask identifiers.
  • Keep track of the connection information, i.e. which source was used for which connection.
  • Annotate connections as weak or non-weak depending on if they are ambiguous or not.
  • Fusing different masks if input data implies that the masks are the same.
• Import widely used databases such as the main NIST library, the KEGG and PlantCyc compounds files.
• Query the database and extract one or all related identifiers for any known identifier.
• Query PubChem and KEGG for identfiers that are not in the local database and save any new associations.
• Output in different formats, including graphs which can easily be visualized using e.g. cytoscape or R (with Rgraphviz).

Metmask in its current form does not:

• Provide a way to identify metabolites based on their Mass spectra, chromatographical retention indexes or similar. Metmask can however associate ambiguous information (such as mass forumlae) and provides a way to search for these.
• Ensure that the identifiers mapped to by the database is correct in terms of what the original creators of the identifiers intended. Metmask only knows what is being imported into the database and it holds that if you put garbage in you get garbage out.

A great advantage of metmask compared to other databases ¹ such as more comprehensive databases such as PubChem is that

• it is easy to tailor to only load the desired sources in the database so that it matches the desired platform. For e.g. GC-MS metabolomics data, this would imply that all stereo-isomers of one metabolite should be behind the same mask since stereo-isomers usually are not resolved.
• comparing to online queries metmask is very fast
• combining different databases is easy whereas online databases may be lacking the source of interest (e.g. one cannot directly query for AraCyc using PubChem but this is easily done in metmask).
  • connections which requires intermediate identifiers is also straight-forward in metmask.

A word of caution, metmask assumes certain basic skills working with a shell. If you do wish to use metmask but feel that you need a graphical user environment, then please post a message on the user forum at http://metmask.sourceforge.net.

To import the KEGG compounds file directly from the KEGG FTP server:

metmask -i kegg

To import a comma delimited file, test.csv, with identifiers structured as

myId,synonym,weak:type
a1,alanine,amino-acid
a2,ala,amino-acid
a1,ala,amino-acid
a3,sucrose,sugar
a4,lysine,amino-acid

we call

metmask --import test.csv --parser simple --master myId --db /tmp/test-db

The first line in a test.csv names the types of identifiers. These will afterwards be possible query using the -t switch e.g.

metmask --query a1 --table myId --goal synonym --db /tmp/test-db

'"alanine"|"ala"'

The input comma delimited file should be structured as:

tablename1 , <confidence code>:tablename2 , tablename3
table1-id1 , table2-id1                   , table3-id1|table3-id2|table3-id3
table1-id1 , table2-id1                   , table3-id1|table3-id2|table3-id3
...

and is in the example above read by the parser "simple". Note that the confidence code can be used to specify whether a table is "weak" or not. In the example, test.csv, type is used to set a metabolite type but this type is not used to group metabolites, only to annotate them. See *Query for further details.

To query the database for everything it knows about alanine::

metmask --query alanine --goal ALL --output mask -d /tmp/test-db

o-o-o-o:
mask:
[1]
myId:
a1 test.csv 3 test.csv 3
a2 test.csv 3
synonym:
alanine test.csv 3
ala test.csv 3 test.csv 3
preferred:
2:ala test.csv 0

or to get all metabolites we specified as type "amino-acid" in a format that is easy to import to other programs.

metmask --query amino-acid --table type --goal myId,synonym -d /tmp/test-db

'"a1"|"a2"','"alanine"|"ala"'
'"a4"','"lysine"'

Note that the input has been masked so that "a2" also is associated with "alanine" even though this was never explicitly specified in the input.

The import also specified a 'master' identifier which is interpreted as the source for connections. In this case, each line in test.csv is therefore read as myId connects to synonym. When a master identifier has been set, we can produce graph like output:

metmask --query a1 --table myId --goal ALL --output graph --db /tmp/test-db -u

myId:a1       synonym:alanine test.csv        False
myId:a1       synonym:ala     test.csv        False
myId:a1       type:amino-acid test.csv        True
myId:a2       synonym:ala     test.csv        False
myId:a2       type:amino-acid test.csv        True

which gives the previous list (which can be imported to Cytoscape for visualization, simply click import from file and set the first column as source node and second column as target node.). The columns are Node 1, Node 2, Source of edge and an indicator for whether the connection should be considered "weak" or not. The "-u" switch tells metmask to also output weak identifiers.

If you have a file of identifiers that you want to query the database for KEGG identifiers you can simply use standared shell re-direction and query as:

cat myFileWithIdentifiers | metmask -g kegg

As an import proceeds metmask ensures that a single identifier only maps to a single mask. If one tries to do an import that matches an already existing mask a conflict arises and the following alternatives are available to resolve the problem:

• Merge the new information to the exiting mask.
• Annotate the overlapping information as 'weak' and create a new mask.

To minimize errors the following rule-set is used to judge if two masks should be merged or not.

1. Two masks coming from the same source are compatible if they share a non-weak identifier.
2. Two masks coming from different sources are compatible if they share at least n types of identifiers, where n is user-defined.
3. Two masks are not compatible if they are associated with non-equal sum-formulae (ignoring single protons).
4. Two masks are not compatible if both carry an indentifiers annotated with 'nevermerge'.

The confidence code is an code that which defines whether masks with those confidence codes are allowed to be merged or not. The code is either an arbitrary string chosen upon import such as "good" or "bad" depending on the quality of the source or one of the reserved codes:

nevermerge

Two masks that carry identifiers tagged with ``nevermerge`` will, as the name suggests never be merged.

weak

An indentifier tagged as ``weak`` does not count when counting overlap between two masks. All identifier are unique, but when adding a mask carrying e.g. the identifier G will not be added to the Glycine mask which also carries G if G was tagged as ``weak``.

The parsers are managed in a plug-in like system. See the source package metmask/parse for examples, by putting a new parser and naming it "_newparser.py" in that directory, it immediately ready to use. See following example for how a parser should be structured.

from metmask.mask import mask
import metmask.parse 
from main import fileFormatError
# function for fixing a string to a vector considering quotations etc
from main import fixLine

class parser :
    """ < your documentation >
    """

    # constructor
    def __init__ (self, parent) :
        # parent is the 'main' class of parser which handles file
        # reading and communication with the metmask database

        # make sure to set the table that you use, these are used to
        # create corresponding tables in the database e.g.
        parent.tables = ['cas', 'mylocalid']
        self.parent = parent

    # the function that does the actual reading
    def process (self) :
        parent = self.parent
        # get a line of input
        ll = parent.getLine()
        while ll :
            # create a mask object
            un = mask({}, parent.mm.idpatterns) 
            # append the information to the mask (see
            # documentation of the mask object)
            un.append(<table Type>, ll[0], ll[1], ll[2])
            # send the mask to the database
            parent.setMask(un)
            # get a new line
            ll = parent.getLine()