- Open Access
Text-mining and information-retrieval services for molecular biology
Genome Biology volume 6, Article number: 224 (2005)
Text-mining in molecular biology - defined as the automatic extraction of information about genes, proteins and their functional relationships from text documents - has emerged as a hybrid discipline on the edges of the fields of information science, bioinformatics and computational linguistics. A range of text-mining applications have been developed recently that will improve access to knowledge for biologists and database annotators.
The use of large-scale experimental techniques and bioinformatic tools has increased the pace at which biologists produce relevant information. This also promotes the growth of the scientific literature, which contains information on those experimental results in the form of free text that is structured in a way that makes it straightforward for humans to read but more difficult for computers to interpret automatically. As a consequence, there is increasing interest in methods that can handle collections of biological texts. Such methods include systems that efficiently retrieve and classify documents in response to complex user queries, and beyond this, systems that carry out a deeper analysis of the literature to extract specific associations, such as protein-protein interactions and protein functions. This deeper analysis is called text-mining. The complex and concise nature of the scientific literature means that the use of text-mining tools developed for generic texts is often impractical; a set of freely available text-mining applications adapted to the needs of biology have been developed, however, and some of them are now available for practical use. In parallel, a number of strategies for evaluating text-mining applications have appeared, with the goal of assessing and improving the field by providing datasets that can be used for training and testing applications.
Finding relevant articles
Throughout the last decade, the amount of electronically accessible textual material has been growing exponentially. Internet-based technologies exploit the availability of these large collections of documents for the development of information-retrieval systems. Currently, biologists and bioinformaticians take advantage of those tools, not only when searching generic documents such as news articles using search engines such as Alta Vista  and Google , but especially when querying publications specific to biomedicine, for example those stored in PubMed [3, 4]. The range of community-wide genome projects, for which Internet-based information exchange is crucial, together with the heavy use of biology databases through web-based tools, means that natural language processing (NLP) techniques could be useful. NLP is based on the use of computers to process language, and it includes techniques developed to provide the basic methodology required for automatically extracting relevant functional information from unstructured data, such as scientific publications. Information retrieval and NLP systems are soon likely to become important not only for extracting information but also for assisting in various aspects of research such as the discovery of new facts, the interpretation of findings, and the design of experiments.
One of the first steps when handling textual data is the extraction of relevant documents from a large collection. This process is commonly known as information retrieval. In the case of indexed web pages, powerful search engines such as Google  return a ranked list of documents relevant to a given user search. There are two basic search strategies: query-based and document-based searches. In query-based searches, documents are returned that contain certain user-specified combinations of keywords. As some words - 'stop words' such as 'and', 'if' and 'the' - are found at a high frequency within most documents and thus display a low information content, they are often excluded during the retrieval process. Keywords may be combined by Boolean operators, such as AND, OR and NOT. The second type of retrieval, document-based searching, aims to return a ranked list of documents similar to a given query document as a whole, rather than to a combination of a few keywords. The most widely used retrieval tool in molecular biology is Entrez [3, 4], the PubMed information retrieval system provided at the US National Center for Biotechnology Information (NCBI) . It supports basic keyword and Boolean query-based searches, as well as document-based searches to return all abstracts that are similar to a given document. The popular search engine Google  has recently incorporated a search tool specific to the academic literature, Google Scholar [6, 7], for the retrieval of scientific articles, reports and books. The ranking of the returned hits is mainly based on the extent to which documents are connected by citations and web links. Other scientific literature databases and search engines include Crossref Search , which enables searches of the full content provided by a set of publishers, and the Nature Publishing Group search engine , which allows advanced search strategies.
Although these tools are useful for many tasks, it is time-consuming to use them for efficient searches and article selection, and such functions must be repeated periodically to keep knowledge up-to-date. As PubMed already contains over 15 million citations of biomedical articles  and is steadily growing (more than 450,000 articles are added every year ), services that periodically retrieve relevant articles and automatically alert the user have been implemented. Among those systems, known as selective dissemination of information (SDI) services, are My NCBI (formerly PubMed Cubby) [4, 11], BioMail  and PubCrawler [13, 14] (these and other services described in this article are listed in Table 1). These, together with some commercial tools, have been evaluated independently , showing that the combined use of different SDI systems results in useful automated searching.
The first step in text mining: identification of biological entities
Biological research is name-centered: proteins are referred to in free text by their names or symbols rather than using the unambiguous identifiers provided by annotation databases (such as SwissProt accession numbers ). Identifying mentions of proteins and genes unambiguously within free text is a fundamental step for the later extraction of functional attributes of these entities. Unfortunately this is a difficult process, partly because of the complex nature and usage of gene and protein names. Genes and proteins maybe referred to in free text in a range of different ways: as full names (for example, porin), as symbols (the Saccharomyces cerevisiae gene POR1), and also through typographical variants (POR-1). Many genes also have several synonyms (such as OMP2 for POR1), or the gene name may be ambiguous  and refer to words that also have a different meanings depending on the context (for example, big brain, the full name for the Drosophila melanogaster gene bib, could also be an anatomical description). Furthermore, it has been suggested that errors in gene names might be introduced automatically by certain applications in bioinformatics .
In the NLP field, the identification of entities in free text is known as named-entity recognition (NER). To identify biological entities such as genes, proteins and drugs automatically and unambiguously within free text, over 50 information-extraction and text-mining tools have recently been implemented, and two community-wide evaluations have been carried out [19, 20]. The top left of Figure 1 shows nine existing NER applications for biology that are provided via an online server or are directly downloadable. Note that the average recovery of biological entities from free text by 15 NER tools was 80%, and the results had an accuracy of 80% ; these figures are significantly lower than in the case of entities found in documents from fields such as economics, which demonstrates the complex nature of protein names.
Proteins and genes are characterized within biological databases through unique identifiers; each identifier is associated with its corresponding protein or nucleotide sequence and functional descriptions. The automatic recognition of entities such as genes and proteins in free text is insufficient if it is not linked to the corresponding database identifiers. Distinguishing between the use of protein names and protein-family names constitutes a serious obstacle in the task of highlighting protein entities in free text, as text passages sometimes refer to the general properties of protein families and at other times to the properties of individual proteins.
Different research communities have addressed the issue of named-entity recognition in biology in different ways. The NLP community has typically tried to identify names by analyzing the syntactic structure of sentences, making use of information about parts of speech in a sentence and the syntactic roles of words, whereas bioinformaticians have instead explored the identification of variants of the names contained in databases, even adapting standard bioinformatics algorithms such as BLAST to the problem of protein-name identification . Neither of these two strategies seems to be efficient by itself, and many intermediate combinations are therefore appearing, including the following examples. GAPSCORE [23, 24] is an easy-to-use online tool for detecting protein and gene names within free text (a 'protein tagger'). The text to be analyzed can be pasted into an online form and submitted to the server, which returns a list of the words observed in the document and a statistical quality score that indicates how probable it is that the each word represents a gene or protein name. Another online protein tagger is NLProt, developed at Columbia University [25, 26]. NLProt is based on a machine learning technique called support vector machines (SVMs) and allows protein identification either in a submitted text or in the text corresponding to a list of submitted PubMed article identifiers. Additional protein taggers include Yapex [27, 28], also available online, and three downloadable tools, AbGene [29, 30], ABNER [31, 32] and KEX [33, 34]. Abbreviations or acronyms are often used as a shorter form to refer to gene names in articles; the Abbreviation Server [35, 36] developed at Stanford University allows a similar search strategy to that used by GAPSCORE to be applied to biomedical abbreviations such as gene symbols. Finally, the AliasServer [37, 38] helps in linking the various aliases of a given gene through different biological databases for various species.
One of the main challenges when linking protein names to database entries is distinguising between proteins that have the same names but belong to different genomes - a process called inter-species gene disambiguation. This is especially cumbersome in the case of mouse and human genes; the same gene symbol is often used in both species and both names are often mentioned in the same textual passage. The complex nature of protein- and gene-name identification is reinforced further by the dynamic nature of gene-name usage and name creation, with official gene names being changed and new synonyms being created ; it is clear that static approaches and dictionaries will not be sufficient for solving the problem.
One step further: mining interactions and relations
Although the identification of biological entities is a crucial step, in practice it is the extraction of associations between proteins and their functional features that poses an interesting biological problem. Several systems have been constructed for extracting annotations of genes and proteins automatically and for detecting protein-protein interactions and regulatory pathways. Protein-protein interactions have attracted particular interest in the light of recent developments in high-throughput proteomics. One system that extracts annotations and detects interactions is the iHOP system that we have implemented at the Spanish National Biotechnology Center . This facilitates the direct linking of information in the INTACT  protein-interaction database with corresponding bibliographic references (Figure 2). As well as highlighting direct associations between genes and functional descriptions, iHOP also includes advanced search modes for discovery and visualization of literature-based protein-interaction networks for a range of organisms, including human, mouse and yeast . The basic approach followed by iHOP is protein-centric: it arranges relevant sentences from the literature around protein names, and the use of co-citation of protein names in each sentence facilitates navigation through the dispersed literature relevant to a particular protein. As a result, users can successively explore the functions of related proteins by building virtual protein-relation networks (Figure 2c). The iHOP system is based on the ideas previously developed for the SUISEKI knowledge-discovery system .
Some other text-mining applications include PreBIND [44, 45], developed to assist in the extraction of protein-protein interactions; the KinasePathway database text-mining system, which extracts interactions between proteins, genes and compounds [46, 47]; and Textpresso [48, 49], an information-retrieval and extraction tool developed for the Caenorhabditis elegans literature in the context of the model-organism database WormBase . Textpresso defines 33 categories of word describing entities or relationships - such as genes, pathways, or regulation - and integrates this 'Textpresso Ontology' with a text-mining system for searching the C. elegans literature. Among the text-mining services available online that focus on automatic annotation extraction are GOAnnotator, which provides associations between protein names and Gene Ontology terms ; KAT [52, 53], a system for deriving terms relevant to annotations such as SwissProt keywords and Gene Ontology terms  from PubMed abstracts for a given query protein; and the iProLINK tool [55, 56], which performs automated extraction of annotations for given protein names and provides information related to the organisms in which proteins are found and the protein families of which they are members. Figure 1 and Table 1 provide an overview of the different systems currently available.
A system with a special focus on the extraction of relationships between genes, proteins and other information is Chilibot ([57, 58]; user registration is required before running queries); it allows searches using gene symbols and keywords, and the color-coded output provides information about gene-expression levels when available. The extraction of complex relationships can be handled by GeneScene [59, 60], a toolkit that provides visualization and navigation facilities for exploring regulatory networks; the tool currently provides information only on the literature on yeast and on the p53 tumor suppressor and the AP1 transcription factor.
Some attempts have been made to merge text-mining methods and bioinformatic methods involving sequence analysis into a single system. The integration of functional information extracted by NLP algorithms with standard bioinformatic methods such as sequence-comparison techniques has been exploited by the Structure Assignment With Text Description (SAWTED) system [61, 62], which can be tested online. It combines a document-comparison algorithm called a 'Vector-cosine model' with the PSI-BLAST sequence retrieval method, which is especially useful for detecting sequences that are distantly related. Another strategy that makes use of sequence information and free text is MedBlast [63, 64]; using the web-based interface of MedBlast, for a given query sequence and optional additional keywords the system returns articles related to the protein corresponding to the query sequence.
Text mining and large gene collections
Technical advances in molecular biology mean that large collections of genes are nowadays often studied simultaneously using genomic approaches. Using conventional information retrieval to link these genes with the associated literature is not efficient, and a large list of irrelevant documents can be returned. For example, microarray experiments result in groups of genes with particular expression patterns; to interpret these groups in terms of the underlying biological meaning, information is needed not only on each individual gene but also on commonalities among the whole group. The functional information is commonly extracted from databases such as SwissProt  or GO , which in turn are nourished by extracting relevant functional features from the literature.
A number of text-mining methods have been developed for linking groups of genes found in microarrays and other experiments directly and automatically with information contained in biomedical article databases. The neighbor divergence per gene (NDPG) approach  uses the literature to score the functional coherence of gene clusters. GEISHA [67, 68] automatically mines the literature for functional terms associated with gene groups and carries out a statistical analysis of the significance of those terms. Among the available online tools for assisting in interpreting microarray data are MedMiner [69, 70], which can be used to filter and organize information from free text obtained from automatic PubMed  and GeneCard  searches and PubGene [72, 73] which has additional visualization capabilities for displaying network information and pathway mapping. The analysis of frequency matrices of term cooccurrences of two lists of keywords is the basis of the PubMatrix system [74, 75], which can be used online after registering. Finally, microGENIE  enables semi-automatic queries of very large collections of genes (UniGene and SwissProt gene names and GenBank accession numbers) in PubMed to speed up the retrieval of relevant articles. It is important to realize that existing text-mining technologies in biology are focused on identification and linking of functional information of proteins in free text, they are currently not providing automatically generated summaries of biologically relevant information.
Towards knowledge discovery
The field of 'BioNLP' - text mining and information extraction for molecular biology - is very recent, but the existing applications are improving steadily. This is partly because of newly available resources, such as collections of annotated documents suitable for training new systems (for example, the GENIA  corpus and the BioCreative  corpus). The improvement also reflects the effect of community-wide assessments such as the BioCreative contest  and the KDD challenge cup , which enable evaluation of the efficiency of different methodologies, and the genomics track of the Text Retrieval Conference (TREC) workshops [79, 80], a forum for developing solutions to information-retrieval and document-classification tasks in biology. The development of controlled, computer-readable vocabularies (ontologies), dictionaries, and functional keywords (Gene Ontology concepts  and SwissProt keywords ) defining relevant biological aspects of proteins have also been valuable for text-mining tools. Because of the restricted availability of full-text articles most of the existing text-mining systems for biology are centered on the analysis of abstracts, but changes in publishing policy and increasing access to repositories of whole articles make mining of full text a likely development in the near future. Some initiatives in this direction have been started already, for example the BioRAT system [81, 82], which processes full-text articles so as to identify target facts.
Perhaps the most likely future developments will be the construction of networks and interactions for discovering new relationships through intermediate entities, followed by the proposal of new functions - this process is referred to as 'knowledge discovery'. Several exploratory attempts have been made to develop knowledge-discovery systems, but they are not yet of general practical use. Our SUISEKI system , for instance, extracts indirect relationships between proteins through associations with intermediate proteins in text. Two online tools that directly address the difficulty of making knowledge-discovery practical are ARROWSMITH [84, 85] and BITOLA [86, 87]. ARROWSMITH [84, 85] aims to discover indirect relations between two entities that are not directly connected in the literature; the indirect relationship can be a substance or disease condition. BITOLA [86, 87] is a biomedical discovery-support system with a focus on the discovery of disease candidate genes, taking advantage of Medical Subject Heading (MeSH) terms.
Undoubtedly, the development of text-mining applications specific for biology is the only way to cope with the increasing amount of free textual data produced in this field. The increasing interest of users in efficiently retrieving and extracting relevant information, the need to keep up with new discoveries described in the literature or in biological databases, and the demands posed by the analysis of high-throughput experiments, are the underlying forces motivating the development of text-mining applications in molecular biology. Those technologies should provide the foundation for future knowledge-discovery tools able to identify previously undiscovered associations, something that will assist in the formulation of models of biological systems.
Schuler G, Epstein J, Ohkawa H, Kans J: Entrez: molecular biology database and retrieval system. Methods Enzymol. 1996, 266: 141-162.
Entrez PubMed. [http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed]
Wheeler D, Church D, Federhen S, Lash A, Madden T, Pontius J, Schuler G, Schriml L, Sequeira E, Tatusova T, Wagner L: Database resources of the National Center for Biotechnology. Nucleic Acids Res. 2003, 31: 28-33. 10.1093/nar/gkg033.
Editorial: The ultimate search engine?. Nat Cell Biol. 2005, 7: 1-
Google Scholar. [http://scholar.google.com]
CrossRef Search, publisher pilot for full-text scholarly research. [http://www.crossref.org/crossrefsearch.html]
Nature Publishing Group search engine. [http://search.nature.com/search/?sp_a=sp1001702d&sp_t=advanced&sp_x_1=ujournal&sp-p=all&sp]
Staab S, Blaschke C, Nedellec C, Park J, Schatz B, Valencia A, Bernardi L, Ratsch E, Kania R, Saric J, Rojas I, Staab S: Mining information for functional genomics. IEEE Intelligent Systems. 2002, 17: 66-80. 10.1109/5254.988491.
Knecht L, Shooshan S: Internet Grateful Med to be retired; reminder of NLM Gateway availability. NLM Tech Bull. 2001, 318: e3-
Hokamp K, Wolfe K: PubCrawler: keeping up comfortably with PubMed and GenBank. Nucleic Acids Res. 2004, 32: W16-W19. 10.1093/nar/gnh017.
Shultz M, DeGroote S: MEDLINE SDI services: how do they compare?. J Med Libr Assoc. 2003, 91: 460-467.
Expasy - SwissProt and TrEMBL. [http://us.expasy.org/sprot]
Chen L, Liu H, Friedman C: Gene name ambiguity of eukaryotic nomenclatures. Bioinformatics. 2005, 21: 248-256. 10.1093/bioinformatics/bth496.
Zeeberg B, Riss J, Kane D, Bussey K, Uchio E, Linehan W, Barrett J, Weinstein J: Mistaken identifiers: gene name errors can be introduced inadvertently when using Excel in bioinformatics. BMC Bioinformatics. 2004, 5: 80-10.1186/1471-2105-5-80.
Hirschman L, Yeh A, Blaschke C, Valencia A: Overview of BioCreAtIvE: critical assessment of information extraction for biology. BMC Bioinformatics. 2005, 6 (Suppl 1): S1-10.1186/1471-2105-6-S1-S1.
Kim J, Ohta T, Tsuruoka Y, Tateisi Y: Introduction to the bioentity recognition task at JNLPBA. Proceedings of the Joint Workshop on Natural Language Processing in Biomedicine and its Applications Geneva. 70-76. 28-29 August 2004, [http://www.genisis.ch/~natlang/JNLPBA04/JNLPBA.final.pdf]
Yeh A, Morgan A, Colosimo M, Hirschman L: BioCreAtIvE task 1A: gene mention finding evaluation. BMC Bioinformatics. 2005, 6 (Suppl 1): S2-10.1186/1471-2105-6-S1-S2.
Krauthammer M, Rzhetsky A, Morozov P, Friedman C: Using BLAST for identifying gene and protein names in journal articles. Gene. 2000, 259: 245-252. 10.1016/S0378-1119(00)00431-5.
Chang J, Schutze H, Altman R: GAPSCORE: finding gene and protein names one word at a time. Bioinformatics. 2004, 20: 216-225. 10.1093/bioinformatics/btg393.
Gene and Protein Name Server. [http://bionlp.stanford.edu/gapscore]
Mika S, Rost B: NLProt: extracting protein names and sequences from papers. Nucleic Acids Res. 2004, 32: W634-W637.
CUBIC: NLProt/Index. [http://cubic.bioc.columbia.edu/services/nlprot]
Franzen K, Eriksson G, Olsson F, Asker L, Liden P, Coster J: Protein names and how to find them. Int J Med Inform. 2002, 67: 49-61. 10.1016/S1386-5056(02)00052-7.
Tanabe L, Wilbur W: Tagging gene and protein names in biomedical text. Bioinformatics. 2002, 18: 1124-1132. 10.1093/bioinformatics/18.8.1124.
Settles B: Biomedical named entity recognition using conditional random fields and rich feature sets. Proc. NLPBA/COLING 2004. 2004
ABNER: a biomedical named entity recognizer. [http://www.cs.wisc.edu/~bsettles/abner]
Fukuda K, Tsunoda T, Tamura A, Takagi T: Toward information extraction: identifying protein names from biological papers. Pac Symp Biocomput. 1998, 3: 707-718.
Chang J, Schuetze H, Altman R: Creating an online dictionary of abbreviations from MEDLINE. J Am Med Inform Assoc. 2002, 9: 612-620. 10.1197/jamia.M1139.
Biomedical Abbreviation Server. [http://bionlp.stanford.edu/abbreviation]
Iragne F, Barre A, Goffard N, DeDaruvar A: AliasServer: a web server to handle multiple aliases used to refer to proteins. Bioinformatics. 2004, 20: 2331-2332. 10.1093/bioinformatics/bth241.
Hoffmann R, Valencia A: Life cycles of successful genes. Trends Genet. 2003, 19: 79-81. 10.1016/S0168-9525(02)00014-8.
Hoffmann R, Valencia A: A gene network for navigating the literature. Nat Genet. 2004, 36: 664-10.1038/ng0704-664.
Hermjakob H, Montecchi-Palazzi L, Lewington C, Mudali S, Kerrien S, Orchard S, Vingron M, Roechert B, Roepstorff P, Valencia A, et al: IntAct: an open source molecular interaction database. Nucleic Acids Res. 2004, 32: D452-D455. 10.1093/nar/gkh052.
Information hyperlinked over proteins (iHOP). [http://www.pdg.cnb.uam.es/UniPub/iHOP]
Blaschke C, Valencia A: The frame-based module of the Suiseki information extraction system. IEEE Intelligent Systems. 2002, 17: 14-20.
Donaldson I, Martin J, deBruijn B, Wolting C, Lay V, Tuekam B, Zhang S, Baskin B, Bader G, Michalickova K, et al: PreBIND and Textomy - mining the biomedical literature for protein-protein interactions using a support vector machine. BMC Bioinformatics. 2003, 4: 11-10.1186/1471-2105-4-11.
BIND - The Biomolecular Interaction Network. [http://bind.ca]
Koike A, Kobayashi Y, Takagi T: Kinase pathway database: an integrated protein-kinase and NLP-based protein-interaction resource. Genome Res. 2003, 13: 1231-1243. 10.1101/gr.835903.
Kinase Pathway database. [http://kinasedb.ontology.ims.u-tokyo.ac.jp]
Muller H, Kenny E, Sternberg P: Textpresso: an ontology-based information retrieval and extraction system for biological literature. PLoS Biol. 2004, 2: e309-
Perez A, Perez-lratxeta C, Bork P, Thode G, Andrade M: Gene annotation from scientific literature using mappings between keyword systems. Bi. Bioinformoatics. 2004, 20: 2084-2091. 10.1093/bioinformatics/bth207.
An Introduction to the Gene Ontology. [http://www.geneontology.org/GO.doc.shtml]
Hu Z, Mani I, Hermoso V, Liu H, Wu C: iProLINK: an integrated protein resource for literature mining. Comput Biol Chem. 2004, 28: 409-416. 10.1016/j.compbiolchem.2004.09.010.
Che H, Sharp B: Content-rich biological network constructed by mining PubMed abstracts. BMC Bioinformatics. 2004, 5: 147-10.1186/1471-2105-5-147.
Leroy G, Chen H: Filling preposition-based templates to capture information from medical abstracts. Pac Symp Biocomput. 2002, 350-361.
MacCallum R, Kelley L, Sternberg M: SAWTED: structure assignment with text description-enhanced detection of remote homologues with automated SWISS-PROT annotation comparisons. Bioinformatics. 2000, 16: 125-129. 10.1093/bioinformatics/16.2.125.
Tu Q, Tang H, Ding D: MedBlast: searching articles related to a biological sequence. Bioinformatics. 2004, 20: 75-77. 10.1093/bioinformatics/btg375.
Al-Shahrour F, Diaz-Uriarte R, Dopazo J: FatiGO: a web tool for finding significant associations of Gene Ontology terms with groups of genes. Bioinformatics. 2004, 20: 578-580. 10.1093/bioinformatics/btg455.
Raychaudhuri S, Altman R: A literature-based method for assessing the functional coherence of a gene group. Bioinformatics. 2003, 19: 396-401. 10.1093/bioinformatics/btg002.
Oliveros J, Blaschke C, Herrero J, Dopazo J, Valencia A: Expression profiles and biological function. Genome Inform Ser Workshop Genome Inform. 2000, 11: 106-117.
DNA Array Analysis with Geisha. [http://www.pdg.cnb.uam.es/blaschke/cgi-bin/geisha]
Tanabe L, Scherf U, Smith L, Lee J, Hunter L, Weinstein J: MedMiner: an Internet text-mining tool for biomedical information, with application to gene expression profiling. Biotechniques. 1999, 27: 1210-1217.
Jenssen T, Laegreid A, Komorowski J, Hovig E: A literature network of human genes for high-throughput analysis of gene expression. Nat Genet. 2001, 28: 21-28. 10.1038/88213.
Becker K, Hosack D, Dennis G, Lempicki R, Bright T, Cheadle C, Engel J: PubMatrix: a tool for multiplex literature mining. BMC Bioinformatics. 2003, 4: 61-10.1186/1471-2105-4-61.
Kim JD, Ohta T, Tateisi Y, Tsujii J: GENIA corpus - semantically annotated corpus for biotextmining. Bioinformatics. 2003, 19: i180-i182. 10.1093/bioinformatics/btg1023.
Yeh A, Hirschman L, Morgan A: Evaluation of text data mining for database curation: lessons learned from the KDD Chal lenge Cup. Bioinformatics. 2003, 19 (Supp11): i331-i339. 10.1093/bioinformatics/btg1046.
Hersh W, Bhupatiraju R: TREC GENOMICS track overview. Proceedings of the Twelfth Text Retrieval Conference 18-21 November Gaithersburg. Edited by: Voorhees EM, Buckland LP. 2003, Gaithers burg: National Institute of Standards and Technology, 14-24.
TREC Genomics Trach. [http://ir.ohsu.edu/genomics]
Corney D, Buxton BF, Langdon W, Jones D: BioRAT: extracting biological information from full-length papers. Bioinformatics. 2004, 20: 3206-3213. 10.1093/bioinformatics/bth386.
Blaschke C, Valencia A: The potential use of SUISEKI as a protein interaction discovery tool. Genome Inform Ser Workshop Genome Inform. 2001, 12: 123-134.
Smalheiser N, Swanson D: Using ARROWSMITH: a computer-assisted approach to formulating and assessing scientific hypotheses. Comput Methods Programs Biomed. 1998, 57: 149-153. 10.1016/S0169-2607(98)00033-9.
Hristovski D, Peterlin B: Literature-based disease candidate gene discovery. Proceedings of Medinfo 2004. Edited by: Fieschi M. 2004, Bethesda: American Medical Informatics Association, 1649-
BITOLA - Biomedical Discovery Support System. [http://www.mf.uni-lj.si/bitola]
Wren J, Garner H: Heuristics for identification of acronym-definition patterns within text: towards an automated construction of comprehensive acronym-definition dictionaries. Methods Inf Med. 2002, 41: 426-434.
ARGH - Biomedical Acronym Resolver. [http://invention.swmed.edu/argh]
Relationship Extractor. [http://www-personal.engin.umich.edu/~murthyr/Relationship_Extractor.html]
Perez-lratxeta C, Bork P, Andrade M: XplorMed: a tool for exploring MEDLINE abstracts. Trends biochem Sci. 2001, 26: 573-575. 10.1016/S0968-0004(01)01926-0.
The work of our group was supported by grants from the European Commission (ORIEL IST-2001-32688, TEMBLOR QLRT-2001-00015, Biosapiens LSHC-CT-2003-505265). We thank Robert Hoffmann for providing Figure 2 and Christian Blaschke, as well as all the members of the group, for interesting discussions.