Usage

BENT can be used for:

➡️ Named Entity Recognition (NER)

➡️ Named Entity Linking (NEL)

➡️ Named Entity Recognition and Linking (NER+NEL)

Contents

Named Entity Recogniton (NER)

BENT includes 10 NER models. Each model corresponds to PubMedBERT fine-tuned on specific datasets.

The models and the description of the training datasets are stored in Hugging Face. The available models are:

  • ‘disease’ or ‘NILDis’: disease entities.

  • ‘chemical’ or ‘NILChem’: chemical entities.

  • ‘gene’ or ‘NILGene’: gene and protein entities.

  • ‘organism’: organism (species, genus, …) entities.

  • ‘bioprocess’: biological process entities (as defined by Gene Ontology-Biological Process sub-ontology)

  • ‘anatomical’: anatomical entities

  • ‘cell_component’: cell component entities

  • ‘cell_type’: cell type entities

  • ‘cell_line’: cell line entities

  • ‘variant’: variant entities (a DNA-level or protein-level mutation as defined by the Human Genome Variation Society nomenclature).

Note

In the current version, the ouput of both NER and NEL is only available for the BRAT standoff format.

Text files as input (only NER)

To recognize disease entities in text (NER) files placed under the directory ‘input/’ set the arguments:

  • recognize: indicate that the NER module will be applied (‘True’)

  • types: the entity types to be recognized in the text.

  • in_dir: directory path containing the text files to be annotated (the directory must contain text files exclusively)

  • out_dir: the output directory that will contain the annotation files

Example:

import bent.annotate as bt

bt.annotate(
        recognize=True,
        types={'disease': ''},
        in_dir='input/txt/',
        out_dir='output/ner/'
)

The annotations files (‘.ann’) will be under the directory ‘output/ner/’.

Note

Ensure that the only files in the ‘input_dir/txt/’ are the text files that you want to annotate.

List of texts as input (only NER)

To recognize entities in texts (NER) provided in an input list set the arguments:

  • recognize: indicate that the NER module will be applied (‘True’)

  • types: the entity types to be recognized in the text.

  • input_text: the text to be annotated (list of strings - each string is a separate document)

  • out_dir: the output directory that will contain the annotation files

Example:

import bent.annotate as bt

# Each element of the list corresponds to a different document text
txt_list = [
    'Caffeine is a central nervous system (CNS) stimulant. It is hypothesized that caffeine innhibits the activation NF-kappaB in A2058 melanoma cells.',
    'The Parkinson disease is a degenerative disorder affecting the motor system. PARK2 gene expression is associated with the Parkinson disease.']

bt.annotate(
        recognize=True,
        types={
            'disease': '',
            'chemical': '',
            'gene': '',
            'anatomical': '',
            'cell_line': '',
            'bioprocess': ''
               },
        input_text=txt_list,
        out_dir='output/ner/'
)

For each input text will be generated an annotation files (‘.ann’), placed under the directory ‘output/ner’.

The output will follow the BRAT format:

doc1.ann:

T1  chemical 0 7    Caffeine
T2  anatomical 13 35    central nervous system
T3  anatomical 37 40    CNS
T4  chemical 77 85  Caffeine
T5  gene 111 120    NF-kappaB
T1  chemical 0 8    Caffeine
T6  cell_line 124 129 A2058

doc2.ann:

T1  disease 3 21    Parkinson disease
T2  anatomical 63 75    motor system
T3  gene 77 82  PARK2
T4  bioprocess 83 98    gene expression
T5  disease 122 139 Parkinson disease

Single text as input (only NER)

To recognize entities in a text (NER) provided as a string and to output the annotations as a ‘dataset’ object set the arguments:

  • recognize: indicate that the NER module will be applied (‘True’)

  • types: the entity types to be recognized in the text.

  • input_text: the text to be annotated (string)

Note that in this case, the argument ‘out_dir’ is ommitted, as we want a ‘dataset’ object to be returned.

Example:

import bent.annotate as bt

txt1 = "Reed's syndrome has several manifestations and symptoms."

dataset = bt.annotate(
        recognize=True,
        types={'disease': ''},
        input_text=txt1,
)

It is possible to access the resulting ‘dataset’ object:

for doc in dataset.documents:
    print(doc.id)

    for entity in doc.entities:
        print(entity.type)
        print(entity.text)

Which will output:

disease
Reed's syndrome

Note

In cases where the input is a string or a list of strings instantiated in the execution script set the argument ‘input_text’. Instead, if the input is text present in external files set the argument ‘in_dir’.

Named Entity Linking (NEL)

BENT includes pre-process dictionaries that allow the linking of recognized entities of different types to entries of the following knowledge bases/graphs/ontologies:

Named Entity Recognition and Linking (NER+NEL)

To apply the complete pipeline of entity extraction (NER+NEL) set the arguments:

  • recognize: indicate that the NER module will be applied (‘True’)

  • link: indicate that the NEL module will be applied (‘True’)

  • types: entity types to recognize and the respective target knowledge bases.

  • in_dir: directory path containing the text files to be annotated (the directory must contain text files exclusively)

  • out_dir: the output directory that will contain the annotation files

Example:

import bent.annotate as bt

bt.annotate(
        recognize=True,
        link=True,
        types={
         'disease': 'medic'
         'chemical': 'chebi',
         'gene': 'ncbi_gene',
         'anatomical': 'uberon',
         'cell_line': 'cellosaurus',
         'bioprocess': 'go_bp'
         },
        in_dir='input/txt/',
        out_dir='output/nel/'
)

It is also possible to apply the pipeline (NER+NEL) to a string or a list or strings, as described above in the sections Single text as input (only NER) and List of texts as input (only NER). The only extra step: set the argument link to ‘True’.

Upload custom knowledge base/graph/ontology

If you want to use a custom knowledge base that is not included in the availabe options, it is necessary to have two text files: terms.txt and edges.txt.

The file terms.txt is a list of the entries of the custom knowledge base with the format:

ID:1  Entry 1  Synonym1;Synonym2;Synonym3
ID:2  Entry 2
ID:3  Entry 3  Synonym1;

Where the first colum corresponds to the identifier defined in knowledge base, the second column corresponds to the text associated with the entry/concept, and the third columns corresponds to the text of the synonym(s) associated with the entry.

edges.txt is a list of is-a (child-parent) relations between the entries of the terms.txt file:

ID:1  ID:3
ID:2  ID:3

Note

In both files, elements in each line are separated by a tab (’\t’).

Run the following code to generate the files for the custom knowledge base, indicating the filenames (terms_filename and edges_filename) and the desired name (kb_name), which in this example is ‘disease_KG’:

import  bent.src.dicts.kb.generate_dicts  as  bkb

bkb.generate(
        custom=True,
        kb_name='disease_KG',
        terms_filename='terms.txt',
        edges_filename='edges.txt',
        input_format='txt'
)

After this step, you can apply the above pipelines using the newly generated knowledge base by setting the dictionary ‘types’:

import bent.annotate as bt

bt.annotate(
        recognize=True,
        link=True,
        types={'disease': 'disease_KG'},
        in_dir='input/txt/',
        out_dir='output/nel/'
)