Skip to content

Load a language model and decode with it during evaluation

This is needed for #213

Depends #251 (closed)

We will support additional keys in the evaluation configuration, to load a language model and set some hyperparameters.

New keys
{
    ...
    "model": {
        ...
        "lm": {
            "path": "", # path to the model i.e. language_model.arpa.gz or language_model.bin
            "weight": 1.0, # language_model_weight
            "temperature": 1.0 # not exposed in LM inference yet
        }
    }
    ...
}

All the language model files should be in the same folder, model.lm.path's parent is the path to that folder. The expected names should be written in the documentation and a verbose message must be displayed if there is a misnaming.

The LM decoder (named lm_decoder) must be loaded after the encoder/decoder in GenericTrainingManager.load_model. This decoder is not initialized, is not unpickled so the loading must happen after all that. Isolate the loading in a dedicated method, it will be quite long since you need to check for the file's existence. It should be loaded in the same way as during inference. Expected filenames are:

  • lexicon.txt for CTCLanguageDecoder.lexicon_path
  • tokens.txt for CTCLanguageDecoder.tokens_path

Once it's loaded, we need to use it decode the prediction.

  1. Store the predicted Tensor while predicting on the batch
  2. Decode using the lm_decoder and store the output under the str_lm key
  3. Store the value in the Inference class