Tensorboard
DAN relies on Tensorboard to log metrics and predictions. This allows you to monitor the progress of your training.
Access
To access Tensorboard, run tensorboard --logdir={output_folder}/results/ in your local terminal.
Then, go on http://localhost:6006 to visualize your training.
Metrics
Two metrics are commonly used to evaluate Automatic Text Recognition models.
- the Character Error Rate (CER) is the percentage of characters that have been transcribed incorrectly by the model.
- the Word Error Rate (WER) is the percentage of words that have been transcribed incorrectly by the model.
Usage
Seven metrics are computed on the train and validation set and logged in Tensorboard. In addition, 5 predictions are also logged.
Training metrics
Several metrics are computed on the training set:
-
train/{dataset}-train_loss_ce: the cross entropy loss function. -
train/{dataset}-train_cer: the CER. -
train/{dataset}-train_cer_no_token: the CER ignoring punctuation marks. -
train/{dataset}-train_ner: the CER ignoring characters (only NE tokens are considered). -
train/{dataset}-train_wer. the WER. -
train/{dataset}-train_wer_no_punct: the WER ignoring punctuation marks. -
train/{dataset}-train_wer_no_token: the WER ignoring Named Entity (NE) tokens (only characters are considered).
These metrics can be visualized in the Scalars tab in Tensorboard, under the train section.
Alternatively, you can find them in the Time series tab.
Validation metrics
The same metrics are computed on the validation set, except for the loss function:
-
val/{dataset}-val_cer: the CER. -
val/{dataset}-val_cer_no_token: the CER ignoring punctuation marks. -
val/{dataset}-val_ner: the CER ignoring characters (only NE tokens are considered). -
val/{dataset}-val_wer. the WER. -
val/{dataset}-val_wer_no_punct: the WER ignoring punctuation marks. -
val/{dataset}-val_wer_no_token: the WER ignoring Named Entity (NE) tokens (only characters are considered).
These metrics can be visualized in the Scalars tab in Tensorboard, under the valid section.
Alternatively, you can find them in the Time series tab.
Predictions on the validation set
Five validation images are also displayed at each epoch, along with their predicted transcription and CER and WER.
To log more or less images, update the training.validation.nb_logged_images parameter in the configuration file. The font and its size can also be changed.
To visualize them, go in the Image tab in Tensorboard.
Select an image to increase its size:
By default, the slider is set to the last validation step. You can move the cursor to view previous transcriptions on the same image:
