From b96f58245bc1ca988e9b9486f5c6ea0c0a9f7aea Mon Sep 17 00:00:00 2001 From: manonBlanco <blanco@teklia.com> Date: Fri, 22 Dec 2023 11:15:01 +0100 Subject: [PATCH] Update documentation --- docs/usage/datasets/analyze.md | 2 -- docs/usage/evaluate/index.md | 48 +++++++++++++++++++++++++++++++--- docs/usage/predict/index.md | 4 +-- 3 files changed, 47 insertions(+), 7 deletions(-) diff --git a/docs/usage/datasets/analyze.md b/docs/usage/datasets/analyze.md index fec46d59..799774e8 100644 --- a/docs/usage/datasets/analyze.md +++ b/docs/usage/datasets/analyze.md @@ -4,8 +4,6 @@ Use the `teklia-dan dataset analyze` command to analyze a dataset. This will display statistics in [Markdown](https://www.markdownguide.org/) format. -The available arguments are: - | Parameter | Description | Type | Default | | --------------- | -------------------------------- | -------------- | ------- | | `--labels` | Path to the `labels.json` file. | `pathlib.Path` | | diff --git a/docs/usage/evaluate/index.md b/docs/usage/evaluate/index.md index f1cc273b..3764e245 100644 --- a/docs/usage/evaluate/index.md +++ b/docs/usage/evaluate/index.md @@ -1,18 +1,60 @@ # Evaluation +## Description + Use the `teklia-dan evaluate` command to evaluate a trained DAN model. To evaluate DAN on your dataset: 1. Create a JSON configuration file. You can base the configuration file off the training one. Refer to the [dedicated page](../train/config.md) for a description of parameters. 1. Run `teklia-dan evaluate --config path/to/your/config.json`. -1. Evaluation results for every split are available in the `results` subfolder of the output folder indicated in your configuration. -1. A metrics Markdown table, providing results for each evaluated split, is also printed in the console (see table example below). -### Example output - Metrics Markdown table +This will, for each evaluated split: + +1. Create a YAML file with the evaluation results in the `results` subfolder of the `training.output_folder` indicated in your configuration. +1. Print in the console a metrics Markdown table (see [table example below](#htr-evaluation)). +1. Print in the console a Nerval metrics Markdown table, if the `dataset.tokens` parameter in your configuration is defined (see [table example below](#htr-and-ner-evaluation)). + +| Parameter | Description | Type | Default | +| -------------------- | -------------------------------------------------------------------------------------------- | -------------- | ------- | +| `--config` | Path to the configuration file. | `pathlib.Path` | | +| `--nerval-threshold` | Distance threshold for the match between gold and predicted entity during Nerval evaluation. | `float` | `0.3` | + +## Example output + +### HTR evaluation + +| Split | CER (HTR-NER) | CER (HTR) | WER (HTR-NER) | WER (HTR) | WER (HTR no punct) | +| :---: | :-----------: | :-------: | :-----------: | :-------: | :----------------: | +| train | x | x | x | x | x | +| val | x | x | x | x | x | +| test | x | x | x | x | x | + +### HTR and NER evaluation | Split | CER (HTR-NER) | CER (HTR) | WER (HTR-NER) | WER (HTR) | WER (HTR no punct) | NER | | :---: | :-----------: | :-------: | :-----------: | :-------: | :----------------: | :-: | | train | x | x | x | x | x | x | | val | x | x | x | x | x | x | | test | x | x | x | x | x | x | + +#### train + +| tag | predicted | matched | Precision | Recall | F1 | Support | +| :-----: | :-------: | :-----: | :-------: | :----: | :-: | :-----: | +| Surname | x | x | x | x | x | x | +| All | x | x | x | x | x | x | + +#### val + +| tag | predicted | matched | Precision | Recall | F1 | Support | +| :-----: | :-------: | :-----: | :-------: | :----: | :-: | :-----: | +| Surname | x | x | x | x | x | x | +| All | x | x | x | x | x | x | + +#### test + +| tag | predicted | matched | Precision | Recall | F1 | Support | +| :-----: | :-------: | :-----: | :-------: | :----: | :-: | :-----: | +| Surname | x | x | x | x | x | x | +| All | x | x | x | x | x | x | diff --git a/docs/usage/predict/index.md b/docs/usage/predict/index.md index b96149db..823399de 100644 --- a/docs/usage/predict/index.md +++ b/docs/usage/predict/index.md @@ -1,9 +1,9 @@ # Prediction -Use the `teklia-dan predict` command to apply a trained DAN model on an image. - ## Description +Use the `teklia-dan predict` command to apply a trained DAN model on an image. + | Parameter | Description | Type | Default | | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | -------------- | ------------- | | `--image-dir` | Path to the folder where the images to predict are stored. Must not be provided with `--image`. | `pathlib.Path` | | -- GitLab