Implement category-based CER/WER
Create a new command atr-ner-eval cer --label label.bio --pred pred.bio
to evaluate CER / WER for each category
Steps:
- for each ground truth/prediction pair
- for each category
- filter text by category
- compute CER/WER using the
editdistance
package
Expected output: a markdown table with CER/WER by category (you can use prettyTable)
| Category | CER (%) | WER (%) | N entities |
|----------|:-------:|:-------:|-----------:|
| person | 5.61 | 24.15 | 1501 |
| date | 3.50 | 19.24 | 62 |
| location | 1.35 | 12.40 | 101 |
| other | 4.65 | 21.36 | 15048 |
Edited by Yoann Schneider