Skip to content

Incorrect WER computation in `display_worst_results`

When evaluating DAN, the WER computed in the 5 worst prediction(s) section is incorrect. In this case it should be 0.

Image name WER Alignment between ground truth - prediction
0c03f08f-0da5-4827-9fff-05c891567aa9.jpg 8.79 18.1Ⓔ sⒶ fⓘ ∅ CⒼ 21Ⓒ 770Ⓚ xxⓊ ∅ ∅ Marieⓔ POULETⓈ Jeanⓒ Marieⓐ ECARDⓆ
18.1Ⓔ sⒶ fⓘ ∅ CⒼ 21Ⓒ 770Ⓚ xxⓊ ∅ ∅ Marieⓔ POULETⓈ Jeanⓒ Marieⓐ ECARDⓆ

This is because the wrong value is stored in inferences. We are saving the micro-averaged WER (display_values), when it should be the WER corresponding to the current batch.

inferences.extend(
    map(
        Inference,
        batch_data["names"],
        batch_values["str_y"],
        batch_values["str_x"],
        batch_values.get("str_lm", repeat("")),
        repeat(display_values["wer"]),
     )
)