Evaluate NER tokens distance
We need a new metric ner
that will evaluate the NER tokens chain. This metric will be automatically added when config["dataset"]["tokens"] is not None
, if not present already.
We will need a new MetricManager.format_string_for_ner
that will simply remove from the string all the text that is not a token. This is almost self.remove_tokens
.