Skip to content
Snippets Groups Projects
Verified Commit f6577084 authored by Yoann Schneider's avatar Yoann Schneider :tennis:
Browse files

Support the case where the model can predict NER entities but doesn't in that case

parent c9e8d46f
No related branches found
No related tags found
1 merge request!457Support the case where the model can predict NER entities but doesn't in that case
......@@ -44,7 +44,8 @@ def compute_offsets_by_level(full_text: str, level: Level, text_list: List[str])
"""
# offsets[idx] = number of characters between text_list[idx-1] and text_list[idx]
offsets = [int(level != Level.Char)] * (len(text_list) - 1)
if level == Level.NER:
# Take entities into account if there are any
if level == Level.NER and text_list:
# Start after the first entity
cursor = len(text_list[0])
for idx, split in enumerate(text_list[1:]):
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment