Skip to content
Snippets Groups Projects
Commit 00bcb965 authored by Manon Blanco's avatar Manon Blanco
Browse files

Merge branch 'parse-offset-ner-no-entities' into 'main'

Support the case where the model can predict NER entities but doesn't in that case

See merge request !457
parents 1ad17b1f f6577084
No related branches found
No related tags found
1 merge request!457Support the case where the model can predict NER entities but doesn't in that case
......@@ -44,7 +44,8 @@ def compute_offsets_by_level(full_text: str, level: Level, text_list: List[str])
"""
# offsets[idx] = number of characters between text_list[idx-1] and text_list[idx]
offsets = [int(level != Level.Char)] * (len(text_list) - 1)
if level == Level.NER:
# Take entities into account if there are any
if level == Level.NER and text_list:
# Start after the first entity
cursor = len(text_list[0])
for idx, split in enumerate(text_list[1:]):
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment