Parse a tokens yaml instead of hardcoding the tokens

Closes #127 (closed) Extracting the entities confidence scores during prediction requires using the project tokens, this adds a --tokens argument. The yaml is then parsed to get the tokens' list.

dan/predict/prediction.py

@@ -359,7 +359,7 @@ def process_batch(
             text = result["text"]
             tokens = parse_tokens(tokens)
             start_tokens, end_tokens = zip(*list(tokens.values()))
-            end_tokens = list(filter(lambda item: item != "", end_tokens))
+            end_tokens = list(filter(bool, end_tokens))
 
             if len(end_tokens):
                 assert len(start_tokens) == len(