Skip to content

Implement tokens generation commands

We will need two new commands:

  • teklia-dan dataset entities, to generate an intermediate entities.yml with the list of entities in the dataset. The user can remove some lines to avoid predicting some entities.
  • teklia-dan dataset tokens, to generate the tokens.yml with random tokens. The user can choose to have ending tokens or not (in this case, no end key at all in the YAML). We will stick with unicode circled characters?

More info:

Edited by Manon Blanco