Compute confidence scores by char, word or line

Closes #33 (closed)

changed milestone to %ML Prod - Next

added P2 Quick Win labels

assigned to @starride

added 1 commit

• 828a5db1 - compute word/line confidence scores

Compare with previous version

added 1 commit

• 8ced0010 - fix lint

Compare with previous version

requested review from @schneider-y

I added the parameter --confidence-score-levels to compute confidence scores averaged by word and/or lines.

The following command will compute confidence scores for each line and for each word.

teklia-dan predict --image dan_humu/example.jpg --model dan_humu/model.pt --parameters dan_humu/parameters.yml --charset dan_humu/charset.pkl --output dan_humu/out/ --scale 0.5 --confidence-score --confidence-score-levels [line, word]

Output JSON:

{
    "text": "Hansteensgt. 2 IV 28/4 - 19\nKj\u00e6re Gerhard.\nTak for Brevet om Boken og Haven\nog Crokus og Blaaveis og tak fordi\nDu vilde be mig derut sammen\nmed Kris og Ragna. Men vet Du\nda ikke, at Kris reiste med sin S\u00f8-\nster Fru Cr\u00f8ger til Lillehammer\nnogle Dage efter Begravelsen? Hen\ndes Adresse er Amtsingeni\u00f8r\nCr\u00f8ger. Hun skriver at de blir\nder til lidt ut i Mai. Nu er hun\nnoksaa medtat skj\u00f8nner jeg af Sorg\nog af L\u00e6ngsel, skriver saameget r\u00f8-\nrende om Oluf. Ragna har det\nherligt, skriver hun. Hun er bare\ngla, og det vet jeg, at \"Oluf er gla over,\nder hvor han nu er. Jeg har saa in-\nderlig ondt af hende, og om Du skrev\net Par Ord tror jeg det vilde gj\u00f8re\nhende godt. - Jeg gl\u00e6der mig over,\nat Du har skrevet en Bok, og\njeg er vis paa, at den er god.",
    "confidences": {
        "total": 0.99,
        "word": [
            1.0,
            1.0,
            1.0,
            0.99,
            ...
        ],
        "line": [
            0.99,
            0.99,
            1.0,
            0.99,
            1.0,
            1.0,
            0.99,
            1.0,
            1.0,
            0.99,
            0.99,
            1.0,
            1.0,
            1.0,
            1.0,
            0.99,
            0.99,
            0.98,
            1.0,
            1.0,
            0.99,
            1.0,
            1.0
        ]
    }
}

By default, only the total (overall) confidence score will be computed.

marked this merge request as ready

added 1 commit

• 0b511947 - remove useless argument

Compare with previous version

added 2 commits

• db79c0e1 - small improvements
• 96327069 - add arguments for word and line separators

Compare with previous version

resolved all threads

As previously discussed, I added two arguments --word-separators and --line-separators to set separators to split text into words or lines.

These parameters are used to compute confidence scores and plot attention maps at word/line level.

added 2 commits

• ead5ff4c - use regex delimiters
• 69a55059 - update doc

Compare with previous version

Now using regular expressions.

We cannot use the custom type with a list of strings:

The argument to type can be any callable that accepts a single string.

resolved all threads

added 1 commit

• bf51947e - Simplify compute_prob_by_separator

Compare with previous version

added 1 commit

• c052ee35 - Fix lint

Compare with previous version

resolved all threads

approved this merge request

merged