Sort charset when loading model during inference

We do it when training, but we don't explicitly sort during model loading at inference.

We should do it, to avoid any issues.