Add an option to compile models during infrerence
Ref: https://redmine.teklia.com/issues/6410
We saw that using torch.compile
can speed up inference.
We need to add:
- an option to compile models before prediction (disabled by default)
- another option to use the
dynamic
mode