Skip to content

Add an option to compile models during infrerence

Ref: https://redmine.teklia.com/issues/6410

We saw that using torch.compile can speed up inference. We need to add:

  • an option to compile models before prediction (disabled by default)
  • another option to use the dynamic mode