Skip to content

Optimize ListTranscriptions

Erwan Rouchet requested to merge optimize-list-transcriptions into master

Closes #480 (closed)

  • Moved the element to a cached property, along with its permission checks, to remove two duplicate queries.
  • Removed the prefetch for element__image__server when not using recursive, since the element is never returned.
  • Changed the element filter when using recursive to use a UNION on the element IDs from ElementPath + the ID of the current element.
    • This no longer loads every single element in RAM.
    • This does not use Q(element=element) | Q(element__in=…) because that makes PostgreSQL ignore almost every index and start using nested loops.
    • This does not use get_descending, because it uses some unnecessary ordering and deduplication.
    • This uses ElementPath.element_id and not Element.id to avoid an unnecessary join.
    • This uses UNION ALL and not just UNION, to avoid an unnecessary cycle of sorting and deduplicating IDs.
  • Some benchmarks on the strange id::varchar sort did not show any difference with a normal id sort, so I left it there, but it does not seem to be as "inefficient" as what the comments say.

This makes recursive ListTranscriptions up to 10 times faster on top-level folders of random corpora with many transcriptions.

Merge request reports

Loading