New endpoint to massively create elements and images

Bastien Abadie
2022-11-25T10:53:40Z

The goal is to have an endpoint capable of creating a large list of elements and images in a single call, replacing lots and lots of calls to CreateImage and CreateElement.

The endpoint would be named CreateIIIFElements and would only support:

internal users

iiif urls

providing width & height (no checks on the backend)

images are considered validated (that's why only internal users are allowed, we can't safely open that to workers for now)

no worker version id

support polygon

only one parent

The payload would have the following fields:

parent_id

list of elements + images descriptions:

iiif_url: full url towards the image

width and height of the image

polygon (nullable, when null, use full image, according to width & height)

name of the element

type of the element (should support ID and slug here, using a pre-flight check/load)

metadatas as a dict of key => value

The endpoint would create in a transaction:

the new images, using a bulk create

the new elements (a pre-fligh check is needed here to list only the new elements)

if an element with this parent + polygon + image exists, skip the rest

the metadatas on new elements (keep it simple here, only adding metadatas to elements that were created)

the element paths on new elements (should not be hard, as you can load initially all the required paths on the parent)

This is needed to speed up a lot socface and other massive ingestion processes (like callico 🤞 )