Document shm_size

content/workers/yaml.md

@@ -15,7 +15,43 @@ The following attributes are required in every `.arkindex.yml` file:
 
 - `version`: Version of the configuration file in use. An error will occur if the version
 number is not set to `2`.
-- `workers`: A list of workers attached to the Git repository.
+- `type`: Type of the repository. Either `iiif` for a repository holding IIIF manifests
+and collections for importing, or `worker` for a repository holding Arkindex
+workers.
+
+## IIIF Import repository attributes
+
+When the `type` is set to `iiif`, the following attribute is mandatory:
+
+- `manifests`: List of Unix-style patterns matching paths to IIIF 2.x manifests and
+IIIF 2.x and 3.x collections. The `*` and `?` characters and the `[abc]` ranges
+behave like for standard Unix patterns, and `**` matches directories
+recursively.
+
+<!-- TODO:
+For more information on using IIIF in Arkindex, see
+[IIIF in Arkindex](@/iiif/_index.md)
+https://gitlab.com/arkindex/doc/-/issues/15
+-->
+
+### Example configuration
+
+```yaml
+---
+version: 2
+type: iiif
+manifests:
+  - mysuperfolder/mysupermanifest.json
+  - mysuperfolder/asubfolder/**/*.json
+```
+
+This would match `mysuperfolder/mysupermanifest.json` starting at the root of
+the repository, then any JSON file in `mysuperfolder/asubfolder` or any of its
+sub-directories.
+
+## Worker repository attributes
+
+When the `type` is set to `worker`, the `workers` attribute is mandatory.
 
 The `workers` attribute is a list of the following:
 
@@ -45,8 +81,7 @@ include:
 
 - `docker`: Regroups Docker-related configuration attributes:
 
-  - `build`: Path towards a Dockerfile used to build this worker, relative to the root of
-  the repository. Defaults to `Dockerfile`.
+  - `build`: Path towards a Dockerfile used to build this worker, relative to the root of the repository. Defaults to `Dockerfile`.
 
   <!--
   TODO: Make the path relative to the YAML file itself, in the case of a
@@ -62,8 +97,8 @@ include:
   custom image from a Dockerfile.
   -->
 
-  - `command`: Custom command line to be used when launching the Docker container for
-  this Worker. By default, the command specified in the Dockerfile will be used.
+  - `command`: Custom command line to be used when launching the Docker container for this Worker. By default, the command specified in the Dockerfile will be used.  
+  - `shm_size`: Size of the available shared memory in `/dev/shm`. The default value is `64M`, but when training machine learning models an increase might be necessary. The given value must be either an integer, or an integer followed by a unit (`b` for bytes, `k` for kilobytes, `m` for megabytes and `g` for gigabytes). If no unit is specified, the default unit is `bytes`. See the [Docker documentation](https://docs.docker.com/engine/reference/run/#runtime-constraints-on-resources).  
 
 - `environment`: Mapping of string keys and string values to define environment variables to be
 set when the Docker image runs.
@@ -290,6 +325,7 @@ workers:
       build: project/Dockerfile
       image: hub.docker.com/project/image:tag
       command: python mysuperscript.py --blabla
+      shm_size: 128m
       environment:
         TOKEN: deadBeefToken
     configuration: