diff --git a/content/workers/yaml.md b/content/workers/yaml.md index 6267b4918d31f3d10633fab7cc78a10c345301ee..9b81626887e56b54e32979cb964cc4555f34f2d8 100644 --- a/content/workers/yaml.md +++ b/content/workers/yaml.md @@ -15,7 +15,43 @@ The following attributes are required in every `.arkindex.yml` file: - `version`: Version of the configuration file in use. An error will occur if the version number is not set to `2`. -- `workers`: A list of workers attached to the Git repository. +- `type`: Type of the repository. Either `iiif` for a repository holding IIIF manifests +and collections for importing, or `worker` for a repository holding Arkindex +workers. + +## IIIF Import repository attributes + +When the `type` is set to `iiif`, the following attribute is mandatory: + +- `manifests`: List of Unix-style patterns matching paths to IIIF 2.x manifests and +IIIF 2.x and 3.x collections. The `*` and `?` characters and the `[abc]` ranges +behave like for standard Unix patterns, and `**` matches directories +recursively. + +<!-- TODO: +For more information on using IIIF in Arkindex, see +[IIIF in Arkindex](@/iiif/_index.md) +https://gitlab.com/arkindex/doc/-/issues/15 +--> + +### Example configuration + +```yaml +--- +version: 2 +type: iiif +manifests: + - mysuperfolder/mysupermanifest.json + - mysuperfolder/asubfolder/**/*.json +``` + +This would match `mysuperfolder/mysupermanifest.json` starting at the root of +the repository, then any JSON file in `mysuperfolder/asubfolder` or any of its +sub-directories. + +## Worker repository attributes + +When the `type` is set to `worker`, the `workers` attribute is mandatory. The `workers` attribute is a list of the following: @@ -45,8 +81,7 @@ include: - `docker`: Regroups Docker-related configuration attributes: - - `build`: Path towards a Dockerfile used to build this worker, relative to the root of - the repository. Defaults to `Dockerfile`. + - `build`: Path towards a Dockerfile used to build this worker, relative to the root of the repository. Defaults to `Dockerfile`. <!-- TODO: Make the path relative to the YAML file itself, in the case of a @@ -62,8 +97,8 @@ include: custom image from a Dockerfile. --> - - `command`: Custom command line to be used when launching the Docker container for - this Worker. By default, the command specified in the Dockerfile will be used. + - `command`: Custom command line to be used when launching the Docker container for this Worker. By default, the command specified in the Dockerfile will be used. + - `shm_size`: Size of the available shared memory in `/dev/shm`. The default value is `64M`, but when training machine learning models an increase might be necessary. The given value must be either an integer, or an integer followed by a unit (`b` for bytes, `k` for kilobytes, `m` for megabytes and `g` for gigabytes). If no unit is specified, the default unit is `bytes`. See the [Docker documentation](https://docs.docker.com/engine/reference/run/#runtime-constraints-on-resources). - `environment`: Mapping of string keys and string values to define environment variables to be set when the Docker image runs. @@ -290,6 +325,7 @@ workers: build: project/Dockerfile image: hub.docker.com/project/image:tag command: python mysuperscript.py --blabla + shm_size: 128m environment: TOKEN: deadBeefToken configuration: