New documentation structure

.pre-commit-config.yaml

@@ -38,7 +38,7 @@ repos:
     rev: v0.39.0
     hooks:
       - id: markdownlint
-        args: ['--disable', 'MD013', 'MD046', '--']
+        args: ['--fix', '--disable', 'MD013', 'MD046', '--']
   - repo: meta
     hooks:
       - id: check-useless-excludes

docs/contents/implem/configure.md → docs/contents/implem/configure/index.md

@@ -71,9 +71,9 @@ This step shows that there are a lot of sources for the actual configuration tha
 
     The convention is to always give the final word to the user. This means that when the user configuration is filled, its values must be the last to override the worker's `config` attribute. If a model configuration was set, its values must override this attribute before the user configuration's.
 
-The worker configuration may specify default values for some parameters (see [this section](../workers/yaml.md#setting-up-user-configurable-parameters) for more details about worker configuration). These default values are stored in the `user_configuration` dictionary attribute.
+The worker configuration may specify default values for some parameters (see [this section](yaml.md#setting-up-user-configurable-parameters) for more details about worker configuration). These default values are stored in the `user_configuration` dictionary attribute.
 
-This is also when the secrets (see [this section](../secrets/usage.md#declaring-secrets-in-workers) to learn more about secrets) are actually downloaded. They are stored in the `secrets` dictionary attribute.
+This is also when the secrets (see [this section](secrets.md#declaring-secrets-in-workers) to learn more about secrets) are actually downloaded. They are stored in the `secrets` dictionary attribute.
 
 An Arkindex-mode exclusive step is done after all that: the cache setup. Some workers benefit a lot, performance-wise, from having a SQLite cache artifact from previous workers. This is mostly used in processes with multiple workers with dependencies, where the second worker needs the results of the first one to work. The database is initialized, the tables created and its version checked as it must match the one supported by the Arkindex instances. The database is then merged with any other database generated by previous worker runs.
 
@@ -81,9 +81,9 @@ Once all information is retrieved and stored in the worker, the configuration is
 
 ## Developer mode
 
-In the developer mode, the worker execution is not linked to anything on Arkindex. Therefore, the only configuration the worker can use is provided via the `--config` CLI argument. It supports YAML-formatted file and it should be similar to the `configuration` section of the [worker configuration file](../workers/yaml.md#single-worker-configuration), without the `user_configuration` details. More details about how to create the local worker configuration are available in [this section](../workers/run-local.md).
+In the developer mode, the worker execution is not linked to anything on Arkindex. Therefore, the only configuration the worker can use is provided via the `--config` CLI argument. It supports YAML-formatted file and it should be similar to the `configuration` section of the [worker configuration file](yaml.md#single-worker-configuration), without the `user_configuration` details. More details about how to create the local worker configuration are available in [this section](../run/local.md).
 
-The multiple configuration sources from the Arkindex-mode are merged into a unique one here. The configuration parameters are parsed as well as the list of required secrets. The secrets are loaded using a local Arkindex client. Again, see the [section about local execution](../workers/run-local.md) for more details.
+The multiple configuration sources from the Arkindex-mode are merged into a unique one here. The configuration parameters are parsed as well as the list of required secrets. The secrets are loaded using a local Arkindex client. Again, see the [section about local execution](../run/local.md) for more details.
 
 One information cannot be retrieved directly from the configuration file and is required in some cases: the ID of the Arkindex corpus which the elements processed belong to. This is retrieved via the `ARKINDEX_CORPUS_ID` environment variable.
 

docs/contents/secrets/usage.md → docs/contents/implem/configure/secrets.md

-# Usage
+# Use secrets in your worker
+
+## Overview
+
+Secrets are text payloads shared securely between the Arkindex instance and any worker. It is generally used to store sensitive values that may give Arkindex users access to any resources that should be private, for example because they cost money or are proprietary.
+
+For more information about secrets, please visit [the Arkindex documentation](https://doc.arkindex.org/secrets/).
 
 ## Accessing secrets in the API
 
@@ -35,7 +41,7 @@ missing from the Arkindex instance.
    ```
 
 For more information on the `.arkindex.yml` file,
-see [YAML configuration](../workers/yaml.md).
+see [YAML configuration](yaml.md).
 
 ## Accessing secrets in Python code
 

docs/contents/workers/yaml.md → docs/contents/implem/configure/yaml.md

@@ -86,18 +86,6 @@ include:
 
 `docker`
 : Regroups Docker-related configuration attributes:
-    <!--
-    TODO: Make the path relative to the YAML file itself, in the case of a
-    separate file for a single worker?
-    https://gitlab.teklia.com/arkindex/tasks/-/issues/95
-    -->
-    <!--
-    TODO: Implement this!
-    https://gitlab.teklia.com/arkindex/tasks/-/issues/93
-    `image`: Tag of an existing Docker image to use for this worker instead of building a
-    custom image from a Dockerfile.
-    -->
-
     - `build`
     : Path towards a Dockerfile used to build this worker, relative to the root of
     the repository. Defaults to `Dockerfile`.

docs/contents/implem/index.md

-# Worker Implementation
+# Implement your worker
 
 This section presents
 
 - the different stages happening during a worker execution:
     - the initialization
-        - the [configuration](./configure.md)
+        - the [configuration](configure/index.md)
         - the execution
     - the conception of a worker
         - the architecture

docs/contents/implem/process/artifacts.md

+# Artifacts
+
+!!! warning
+
+    This section is under construction.

docs/contents/implem/process/helpers.md

+# Arkindex API Helpers
+
+!!! warning
+
+    This section is under construction.

docs/contents/implem/process/index.md

+# Processing
+
+!!! warning
+
+    This section is under construction.

docs/contents/implem/run/docker.md

+# Run your worker in Docker
+
+!!! warning
+
+    This section is under construction.

docs/contents/implem/run/index.md

+# Run your worker
+
+!!! warning
+
+    This section is under construction.

docs/contents/workers/run-local.md → docs/contents/implem/run/local.md

-# Running your worker locally
+# Run your worker locally
 
 Once you have implemented a worker, you can run it on some Arkindex elements on your own machine to test it.