Skip to content
Snippets Groups Projects
Verified Commit b3f159f7 authored by Erwan Rouchet's avatar Erwan Rouchet
Browse files

GitLab migration

parent c25fc41d
No related branches found
No related tags found
1 merge request!96GitLab migration
Pipeline #62844 passed
...@@ -583,7 +583,7 @@ An example of a to-do comment with a GitLab issue follows: ...@@ -583,7 +583,7 @@ An example of a to-do comment with a GitLab issue follows:
```markdown ```markdown
<!-- <!--
TODO: Update screenshots for v1.0 TODO: Update screenshots for v1.0
https://gitlab.com/teklia/arkindex/doc/issues/99999 https://gitlab.teklia.com/arkindex/doc/issues/99999
--> -->
``` ```
......
...@@ -8,7 +8,7 @@ path = "howto/on-premise/configuration/" ...@@ -8,7 +8,7 @@ path = "howto/on-premise/configuration/"
You will find on this page all the configuration settings available for the Arkindex backend. These settings must be stored in a YAML file, and exposed using a Docker volume to the backend and worker container. The configuration path is set through `CONFIG_PATH` environment variable. You will find on this page all the configuration settings available for the Arkindex backend. These settings must be stored in a YAML file, and exposed using a Docker volume to the backend and worker container. The configuration path is set through `CONFIG_PATH` environment variable.
A minimal file is available on [our public repository](https://gitlab.com/teklia/arkindex/public-architecture/-/blob/master/config.yml). A minimal file is available on [our public repository](https://gitlab.teklia.com/arkindex/public-architecture/-/blob/master/config.yml).
## `allowed_hosts` ## `allowed_hosts`
...@@ -468,7 +468,7 @@ The AWS region the S3 buckets are located in. This has no effect when `endpoint` ...@@ -468,7 +468,7 @@ The AWS region the S3 buckets are located in. This has no effect when `endpoint`
### `tasks_image` ### `tasks_image`
A Docker tag for the Arkindex tasks image. This defaults to `registry.gitlab.com/arkindex/tasks`. When the image is specified without a tag, `:latest` is assumed. A Docker tag for the Arkindex tasks image. This defaults to `registry.gitlab.teklia.com/arkindex/tasks`. When the image is specified without a tag, `:latest` is assumed.
This is used to specify which image to use for Ponos tasks when running imports from DataFiles, existing elements, S3 buckets, or when generating thumbnails. This is used to specify which image to use for Ponos tasks when running imports from DataFiles, existing elements, S3 buckets, or when generating thumbnails.
......
...@@ -46,9 +46,9 @@ Optional services: ...@@ -46,9 +46,9 @@ Optional services:
## Arkindex software ## Arkindex software
Teklia will provide you with several docker images (to load using [docker load](https://docs.docker.com/engine/reference/commandline/load/)): Teklia will provide you with several docker images (to load using [docker load](https://docs.docker.com/engine/reference/commandline/load/)):
- the backend image, tagged `registry.gitlab.com/teklia/arkindex/backend:X.Y.Z`, must be present on your application server, - the backend image, tagged `registry.gitlab.teklia.com/arkindex/backend:X.Y.Z`, must be present on your application server,
- the tasks image, `registry.gitlab.com/teklia/arkindex/tasks:X.Y.Z`, will be used to by the remote workers (file imports, thumbnails generation, ...). - the tasks image, `registry.gitlab.teklia.com/arkindex/tasks:X.Y.Z`, will be used to by the remote workers (file imports, thumbnails generation, ...).
- the ponos image, `registry.gitlab.com/teklia/arkindex/ponos:X.Y.Z` will be used to actually run the asynchronous tasks across all your remote workers. - the ponos image, `registry.gitlab.teklia.com/arkindex/ponos-agent:X.Y.Z` will be used to actually run the asynchronous tasks across all your remote workers.
- -
{{ figure(image="howto/on_premise/stack.png", height=250, caption="Arkindex Platform and a single Worker") }} {{ figure(image="howto/on_premise/stack.png", height=250, caption="Arkindex Platform and a single Worker") }}
...@@ -56,12 +56,12 @@ The backend image mentioned above will run in two containers on your application ...@@ -56,12 +56,12 @@ The backend image mentioned above will run in two containers on your application
1. for the API, this is really the heart of Arkindex, 1. for the API, this is really the heart of Arkindex,
2. for the local asynchronous tasks that can directly reach the database (sqlite export, element deletion, ...) 2. for the local asynchronous tasks that can directly reach the database (sqlite export, element deletion, ...)
We recommend you to use our own CDN for the **frontend files**. Simply use the `assets.teklia.com` as source for static files in the backend configuration (you can look into [our own example](https://gitlab.com/teklia/arkindex/public-architecture/-/blob/master/config.yml)). We recommend you to use our own CDN for the **frontend files**. Simply use the `assets.teklia.com` as source for static files in the backend configuration (you can look into [our own example](https://gitlab.teklia.com/arkindex/public-architecture/-/blob/master/config.yml)).
### Docker-Compose ### Docker-Compose
We documented a working example of docker-compose setup in a [dedicated public repository](https://gitlab.com/teklia/arkindex/public-architecture/). You can clone this repository and use it as a starting point for your own deployment. We documented a working example of docker-compose setup in a [dedicated public repository](https://gitlab.teklia.com/arkindex/public-architecture/). You can clone this repository and use it as a starting point for your own deployment.
Of course your setup may differ, you could use external services (databases, search engine, file storage, ...). You just need to run our own software through Docker, the other parts can be externalized. Of course your setup may differ, you could use external services (databases, search engine, file storage, ...). You just need to run our own software through Docker, the other parts can be externalized.
...@@ -69,13 +69,13 @@ Of course your setup may differ, you could use external services (databases, sea ...@@ -69,13 +69,13 @@ Of course your setup may differ, you could use external services (databases, sea
All the configuration options for the backend are detailed [on this page](@/howto/on_premise/configuration.md). All the configuration options for the backend are detailed [on this page](@/howto/on_premise/configuration.md).
A minimal configuration file is also available in the [public repository](https://gitlab.com/teklia/arkindex/public-architecture/-/blob/master/config.yml). A minimal configuration file is also available in the [public repository](https://gitlab.teklia.com/arkindex/public-architecture/-/blob/master/config.yml).
### Ponos ### Ponos
If your setup requires Machine Learning process, you'll need at least one **Ponos Agent** on a dedicated server. If your setup requires Machine Learning process, you'll need at least one **Ponos Agent** on a dedicated server.
The setup of this kind of server is easier, as it only requires to run the agent (from Docker image `registry.gitlab.com/teklia/arkindex/ponos`) and configure it. The tasks will then be triggered by the agent automatically. The setup of this kind of server is easier, as it only requires to run the agent (from Docker image `registry.gitlab.teklia.com/arkindex/ponos-agent`) and configure it. The tasks will then be triggered by the agent automatically.
To begin the setup, you'll need 2 private keys: one for the backend, another for the agent. Each agent needs a dedicated key to authenticate itself. To begin the setup, you'll need 2 private keys: one for the backend, another for the agent. Each agent needs a dedicated key to authenticate itself.
...@@ -108,7 +108,7 @@ docker run \ ...@@ -108,7 +108,7 @@ docker run \
-v ./agent.yml:/etc/ponos/agent.yml:ro \ -v ./agent.yml:/etc/ponos/agent.yml:ro \
-v ./agent.key:/etc/ponos/agent.key:ro \ -v ./agent.key:/etc/ponos/agent.key:ro \
-v ponos_data:/data -v ponos_data:/data
registry.gitlab.com/teklia/arkindex/ponos:X.Y.Z registry.gitlab.teklia.com/arkindex/ponos-agent:X.Y.Z
``` ```
Please note that the agent requires a write access on the local Docker socket in order to create new containers that will run the tasks. Please note that the agent requires a write access on the local Docker socket in order to create new containers that will run the tasks.
......
...@@ -149,7 +149,7 @@ When clicking on the **Retry** button or creating a new process with similar wor ...@@ -149,7 +149,7 @@ When clicking on the **Retry** button or creating a new process with similar wor
If some elements are locked in a `started` state and could never complete, you will have to wait a 1 hour timeout to start processing them again with this worker. If some elements are locked in a `started` state and could never complete, you will have to wait a 1 hour timeout to start processing them again with this worker.
{% warning() %} {% warning() %}
[Base worker package](https://gitlab.com/teklia/workers/base-worker/) must be at version `0.2.2` or above for a worker to skip processed elements. [Base worker package](https://gitlab.teklia.com/workers/base-worker/) must be at version `0.2.2` or above for a worker to skip processed elements.
{% end %} {% end %}
It is also possible to list all existing activities at a project's level. This page is accessible from the main page of a project by clicking on **Action** > **Workers activity**. It is also possible to list all existing activities at a project's level. This page is accessible from the main page of a project by clicking on **Action** > **Workers activity**.
......
...@@ -25,7 +25,7 @@ These complex tools are easily launched from your web browser, and run on powerf ...@@ -25,7 +25,7 @@ These complex tools are easily launched from your web browser, and run on powerf
{{ figure(image="overview/process.png", height=240, caption="Easily view the progress of Machine Learning workflows") }} {{ figure(image="overview/process.png", height=240, caption="Easily view the progress of Machine Learning workflows") }}
<!-- TODO: Add link on how to create a process: https://gitlab.com/teklia/arkindex/doc/-/issues/8 --> <!-- TODO: Add link on how to create a process: https://gitlab.teklia.com/arkindex/doc/-/issues/8 -->
## Data Import ## Data Import
...@@ -36,9 +36,9 @@ Arkindex can import your data from different sources: ...@@ -36,9 +36,9 @@ Arkindex can import your data from different sources:
- an [Amazon S3](https://aws.amazon.com/s3/) bucket - an [Amazon S3](https://aws.amazon.com/s3/) bucket
- [Transkribus](https://readcoop.eu/transkribus) collections - [Transkribus](https://readcoop.eu/transkribus) collections
<!-- TODO: Add link on how to import from transkribus: https://gitlab.com/teklia/arkindex/doc/-/issues/23 --> <!-- TODO: Add link on how to import from transkribus: https://gitlab.teklia.com/arkindex/doc/-/issues/23 -->
<!-- TODO: Add link on how to create a process: https://gitlab.com/teklia/arkindex/doc/-/issues/7 --> <!-- TODO: Add link on how to create a process: https://gitlab.teklia.com/arkindex/doc/-/issues/7 -->
## Customizable user access ## Customizable user access
......
...@@ -16,7 +16,7 @@ We have open-sourced two API clients in Python to ease the usage of our platform ...@@ -16,7 +16,7 @@ We have open-sourced two API clients in Python to ease the usage of our platform
### Python API client ### Python API client
A low-level API client in Python is available on PyPI as [arkindex-client](https://pypi.org/project/arkindex-client/). Its source code is freely available on [this repository](https://gitlab.com/teklia/arkindex/api-client/). A low-level API client in Python is available on PyPI as [arkindex-client](https://pypi.org/project/arkindex-client/). Its source code is freely available on [this repository](https://gitlab.teklia.com/arkindex/api-client/).
You can install it on your computer (provided you have Python 3.6+ installed) using pip: You can install it on your computer (provided you have Python 3.6+ installed) using pip:
...@@ -41,7 +41,7 @@ More documentation is available on the [official package page](https://pypi.org/ ...@@ -41,7 +41,7 @@ More documentation is available on the [official package page](https://pypi.org/
### Command line tool ### Command line tool
A more high-level tool is available using the command line. It implements common operations to be more efficient with Arkindex. A more high-level tool is available using the command line. It implements common operations to be more efficient with Arkindex.
It is available on PyPI as [arkindex-cli](https://pypi.org/project/arkindex-cli/). Its source code is freely available on [this repository](https://gitlab.com/teklia/arkindex/cli/). It is available on PyPI as [arkindex-cli](https://pypi.org/project/arkindex-cli/). Its source code is freely available on [this repository](https://gitlab.teklia.com/arkindex/cli/).
A dedicated detailed documentation is available at [cli.arkindex.org](https://cli.arkindex.org/). A dedicated detailed documentation is available at [cli.arkindex.org](https://cli.arkindex.org/).
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment