diff --git a/content/deployment/ponos.md b/content/deployment/ponos.md new file mode 100644 index 0000000000000000000000000000000000000000..59b0a4d58308575fdba2b5a4a176b468c6075180 --- /dev/null +++ b/content/deployment/ponos.md @@ -0,0 +1,52 @@ ++++ +title = "Ponos agent" +description = "Deploy Ponos agents on your own infrastucture to run machine learning processes on GPU" + +weight = 50 ++++ + +{% info() %} +This feature is only available for Enterprise Edition customers. +{% end %} + +If your setup requires Machine Learning process, you'll need at least one **Ponos Agent** on a dedicated server. + +The setup of this kind of server is easier, as it only requires to run the agent (from Docker image `registry.gitlab.teklia.com/arkindex/ponos-agent`) and configure it. The tasks will then be triggered by the agent automatically. + +To begin the setup, you'll need 2 private keys: one for the backend, another for the agent. Each agent needs a dedicated key to authenticate itself. + +To generate a valid private key: + +```sh +openssl ecparam -name secp384r1 -genkey -noout > agent.key +``` + +A YAML configuration file is also required: + +```yaml +--- +# Save as agent.yml +url: https://ark.localhost/ +farm_id: XXXXX +seed: YYYYY +data_dir: /data +private_key: /etc/ponos/agent.key +``` + +The `farm_id` and `seed` information can be found in the Arkindex administration interface under the section **Ponos > Farms**. + +You can then run the agent as: + +```sh +docker run \ + --name=ponos \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -v ./agent.yml:/etc/ponos/agent.yml:ro \ + -v ./agent.key:/etc/ponos/agent.key:ro \ + -v ponos_data:/data + registry.gitlab.teklia.com/arkindex/ponos-agent:X.Y.Z +``` + +Please note that the agent requires a write access on the local Docker socket in order to create new containers that will run the tasks. + +The `ponos_data` docker volume is not required, but will allow to retrieve debug logs outside of the agent container. diff --git a/content/deployment/setup.md b/content/deployment/setup.md index 64974ad998b03a5263b4fcdfc4eeb5e3b02ebd13..0ee19ce0ca422fd2d679ce1a68743c89613cf129 100644 --- a/content/deployment/setup.md +++ b/content/deployment/setup.md @@ -1,16 +1,21 @@ +++ -title = "Docker setup" +title = "Setup with Docker" description = "Deploy Arkindex on your own infrastucture using Linux and docker-compose" weight = 30 +++ +In this documentation, we'll provide information on how to setup **Arkindex Community Edition** through Docker, on a single server. + +If you need more information for a cluster or cloud setup, or using Enterprise Edition, please [contact us](https://teklia.com/contact/). + + ## Requirements -- A bare metal server running Linux Ubuntu LTS (20.04 or 22.04) for the platform - - If you plan to run Machine Learning processes, you'll need another server +- A bare metal server running Linux Ubuntu LTS (22.04) for the platform + - If you plan to run Machine Learning processes, you'll need another server with a GPU - [Docker installed on that server](https://docs.docker.com/desktop/install/linux-install/) -- [docker-compose](https://docs.docker.com/desktop/install/linux-install/) +- A domain name for that server ## Third-party services @@ -28,13 +33,19 @@ Optional services: - a IIIF server for your images, [cantaloupe](https://cantaloupe-project.github.io/) - a search engine to lookup your transcriptions: [Apache Solr](https://solr.apache.org/) +You can find a detailed [docker-compose](https://gitlab.teklia.com/arkindex/backend/-/tree/master/docker?ref_type=heads) file in the Arkindex backend repository. + +Of course, you'll need to tweak the file so that it matches your own settings and domain name: +- replace all `ark.localhost` references with your own domain name +- remove the minio references if you are using Amazon S3 buckets (or any other compatible solution) +- remove solr references if you do not need search capabilities + ## Arkindex software Teklia will provide you with several docker images (to load using [docker load](https://docs.docker.com/engine/reference/commandline/load/)): - the backend image, tagged `registry.gitlab.teklia.com/arkindex/backend:X.Y.Z`, must be present on your application server, - the tasks image, `registry.gitlab.teklia.com/arkindex/tasks:X.Y.Z`, will be used to by the remote workers (file imports, thumbnails generation, ...). -- the ponos image, `registry.gitlab.teklia.com/arkindex/ponos-agent:X.Y.Z` will be used to actually run the asynchronous tasks across all your remote workers. - {{ figure(image="deployment/stack.png", height=250, caption="Arkindex Platform and a single Worker") }}