Contributing & Licensing

content/_index.md

@@ -10,3 +10,5 @@ It allows you to scan any document in any file format (PDF Document, image file,
 Arkindex can read text in **multiple languages** such as French, English, Arabic, Chinese, but also in historical languages like Latin or ancient French for historical research purposes.
 
 ⭐ A good starting point in this documentation is the [Overview page](@/overview/_index.md).
+
+💰 Arkindex is freely available as an open-source solution for hobbyists. An enterprise edition also exists for companies with larger projects. More information on the [Licensing](@/overview/license.md).

content/develop/_index.md

++++
+title = "Development"
+sort_by = "weight"
+weight = 100
+insert_anchor_links = "right"
++++
+
+This documentation is aimed at web developers who want to contribute to the Arkindex platform.
+
+In this section you'll find out how to:
+
+1. setup [Arkindex locally](@/develop/setup.md), in development mode,
+2. [contribute patches](@/develop/contributing.md) to improve Arkindex.

content/develop/contributing.md

++++
+title = "Contributing patches to Arkindex"
+weight = 20
++++
+
+All contributions are welcome to improve Arkindex, whether they are code, translations, design suggestions, small fixes, etc.
+
+If you want to discuss some ideas, send us an email at `team (at) teklia.com`!
+
+## Submitting a patch to Arkindex's source code
+
+### Find issues
+
+We use GitLab to track issues (either bugs, or new features).
+
+You can browse Arkindex's existing issues:
+
+- either through the current [milestone](https://gitlab.teklia.com/groups/arkindex/-/milestones), containing the currently most important issues, as they are the most time-sensitive;
+- or through the [full issues list](https://gitlab.teklia.com/groups/arkindex/-/issues/)
+
+{% info() %}
+Please ask first if you can work on an issue, by simply commenting on it. Be aware that some issues may be way more complex than others.
+{% end %}
+
+#### Priorities
+
+Issues should have a priority label:
+
+- **P1** are high priority, usually bugs badly affecting user experience,
+- **P2** are at the normal priority level
+- **P3** are low priority
+
+If nothing is set, assume **P2** by default.
+
+### Work on a branch
+
+To start working on a patch, you must first create a **git branch**, based on the current `master` branch:
+
+```bash
+# Start from master
+git checkout master
+
+# Update master to latest available revision
+git pull origin master
+
+# Create a new branch
+git checkout -b my-new-branch
+```
+
+Each new commit will then be stored on that new branch named `my-new-branch`.
+
+To name your branch, use a name:
+
+1. in English,
+2. in lowercase,
+3. without spaces, use dashes `-` to link words,
+4. explicit and related to your current work.
+
+Here are some examples of suitable branch names:
+
+- `remove-model-x` when the goal is to remove a Django model named `X`
+- `fix-invalid-chars-search` when the patch fixes a **bug** related to search
+- `bump-dependency-y`
+- `add-super-feature`
+- `feature-z`
+
+Please avoid:
+
+- naming your branch with the issue ID, we prefer explicit naming here,
+- using a language other than English,
+- spaces, camelCase, underscores, etc.
+
+### Publish your work
+
+You are allowed to push directly on Arkindex's repositories, except for the `master` branch. The goal is for your code to be merged into `master` once the following steps are completed:
+
+1. unit tests are all OK, meaning that all jobs in the CI stage named `test` ended in success
+2. formatting has been validated by a tool, meaning that all jobs in the CI stage named `checks` ended in success
+3. the code itself has been approved by a human reviewer
+
+In order for your contribution to be approved, you need to [create a Merge Request (also called MR)](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html) on GitLab.
+
+When pushing [your code from your local branch](https://docs.gitlab.com/ee/user/project/merge_requests/creating_merge_requests.html#when-you-use-git-commands-locally), you will notice some output in the console with a link towards `gitlab.teklia.com`: it will allow you to create a Merge Request (or view the previously created one) in 2 clicks.
+
+Once your work is ready, configure your Merge Request as follows:
+
+- Assign yourself as the **Assignee**.
+- Assign [@erouchet](https://gitlab.teklia.com/erouchet) as the **Reviewer**: he will either review or re-assign to another reviewer when needed.
+- Set an explicit name, in English, properly formatted.
+- Add a reference to the issue you are working on in the description:
+    * `Closes #XYZ` if you fully solve the mentioned issue,
+    * `Ref #XYZ` if you only want to link your Merge Request to the issue.
+- Fill in the description with an explanation of the changes you made.
+- If the related issue has a milestone set, set the same milestone on this Merge Request.
+
+If you are not confident the work being published is yet ready for review, you can prefix your Merge Request name with `Draft:`; that will tell the reviewer to wait a bit before diving into your code. Do not forget to remove that prefix once your code is ready.
+
+The reviewer may leave some comments directly on the Merge Request, asking you for updates. Please resolve all of them (or discuss them if you disagree), publish some commits fixing the issues, and then [ask for a new review](https://docs.gitlab.com/ee/user/project/merge_requests/reviews/#request-a-new-review). Rinse and repeat until the reviewer approves and merges your code!
+
+### Update your branch
+
+As other developers are working on Arkindex, sometimes features/bugfixes/etc that impact the same areas of code as your own merge Request will land on master before your Merge Request does. You may get conflicts here, and need to solve them using a **rebase**.
+
+{% info() %}
+It is your responsibility, as a developer, to maintain your code in a **mergeable state**: no conflicts and up to date with the latest `master`.
+{% end %}
+
+To update using **rebase**, while working on your branch:
+
+```bash
+# Retrieve the latest updates from master
+git fetch origin master
+
+# Now the remote reference to origin/master has been updated, you can rebase on top of it
+# Be aware that the local reference to master is not yet updated, it is only updated with git pull
+git rebase origin/master
+```
+
+The `git rebase` operation will ask you to manually solve conflicts (if any). Please [follow this guide](https://www.atlassian.com/git/tutorials/rewriting-history/git-rebase) or ask us for help if you are lost.
+
+### Landing your code
+
+If you have reached this step, congratulations and many thanks! Your code has been approved and should already be merged, as that is the responsibility of the reviewer.
+
+Your work will be shipped in [the next Arkindex release](https://teklia.com/solutions/arkindex/releases/) along with latest features, bugfixes and other contributions.

content/develop/setup.md

++++
+title = "Arkindex setup for developers"
+weight = 10
++++
+
+Following this documentation allows you to set up a locally running Arkindex instance ([Community Edition](@/overview/license.md)) for development, where you can make any changes to the code and immediately see how they affect the backend Rest API and frontend display.
+
+
+{% warning() %}
+This setup is NOT suitable for production use. To find out how to deploy Arkindex to production (using Docker images), you should refer to the [deployment](../how_to/on_premise/index.md) documentation.
+{% end %}
+
+## Requirements
+
+We recommend you use a Linux distribution such as Debian or Ubuntu to follow these instructions. We at Teklia exclusively use Ubuntu LTS 22.04.
+
+Minimal software requirements are:
+- [Docker 24+](https://docs.docker.com/engine/install/#supported-platforms)
+- Python 3.10
+- pip and [virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/latest/)
+- Node 16
+- Git
+- Make
+- [mkcert](https://github.com/FiloSottile/mkcert?tab=readme-ov-file#installation)
+- [GeoDjango system dependencies](https://docs.djangoproject.com/en/3.1/ref/contrib/gis/install/geolibs/): `sudo apt install binutils libproj-dev gdal-bin`
+
+
+## Cloning the source code
+
+In order to set up your local Arkindex instance, you first need to clone two projects: Arkindex **backend** and **frontend**.
+
+You should clone them in the same parent folder, as some commands will assume specific relative paths.
+
+
+### Anonymous clone
+
+You can clone the repositories anonymously using HTTPS clone:
+
+```
+git clone https://gitlab.teklia.com/arkindex/backend.git
+git clone https://gitlab.teklia.com/arkindex/frontend.git
+```
+
+### Authenticated clone
+
+If you want to contribute to Arkindex and submit patches, we recommend you use SSH cloning. To do so, you need to create an account on our own Gitlab instance at [gitlab.teklia.com](https://gitlab.teklia.com/users/sign_up).
+
+In order to use SSH cloning, you need:
+
+- a working [git](https://git-scm.com/) client installation on your computer
+- an SSH key to authenticate yourself on a GitLab instance
+
+You can find more information [here](https://docs.gitlab.com/ee/user/ssh.html) on how-to setup an SSH key for git access.
+
+#### Add your SSH key
+
+Once you are connected to [gitlab.teklia.com](https://gitlab.teklia.com/), you need to add your **SSH public key** to your user account on Teklia's GitLab instance. To do so, you can follow [the official GitLab instructions](https://docs.gitlab.com/ee/user/ssh.html#add-an-ssh-key-to-your-gitlab-account).
+
+#### Activate 2FA
+
+To gain and keep access to Arkindex's GitLab repositories as a contributor, your GitLab account must be protected with Multiple Factor Authentication (generally using one time passwords generated from your phone).
+
+You will need to setup an app like [Google Authenticator](https://support.google.com/accounts/answer/1066447?hl=fr&co=GENIE.Platform%3DAndroid) or [Duo Mobile](https://duo.com/product/multi-factor-authentication-mfa/duo-mobile-app) on your phone, and [then follow these steps](https://docs.gitlab.com/ee/user/profile/account/two_factor_authentication.html#enabling-2fa).
+
+If you do not complete the 2FA initial activation within 72 hours of creating your account, it will be automatically disabled.
+
+#### Cloning with SSH
+
+Finally you can clone the projects:
+
+```
+git clone git@gitlab.teklia.com:arkindex/backend.git
+git clone git@gitlab.teklia.com:arkindex/frontend.git
+```
+
+## Run the third-party services
+
+Arkindex relies on several third-party open-source services to store data and communicate across components:
+- Postgresql
+- Redis
+- MinIO
+- Cantaloupe IIIF server
+
+All these services run through docker, and are configured in a `docker-compose.yml` file stored in the **backend** repository.
+
+A convenient Makefile shortcut is available to start all these services in a dedicated shell:
+
+```
+cd backend
+make services
+```
+
+You should see a few services start and log their output in the shell.
+
+On a first start, **mkcert** will generate a Certificate Authority for your instance so that it's only one exposed using HTTPS.
+
+## Run the backend
+
+In order to setup the Arkindex backend, you must first (in a different terminal than the one where the services are running) create a Python virtual environment and populate it:
+
+```
+cd backend
+mkvirtualenv -a $(pwd) -p /usr/bin/python3.10 arkindex
+pip install -e .
+```
+
+Next you need to configure your Arkindex instance, by copying the sample config file to the expected path:
+
+```
+cp config.yml.sample arkindex/config.yml
+```
+Later on, you'll be able to edit `arkindex/config.yml` to further configure your Arkindex instance.
+
+After a few moment of Python dependencies installation, you should be able to run commands using the `arkindex` script.
+
+First, you need to build the initial empty database structure:
+
+```
+arkindex migrate
+```
+
+Then populate it with some common settings for developers:
+ 
+```
+arkindex bootstrap
+```
+
+And create your own administrator account:
+
+```
+arkindex createsuperuser
+```
+
+Finally, you can run the development webserver:
+
+```
+arkindex runserver
+```
+
+You should be able to login on the admin portal through `http://localhost:8000/admin/`
+
+
+## Run the frontend
+
+The frontend installation only requires **npm**:
+
+```
+cd frontend
+npm install
+```
+
+From there, you can run the development server:
+
+```
+npm run start
+```
+
+And access the whole interface at `http://localhost:8080`
+
+{% info() %}
+The backend and services must be running while using the frontend interface, otherwise all you'll get is a "server unreachable" error page.
+{% end %}

content/overview/_index.md

@@ -11,6 +11,8 @@ All of its features are fully accessible from its web interface, but also from a
 
 {{ figure(image="overview/frontend.png", height=300, caption="Browse all your documents using a clean web interface") }}
 
+Two versions of Arkindex are available: [open-source and enterprise](@/overview/license.md).
+
 ## Document management
 
 Arkindex is first and foremost a web platform to process thousands of documents. Document management is an essential part of its features. We use a generic term `Element` to talk about a Document and its components. We also use a graph structure in database to be able to represent any of your documents.
@@ -42,6 +44,10 @@ Arkindex can import your data from different sources:
 
 ## Customizable user access
 
+{% info() %}
+Limited to Enterprise Edition
+{% end %}
+
 Each Arkindex platform instance can be shared amongst multiple people, projects, and teams. As a project administrator you will have fine-grained control over which users can access your document collections (so called `Projects` in Arkindex).
 
 You can read more about [controlling user access](@/users/_index.md).

content/overview/license.md

++++
+title = "Licensing"
+weight = 100
++++
+
+The Arkindex platform is available under two editions, each with its own license.
+
+
+## Community Edition (CE)
+
+Arkindex is freely available under the open-source license [AGPL-v3](https://www.gnu.org/licenses/agpl-3.0.en.html). All projects using Arkindex must also be open-source.
+
+You can run your Arkindex instance, share it with other people, train models on your own hardware, and even modify its source code! Documentation is available [for developers](@/develop/_index.md) who wish to contribute.
+
+Some features are missing from this edition in comparison to the **Enterprise Edition**, but they should not prevent hobbyists or even researchers from using and benefiting from Arkindex:
+
+- No access control for projects (all projects are available to all logged-in users)
+- No super-computer support through [slurm](https://slurm.schedmd.com/overview.html)
+- Machine Learning tasks need direct access to the database and cannot be network-segregated
+- Export generation through snapshots is not available
+
+
+## Enterprise Edition (EE)
+
+Arkindex is also available under a license that is compatible with commercial usage, and can bypass the open-source requirements of AGPL-v3.
+
+The Enterprise Edition is fully featured, with fine-grained access control over projects, slurm support, execution of Machine Learning tasks across unlimited servers and GPUs, etc.
+
+If you are an enterprise or institution interested in hosting your own Arkindex platform, please [contact us](https://teklia.com/contact). We also offer contracting work to develop new features, train your employees to use Arkindex, or deploy the platform on your own stack.