From fb4c9a94fba9680eb580c47bb5d2352897aa43cc Mon Sep 17 00:00:00 2001
From: Manon Blanco <blanco@teklia.com>
Date: Thu, 28 Mar 2024 15:57:10 +0000
Subject: [PATCH] Document the scope to use for the GitLab token and remove
duplicated documentation
---
README.md | 58 ---------------------------------
docs/get_started/development.md | 13 ++++++++
docs/get_started/index.md | 16 +++++++--
3 files changed, 27 insertions(+), 60 deletions(-)
diff --git a/README.md b/README.md
index fe3a6e14..655fc156 100644
--- a/README.md
+++ b/README.md
@@ -6,64 +6,6 @@ For more details about this package, make sure to see the documentation availabl
This is an open-source project, licensed using [the MIT license](https://opensource.org/license/mit/).
-## Development
-
-For development and tests purpose it may be useful to install the project as a editable package with pip.
-
-This package is based on a GitLab package registry containing all the nerval source code.
-You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) and access to the [nerval repository](https://gitlab.teklia.com/ner/nerval) in order to install this module. You will need to add the below to your `~/.netrc` file:
-
-```shell
-machine gitlab.teklia.com
-login __token__
-password <YOUR_PERSONAL_TOKEN>
-```
-
-Then you can install the package as a editable package with pip:
-
-```shell
-pip3 install --index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple -e .
-```
-
-### Linter
-
-Code syntax is analyzed before submitting the code.\
-To run the linter tools suite you may use pre-commit.
-
-```shell
-pip install pre-commit
-pre-commit run -a
-```
-
-### Run tests
-
-Tests are executed with `tox` using [pytest](https://pytest.org).
-To install `tox`,
-
-```shell
-pip install tox
-tox
-```
-
-To reload the test virtual environment you can use `tox -r`
-
-Run a single test module: `tox -- <test_path>`
-Run a single test: `tox -- <test_path>::<test_function>`
-
-The tests use a large file stored via [Git-LFS](https://docs.gitlab.com/ee/topics/git/lfs/). Make sure to run `git-lfs pull` before running them.
-
-### Update documentation
-
-Please keep the documentation updated when modifying or adding features.
-It's pretty easy to do:
-
-```shell
-pip install -r doc-requirements.txt
-mkdocs serve
-```
-
-You can then write in Markdown in the relevant `docs/*.md` files, and see live output on <http://localhost:8000>.
-
## Inference
To apply DAN to an image, one needs to first add a few imports and to load an image. Note that the image should be in RGB.
diff --git a/docs/get_started/development.md b/docs/get_started/development.md
index 0c2ff165..d94ca211 100644
--- a/docs/get_started/development.md
+++ b/docs/get_started/development.md
@@ -26,16 +26,25 @@ tox
To recreate tox virtual environment (e.g. a dependencies update), you may run `tox -r`.
+Run a single test module: `tox -- <test_path>`
+Run a single test: `tox -- <test_path>::<test_function>`
+
+The tests use a large file stored via [Git-LFS](https://docs.gitlab.com/ee/topics/git/lfs/). Make sure to run `git-lfs pull` before running them.
+
### Commands
As unit tests do not test *everything*, it is sometimes necessary to use DAN commands directly to test developments.
+#### Training command
+
The library already has all the documents needed to run the [training command](../usage/train/index.md) on a minimalist dataset. You can use the configuration available at `configs/tests.json`. It is already populated with the parameters used in the unit tests.
```shell
teklia-dan train --config configs/tests.json
```
+#### Predict command
+
The library already has all the documents needed to run the [predict command](../usage/predict/index.md) with a minimalist model. In the `tests/data/prediction` directory, you can run the following command and add any extra parameters you need:
```shell
@@ -46,12 +55,16 @@ teklia-dan predict \
--output /tmp/dan-predict
```
+#### Evaluation command
+
The library already has all the documents needed to run the [evaluation command](../usage/evaluate/index.md) on a minimalist dataset. You can use the configuration available at `configs/eval.json`. It is already populated with the parameters used in the unit tests.
```shell
teklia-dan evaluate --config configs/eval.json
```
+#### Convert command
+
If you want to evaluate a NER models with you own scripts, you can convert DAN's predictions in [BIO](<https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)>) format, using the [convert command](../usage/convert/index.md).
```shell
diff --git a/docs/get_started/index.md b/docs/get_started/index.md
index a50ac65d..066ae84b 100644
--- a/docs/get_started/index.md
+++ b/docs/get_started/index.md
@@ -3,7 +3,7 @@
## Installation
DAN is based on a GitLab package registry containing all the nerval source code.
-You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) and access to the [nerval repository](https://gitlab.teklia.com/ner/nerval) in order to install this module. You will need to add the below to your `~/.netrc` file:
+You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with [scope](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#personal-access-token-scopes) `read_api` in order to install this module. You will need to add the below to your `~/.netrc` file:
```shell
machine gitlab.teklia.com
@@ -35,7 +35,7 @@ pip install -r requirements.txt
To install DAN directly, you can install it via pip:
```shell
-pip3 install --index-url https://gitlab.teklia.com/api/v4/projects/98/packages/pypi/simple --extra-index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple teklia-dan
+pip install --index-url https://gitlab.teklia.com/api/v4/projects/98/packages/pypi/simple --extra-index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple teklia-dan
```
The `--index-url` argument is required to find the `DAN` package, the `--extra-index-url` argument is needed to find the `nerval` dependency.
@@ -48,6 +48,18 @@ To learn more about the newly installed `teklia-dan` command, make sure to run:
teklia-dan --help
```
+### As an editable package
+
+For development and tests purpose it may be useful to install the project as a editable package with pip.
+
+To install DAN as an editable package, you can install it via pip:
+
+```shell
+pip install --index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple -e .
+```
+
+The `--index-url` argument is required to find the `nerval` dependency.
+
Get started with:
- [Developments](development.md)
--
GitLab