@@ -6,64 +6,6 @@ For more details about this package, make sure to see the documentation availabl
This is an open-source project, licensed using [the MIT license](https://opensource.org/license/mit/).
## Development
For development and tests purpose it may be useful to install the project as a editable package with pip.
This package is based on a GitLab package registry containing all the nerval source code.
You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) and access to the [nerval repository](https://gitlab.teklia.com/ner/nerval) in order to install this module. You will need to add the below to your `~/.netrc` file:
```shell
machine gitlab.teklia.com
login __token__
password <YOUR_PERSONAL_TOKEN>
```
Then you can install the package as a editable package with pip:
To recreate tox virtual environment (e.g. a dependencies update), you may run `tox -r`.
Run a single test module: `tox -- <test_path>`
Run a single test: `tox -- <test_path>::<test_function>`
The tests use a large file stored via [Git-LFS](https://docs.gitlab.com/ee/topics/git/lfs/). Make sure to run `git-lfs pull` before running them.
### Commands
As unit tests do not test *everything*, it is sometimes necessary to use DAN commands directly to test developments.
#### Training command
The library already has all the documents needed to run the [training command](../usage/train/index.md) on a minimalist dataset. You can use the configuration available at `configs/tests.json`. It is already populated with the parameters used in the unit tests.
```shell
teklia-dan train --config configs/tests.json
```
#### Predict command
The library already has all the documents needed to run the [predict command](../usage/predict/index.md) with a minimalist model. In the `tests/data/prediction` directory, you can run the following command and add any extra parameters you need:
```shell
...
...
@@ -46,12 +55,16 @@ teklia-dan predict \
--output /tmp/dan-predict
```
#### Evaluation command
The library already has all the documents needed to run the [evaluation command](../usage/evaluate/index.md) on a minimalist dataset. You can use the configuration available at `configs/eval.json`. It is already populated with the parameters used in the unit tests.
```shell
teklia-dan evaluate --config configs/eval.json
```
#### Convert command
If you want to evaluate a NER models with you own scripts, you can convert DAN's predictions in [BIO](<https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)>) format, using the [convert command](../usage/convert/index.md).
DAN is based on a GitLab package registry containing all the nerval source code.
You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html)and access to the[nerval repository](https://gitlab.teklia.com/ner/nerval) in order to install this module. You will need to add the below to your `~/.netrc` file:
You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html)with [scope](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#personal-access-token-scopes)`read_api` in order to install this module. You will need to add the below to your `~/.netrc` file:
```shell
machine gitlab.teklia.com
...
...
@@ -35,7 +35,7 @@ pip install -r requirements.txt
To install DAN directly, you can install it via pip: