Skip to content
Snippets Groups Projects
Commit fb4c9a94 authored by Manon Blanco's avatar Manon Blanco Committed by Yoann Schneider
Browse files

Document the scope to use for the GitLab token and remove duplicated documentation

parent dab1d6cd
No related branches found
No related tags found
1 merge request!395Document the scope to use for the GitLab token and remove duplicated documentation
......@@ -6,64 +6,6 @@ For more details about this package, make sure to see the documentation availabl
This is an open-source project, licensed using [the MIT license](https://opensource.org/license/mit/).
## Development
For development and tests purpose it may be useful to install the project as a editable package with pip.
This package is based on a GitLab package registry containing all the nerval source code.
You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) and access to the [nerval repository](https://gitlab.teklia.com/ner/nerval) in order to install this module. You will need to add the below to your `~/.netrc` file:
```shell
machine gitlab.teklia.com
login __token__
password <YOUR_PERSONAL_TOKEN>
```
Then you can install the package as a editable package with pip:
```shell
pip3 install --index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple -e .
```
### Linter
Code syntax is analyzed before submitting the code.\
To run the linter tools suite you may use pre-commit.
```shell
pip install pre-commit
pre-commit run -a
```
### Run tests
Tests are executed with `tox` using [pytest](https://pytest.org).
To install `tox`,
```shell
pip install tox
tox
```
To reload the test virtual environment you can use `tox -r`
Run a single test module: `tox -- <test_path>`
Run a single test: `tox -- <test_path>::<test_function>`
The tests use a large file stored via [Git-LFS](https://docs.gitlab.com/ee/topics/git/lfs/). Make sure to run `git-lfs pull` before running them.
### Update documentation
Please keep the documentation updated when modifying or adding features.
It's pretty easy to do:
```shell
pip install -r doc-requirements.txt
mkdocs serve
```
You can then write in Markdown in the relevant `docs/*.md` files, and see live output on <http://localhost:8000>.
## Inference
To apply DAN to an image, one needs to first add a few imports and to load an image. Note that the image should be in RGB.
......
......@@ -26,16 +26,25 @@ tox
To recreate tox virtual environment (e.g. a dependencies update), you may run `tox -r`.
Run a single test module: `tox -- <test_path>`
Run a single test: `tox -- <test_path>::<test_function>`
The tests use a large file stored via [Git-LFS](https://docs.gitlab.com/ee/topics/git/lfs/). Make sure to run `git-lfs pull` before running them.
### Commands
As unit tests do not test *everything*, it is sometimes necessary to use DAN commands directly to test developments.
#### Training command
The library already has all the documents needed to run the [training command](../usage/train/index.md) on a minimalist dataset. You can use the configuration available at `configs/tests.json`. It is already populated with the parameters used in the unit tests.
```shell
teklia-dan train --config configs/tests.json
```
#### Predict command
The library already has all the documents needed to run the [predict command](../usage/predict/index.md) with a minimalist model. In the `tests/data/prediction` directory, you can run the following command and add any extra parameters you need:
```shell
......@@ -46,12 +55,16 @@ teklia-dan predict \
--output /tmp/dan-predict
```
#### Evaluation command
The library already has all the documents needed to run the [evaluation command](../usage/evaluate/index.md) on a minimalist dataset. You can use the configuration available at `configs/eval.json`. It is already populated with the parameters used in the unit tests.
```shell
teklia-dan evaluate --config configs/eval.json
```
#### Convert command
If you want to evaluate a NER models with you own scripts, you can convert DAN's predictions in [BIO](<https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)>) format, using the [convert command](../usage/convert/index.md).
```shell
......
......@@ -3,7 +3,7 @@
## Installation
DAN is based on a GitLab package registry containing all the nerval source code.
You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) and access to the [nerval repository](https://gitlab.teklia.com/ner/nerval) in order to install this module. You will need to add the below to your `~/.netrc` file:
You need [a personal access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with [scope](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#personal-access-token-scopes) `read_api` in order to install this module. You will need to add the below to your `~/.netrc` file:
```shell
machine gitlab.teklia.com
......@@ -35,7 +35,7 @@ pip install -r requirements.txt
To install DAN directly, you can install it via pip:
```shell
pip3 install --index-url https://gitlab.teklia.com/api/v4/projects/98/packages/pypi/simple --extra-index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple teklia-dan
pip install --index-url https://gitlab.teklia.com/api/v4/projects/98/packages/pypi/simple --extra-index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple teklia-dan
```
The `--index-url` argument is required to find the `DAN` package, the `--extra-index-url` argument is needed to find the `nerval` dependency.
......@@ -48,6 +48,18 @@ To learn more about the newly installed `teklia-dan` command, make sure to run:
teklia-dan --help
```
### As an editable package
For development and tests purpose it may be useful to install the project as a editable package with pip.
To install DAN as an editable package, you can install it via pip:
```shell
pip install --index-url https://gitlab.teklia.com/api/v4/projects/210/packages/pypi/simple -e .
```
The `--index-url` argument is required to find the `nerval` dependency.
Get started with:
- [Developments](development.md)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment