Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
B
Base Worker
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Workers
Base Worker
Commits
e9418f9a
Verified
Commit
e9418f9a
authored
2 years ago
by
Yoann Schneider
Browse files
Options
Downloads
Patches
Plain Diff
draft local execution
parent
4561df33
No related branches found
No related tags found
1 merge request
!292
Draft: Document local execution
Pipeline
#80032
passed
1 year ago
Stage: test
Stage: build
Stage: release
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
docs/contents/workers/run-local.md
+29
-38
29 additions, 38 deletions
docs/contents/workers/run-local.md
with
29 additions
and
38 deletions
docs/contents/workers/run-local.md
+
29
−
38
View file @
e9418f9a
...
...
@@ -3,77 +3,62 @@
Once you have implemented a worker, you can run it on some Arkindex elements
on your own machine to test it.
!!! warning
This section has been deprecated as of the latest version of base-worker.
## Retrieving credentials
For a worker to run properly, you will need two types of credentials:
-
An API token that gives the worker access to the API
-
A worker version ID that lets the worker send results to Arkindex and report
that those come from this particular worker version
-
An API token that gives the worker access to the API and to the corpora you have access to,
-
A worker run ID to disable read-only mode
Read-only mode was designed to prevent excessive publication on Arkindex.
Creating elements, transcriptions and entities is far easier and faster than deleting them. Thus, disabling it is discouraged as
any publication should be done in Arkindex in a process.
### Retrieving a token
For the worker to run, you will need an Arkindex authentication token.
You can use your own account's token when testing on your own machine.
You can retrieve your personal API Token from your
[
profile page
](
https://doc.arkindex.org/users/auth/
index.md
#personal-token
)
.
You can retrieve your personal API Token from your
[
profile page
](
https://doc.arkindex.org/users/auth/#personal-token
)
.
### Retrieving a worker
versio
n ID
### Retrieving a worker
ru
n ID
A worker
versio
n ID will be required in order to publish results. If your worker
A worker
ru
n ID will be required in order to publish results. If your worker
does not create any Arkindex element, classification, transcription, etc., you
may skip this step.
If this particular worker was already configured on this instance, you can use
its existing worker version ID; otherwise, you will need to ask an Arkindex
administrator to create a fake version ID.
#### To retrieve a worker
versio
n ID from an existing
worker
#### To retrieve a worker
ru
n ID from an existing
process
1.
Open a web browser and browse to the Arkindex instance.
2.
In the top
-right user menu, click on
**My repositorie
s**
.
2.
In the top
navigation bar, click on
**Proces
s**
.
3.
Click on your worker, listed in the
**Workers**
column
.
3.
Select any process with
`Workers`
mode to view its status page
.
4.
Rewrite the URL in your browser's address bar, to look like
`https://<arkindex_url>/api/v1/
worker
s/
<
worker
_id>/version
s/`
`https://<arkindex_url>/api/v1/
proces
s/workers/`
- Replace `process` by `api/v1`
-
Add a slash character (`/`)
at the end
- Replace `process` by `api/v1
/process
`
-
Remove the `0`
at the end
In the JSON output from this API endpoint, the first value next to
`"id"`
is
the worker
version ID
.
the worker
run ID of the first
`WorkerRun`
linked to this process
.
#### To create a fake worker as an administrator
#### To create a fake worker
run
as an administrator
This action can only be done as an Arkindex administrator with shell access.
1.
In the backend's Docker image, run:
In the backend's Docker image, run:
```
arkindex fake_worker_
version --name <NAME> --slug <SLUG> --url <URL>
arkindex fake_worker_
run ...
```
Replace
`<NAME>`
,
`<SLUG>`
and
`<URL>`
with the name, slug and GitLab
repository URL, respectively.
A Git repository is created with a fake OAuth access token. A fake Git revision
is added to this repository, and a fake worker version from a fake worker is
linked to this revision. You should get the following output:
```
Created a worker
versio
n: 392bd299-bc8f-4ec6-aa3c-e6503ecc7730
Created a worker
ru
n: 392bd299-bc8f-4ec6-aa3c-e6503ecc7730
```
!!! warning
This feature should only be used when a normal worker cannot be created using the Git workflow.
## Setting credentials
In a shell you need to set 3 environment variables to transmit your credentials
...
...
@@ -85,9 +70,9 @@ and Arkindex instance information to the worker:
`ARKINDEX_API_TOKEN`
: The API token you retrieved earlier, on your profile page.
`WORKER_
VERSIO
N_ID`
`
ARKINDEX_
WORKER_
RU
N_ID`
: The worker version ID you retrieved earlier. Can be omitted if the worker does
not create new data in Arkindex.
not create new data in Arkindex
or you want to stay in read-only mode
.
### To set credentials for your worker
...
...
@@ -97,7 +82,7 @@ not create new data in Arkindex.
```
sh
export
ARKINDEX_API_URL
=
"https://arkindex.teklia.com"
export
ARKINDEX_API_TOKEN
=
"YOUR_TOKEN_HERE"
export
WORKER_
VERSIO
N_ID
=
"xxxxx"
export
ARKINDEX_
WORKER_
RU
N_ID
=
"xxxxx"
```
!!! warning
...
...
@@ -118,3 +103,9 @@ in the browser's address bar when browsing an element on Arkindex.
2.
Run
`worker-X`
, where
`X`
is the slug of your worker, followed by
`--element=Y`
where
`Y`
is the ID of an element. You can repeat
`--element`
as many times as you need to process multiple elements.
### Use secrets
Some workers use secrets to store sensitive information. While running on Arkindex, the worker
will make API calls to retrieve them. You can also mimic that process by setting up your local Arkindex secrets
by following the tutorial described
[
on the Arkindex documentation
](
https://doc.arkindex.org/secrets/local/
)
.
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment