Use more properties
Refs https://redmine.teklia.com/issues/7754
-
BaseWorker
- two endpoints are called when configuring a worker (
RetrieveWorkerRun
andRetrieveTaskFromAgent
). We can cache these results (cached_property
) to call these endpoints once and use the results later. -
support_cache
should be still an attribute -
cache_path
can be a variable - all other attributes can be a
cached_property
- two endpoints are called when configuring a worker (
-
ElementsWorker
- attributes can be moved to the dedicated mixin
Details
BaseWorker
-
worker_run
use acached_property
to storeRetrieveWorkerRun
results -
task
use acached_property
to storeRetrieveTaskFromAgent
results
Misc
attribute | default value | configure | configure_for_developers |
---|---|---|---|
work_dir |
Path(os.environ["PONOS_DATA"], "current") |
Path(os.environ.get("XDG_DATA_HOME", "~/.local/share"), "arkindex").expanduser() |
|
process_information |
None |
worker_run["process"] |
|
_corpus_id |
None |
worker_run["process"]["corpus"] |
os.environ.get("ARKINDEX_CORPUS_ID") |
-
work_dir
use acached_property
-
process_information
use acached_property
-
_corpus_id
remove it ->corpus_id
use acached_property
Cache
attribute | default value | configure | configure_for_developers |
---|---|---|---|
support_cache |
support_cache |
||
use_cache |
False |
see in configure_cache
|
|
cache_path |
see in configure_cache
|
-
support_cache
use an attribute -
use_cache
use acached_property
-
cache_path
use a variable (only used inconfigure_cache
)
Task
attribute | default value | configure or configure_cache | configure_for_developers |
---|---|---|---|
task_id |
os.environ.get("PONOS_TASK") |
||
task_chunk |
os.environ.get("ARKINDEX_TASK_CHUNK") |
||
task_data_dir |
Path(os.environ.get("PONOS_DATA", "/data")) |
||
task_parents |
[] |
task["parents"] |
-
task_id
use acached_property
-
task_chunk
use acached_property
-
task_data_dir
use acached_property
-
task_parents
use acached_property
Worker/Model
attribute | default value | configure | configure_for_developers |
---|---|---|---|
worker_run_id |
os.environ.get("ARKINDEX_WORKER_RUN_ID") |
||
worker_details |
{} |
worker_run["worker_version"]["worker"] |
{"name": "Local worker"} |
model_version_id |
None |
worker_run.get("model_version", {}).get("id", None) |
os.environ.get("ARKINDEX_MODEL_VERSION_ID") |
model_details |
{} |
worker_run.get("model_version", {}).get("model", {}) |
{"id": os.environ.get("ARKINDEX_MODEL_ID")} |
-
worker_run_id
use acached_property
-
worker_details
use acached_property
-
model_version_id
use acached_property
-
model_details
use acached_property
Configuration/Secrets
attribute | default value | configure | configure_for_developers |
---|---|---|---|
user_configuration |
{} |
worker_version["configuration"][" "user_configuration""] + worker_run.get("configuration", {}).get("configuration", {})
|
|
model_configuration |
{} |
worker_run.get("model_version", {}).get("configuration", {}) |
|
config |
{} |
worker_run["worker_version"]["configuration"].get("configuration", {}) |
yaml.safe_load(self.args.config) |
secrets |
{} |
{name: self.load_secret(Path(name)) for name in worker_run["worker_version"]["configuration"].get("secrets", [])} |
{name: self.load_secret(Path(name)) for name in self.config.get("secrets", [])} |
-
user_configuration
use acached_property
-
model_configuration
use acached_property
-
config
use acached_property
-
secrets
use acached_property
ElementsWorker
attribute | default value |
---|---|
classes |
{} |
entity_types |
{} |
_worker_version_cache |
{} |
-
classes
move attribute inClassificationMixin
-
entity_types
move attribute inEntityMixin
-
_worker_version_cache
move attribute inWorkerVersionMixin