Skip to content

Use more properties

Refs https://redmine.teklia.com/issues/7754

  • BaseWorker
    • two endpoints are called when configuring a worker (RetrieveWorkerRun and RetrieveTaskFromAgent). We can cache these results (cached_property) to call these endpoints once and use the results later.
    • support_cache should be still an attribute
    • cache_path can be a variable
    • all other attributes can be a cached_property
  • ElementsWorker
    • attributes can be moved to the dedicated mixin
Details

BaseWorker

  • worker_run use a cached_property to store RetrieveWorkerRun results
  • task use a cached_property to store RetrieveTaskFromAgent results

Misc

attribute default value configure configure_for_developers
work_dir Path(os.environ["PONOS_DATA"], "current") Path(os.environ.get("XDG_DATA_HOME", "~/.local/share"), "arkindex").expanduser()
process_information None worker_run["process"]
_corpus_id None worker_run["process"]["corpus"] os.environ.get("ARKINDEX_CORPUS_ID")
  • work_dir use a cached_property
  • process_information use a cached_property
  • _corpus_id remove it -> corpus_id use a cached_property

Cache

attribute default value configure configure_for_developers
support_cache support_cache
use_cache False see in configure_cache
cache_path see in configure_cache
  • support_cache use an attribute
  • use_cache use a cached_property
  • cache_path use a variable (only used in configure_cache)

Task

attribute default value configure or configure_cache configure_for_developers
task_id os.environ.get("PONOS_TASK")
task_chunk os.environ.get("ARKINDEX_TASK_CHUNK")
task_data_dir Path(os.environ.get("PONOS_DATA", "/data"))
task_parents [] task["parents"]
  • task_id use a cached_property
  • task_chunk use a cached_property
  • task_data_dir use a cached_property
  • task_parents use a cached_property

Worker/Model

attribute default value configure configure_for_developers
worker_run_id os.environ.get("ARKINDEX_WORKER_RUN_ID")
worker_details {} worker_run["worker_version"]["worker"] {"name": "Local worker"}
model_version_id None worker_run.get("model_version", {}).get("id", None) os.environ.get("ARKINDEX_MODEL_VERSION_ID")
model_details {} worker_run.get("model_version", {}).get("model", {}) {"id": os.environ.get("ARKINDEX_MODEL_ID")}
  • worker_run_id use a cached_property
  • worker_details use a cached_property
  • model_version_id use a cached_property
  • model_details use a cached_property

Configuration/Secrets

attribute default value configure configure_for_developers
user_configuration {} worker_version["configuration"][" "user_configuration""] + worker_run.get("configuration", {}).get("configuration", {})
model_configuration {} worker_run.get("model_version", {}).get("configuration", {})
config {}  worker_run["worker_version"]["configuration"].get("configuration", {}) yaml.safe_load(self.args.config)
secrets {} {name: self.load_secret(Path(name)) for name in worker_run["worker_version"]["configuration"].get("secrets", [])} {name: self.load_secret(Path(name)) for name in self.config.get("secrets", [])}
  • user_configuration use a cached_property
  • model_configuration use a cached_property
  • config use a cached_property
  • secrets use a cached_property

ElementsWorker

attribute default value
classes {}
entity_types {}
_worker_version_cache {}
  • classes move attribute in ClassificationMixin
  • entity_types move attribute in EntityMixin
  • _worker_version_cache move attribute in WorkerVersionMixin