Skip to content

Make atr-data-generator compatible with new Arkindex exports

I tried extracting data from a recent Arkindex export (2024-08-06) and got the following error:

starride@straton:~/data$ atr-data-generator extract --database-path hugin-munin-norhand-v3-20240806-204721.sqlite --config data_extraction_norhand_v3_ufcn_pylaia.yml 
Traceback (most recent call last):
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 3252, in execute_sql
    cursor.execute(sql, params or ())
sqlite3.OperationalError: no such column: t1.worker_version_id

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/users/starride/miniconda3/envs/pylaia_clean/bin/atr-data-generator", line 8, in <module>
    sys.exit(main())
  File "/home/users/starride/git_repos/data-generator/atr_data_generator/cli.py", line 22, in main
    _func(**kwargs)
  File "/home/users/starride/git_repos/data-generator/atr_data_generator/extract/__init__.py", line 124, in main
    data_generator(**kwargs).run(db_path=database_path)
  File "/home/users/starride/git_repos/data-generator/atr_data_generator/extract/base.py", line 202, in run
    self.process_parent(parent.element, split)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 4657, in __get__
    return self.get_rel_instance(instance)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 4648, in get_rel_instance
    obj = self.rel_model.get(self.field.rel_field == value)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 6712, in get
    return sq.get()
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 7160, in get
    return clone.execute(database)[0]
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 1972, in inner
    return method(self, database, *args, **kwargs)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 2043, in execute
    return self._execute(database)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 2216, in _execute
    cursor = database.execute(self)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 3260, in execute
    return self.execute_sql(sql, params)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 3250, in execute_sql
    with __exception_wrapper__:
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 3020, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 192, in reraise
    raise value.with_traceback(tb)
  File "/home/users/starride/miniconda3/envs/pylaia_clean/lib/python3.10/site-packages/peewee.py", line 3252, in execute_sql
    cursor.execute(sql, params or ())
peewee.OperationalError: no such column: t1.worker_version_id

It works as expected with older exports (2024-07-31).