Extract data using an SQLite export

As many of our ML projects, training a DAN model should be easy and relatively fast to launch. The data extraction part is currently done by API calls. A more efficient way would be to use an SQLite export and making raw SQL queries on them to generate the dataset.

This issue is mostly about replacing API calls by SQL queries.

changed milestone to %ML Prod - October 2022 n°2

added P3 label

changed milestone to %ML Prod - November 2022 n°1

changed milestone to %ML Prod - November 2022 n°2

changed milestone to %ML Prod - Next

changed milestone to %ML Prod - December 2022 n°2

changed milestone to %ML Prod - Next

removed P3 label

added P2 label

This should be done soonish (good job for @NolanB ?)

This is slowing some projects https://redmine.teklia.com/issues/2942

changed milestone to %ML Prod - January 2023 n°2

assigned to @NolanB

changed the description

Please publish ASAP

mentioned in merge request !50 (merged)

By Nolan on 2023-01-27T10:10:25 (imported from GitLab)

I made sql queries that I tested with another script. Now I try to integrate them instead of api requests.

MR : https://gitlab.com/teklia/atr/dan/-/merge_requests/50

By Nolan on 2023-01-27T10:13:05 (imported from GitLab)

changed milestone to %ML Prod - February 2023 n°1

assigned to @schneider-y and unassigned @NolanB

changed milestone to %ML Prod - February 2023 n°2

This will be needed for our next projects, this should be done next week

added P1 label and removed P2 label

changed milestone to %ML Prod - March 2023 n°1

changed milestone to %ML Prod - March 2023 n°2

added P2 label and removed P1 label

mentioned in issue #39

closed