CI tests using tox are ignoring the OpenAPI schema artifact
While troubleshooting schema stuff on base-worker!5, we discovered that all projects using tox
for unit tests and requiring the API client do not use the backend-openapi
artifact from the backend's CI, even when it is downloaded and the variable is set.
- In this pipeline, unit tests were properly running but 46 tests took 156s. 3.3s per test is pretty close to the ~3 seconds it takes for an API client to load the schema from
arkindex.teklia.com
. After much investigation, it turned out thattox
is probably not letting theARKINDEX_API_SCHEMA_URL
through, causing the client to fall back to the prod. - To test this, a commit was added with a new option in tox:
passenv = ARKINDEX_API_SCHEMA_URL
. This succesfully broke the CI as the schema artifact was not being downloaded here. - Another commit added the artifact downloading, and everything worked again in only 60s. More optimizations are possible but are not related to this situation.
It turns out this occurs in every project currently using tox, and we could not find out because using and testing new endpoints before they are released is rather rare. No errors or warnings ever occurred because the schema is retrieved from arkindex.teklia.com
instead, and since tests in most projects are optimized to avoid loading the OpenAPI schema too many times, the overall impact on test execution was minimal.