Skip to content

Wiki page on troubleshooting intermittent or hard to reproduce test failures

  • How to identify if a test impacts another test (what led to https://gitlab.com/arkindex/backend/-/merge_requests/1644)
  • How to reproduce a test in CI conditions (check that https://wiki.vpn/en/arkindex/dev-doc/ci-test is up to date and link to it)
  • Using manage.py test options
    • Filter tests by name or pattern
    • Run tests in reverse
    • Run tests in a random order
    • PDB
    • Keep the test DB after running the tests
    • Stop at the first failure
    • Testing repeatedly
  • Browsing the test DB with manage.py testserver
  • Getting more debug information
    • pprint
    • rich.inspect
    • rich.traceback
    • Logging SQL queries
    • HTML test coverage report
  • Handling intermittent failures before they are fixed
    • Skipping tests
    • Expected failures
  • Checklist (things you ought to look at if you really don't know what could cause a test to fail somewhere, but not somewhere else)