Skip to content

RecursionError or segfault when starting a process with two runs of the same version depending on each other

Sentry issue: ARKINDEX-BACKEND-1Q9

  1. Create a process.

  2. Add the same worker version twice. This is not possible in the current frontend, but is possible with the API client or after frontend#1267 (closed)

  3. Make one worker run depend on the other worker run. Do that only once, or you will get #1589 (closed).

  4. Start the process.

  5. Locally, get a HTTP 500 for a RecursionError. In preprod, get a HTTP 502 and a segmentation fault from Gunicorn.

    [Thu Aug 24 16:57:11 2023] gunicorn: worke[1558468]: segfault at 7ffcf9b63e00 ip 00007fa188eef297 sp 00007ffcf9b63df0 error 6 in libpython3.10.so.1.0[7fa188dd5000+1a1000]
    [Thu Aug 24 16:57:11 2023] Code: ff e9 0c df ef ff 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 41 57 41 56 41 55 41 54 49 89 cc 55 53 48 89 d3 48 81 ec 98 03 00 00 <48> 89 7c 24 10 48 89 d7 48 89 74 24 18 be 3a 00 00 00 48 89 54 24
  6. Cry.

This might have the same root cause as #1590 (closed), but is worse since it can make a Gunicorn worker crash entirely instead of just a regular exception. Spamming the request might also cause a significant increase in RAM usage since there will be lots of huge stacks.