Specific payload crashes CreateElement
A 500 error is generated on a specific payload, provided by @melodie.boillet
- Extract the raw payload from the script
- Find underlying issue
- Fix it
See https://gitlab.com/arkindex/requests/-/issues/297 for more details, including step to reproduce, and a script to generate the bug
This issue is probably related to #635 (closed), reproducible though
Activity
I don't have access to, https://gitlab.com/teklia/generic_kaldi_htr/-/tree/melodie.boillet-master-patch-24452/arkindex/home-cartularies. Could someone paste it here please ?
-
I reproduced with this payload
{'corpus': '4de05414-7d85-4a44-a7f1-1de5eb4f2a30', 'name': '1', 'parent': 'ab63eae8-8fde-47fc-8eb9-4c3f23983371', 'polygon': [(5113.0, 1977.0), (5123.0, 1977.0), (5137.0, 1969.0), (5164.0, 1945.0), (5206.0, 1945.0), (5220.0, 1939.0), (5236.0, 1947.0), (5258.0, 1946.0), (5261.0, 1943.0), (5265.0, 1946.0), (5268.0, 1943.0), (5275.0, 1943.0), (5280.0, 1947.0), (5290.0, 1947.0), (5301.0, 1958.0), (5311.0, 1960.0), (5316.0, 1965.0), (5321.0, 1962.0), (5340.0, 1974.0), (5346.0, 1969.0), (5354.0, 1975.0), (5364.0, 1965.0), (5369.0, 1969.0), (5376.0, 1962.0), (5383.0, 1961.0), (5396.0, 1970.0), (5401.0, 1967.0), (5409.0, 1969.0), (5421.0, 1979.0), (5431.0, 1978.0), (5448.0, 1964.0), (5465.0, 1968.0), (5492.0, 1945.0), (5496.0, 1945.0), (5513.0, 1963.0), (5520.0, 1961.0), (5523.0, 1965.0), (5533.0, 1965.0), (5545.0, 1961.0), (5565.0, 1963.0), (5572.0, 1957.0), (5589.0, 1956.0), (5601.0, 1945.0), (5612.0, 1949.0), (5624.0, 1949.0), (5629.0, 1944.0), (5639.0, 1946.0), (5678.0, 1988.0), (5688.0, 1992.0), (5710.0, 1974.0), (5715.0, 1973.0), (5720.0, 1978.0), (5727.0, 1973.0), (5742.0, 1972.0), (5768.0, 1992.0), (5777.0, 1985.0), (5785.0, 1986.0), (5802.0, 1971.0), (5809.0, 1977.0), (5816.0, 1970.0), (5822.0, 1970.0), (5839.0, 1983.0), (5854.0, 1983.0), (5871.0, 2001.0), (5880.0, 2001.0), (5909.0, 1974.0), (5916.0, 1971.0), (5922.0, 1976.0), (5931.0, 1973.0), (5934.0, 1977.0), (5939.0, 1977.0), (5950.0, 1969.0), (5954.0, 1974.0), (5965.0, 1968.0), (5973.0, 1973.0), (5980.0, 1967.0), (5983.0, 1969.0), (5988.0, 1966.0), (6000.0, 1966.0), (6002.0, 1906.0), (5977.0, 1910.0), (5974.0, 1906.0), (5956.0, 1899.0), (5946.0, 1889.0), (5941.0, 1893.0), (5934.0, 1890.0), (5914.0, 1906.0), (5910.0, 1905.0), (5898.0, 1917.0), (5890.0, 1918.0), (5876.0, 1926.0), (5873.0, 1922.0), (5860.0, 1917.0), (5851.0, 1918.0), (5843.0, 1913.0), (5835.0, 1914.0), (5827.0, 1905.0), (5818.0, 1902.0), (5812.0, 1896.0), (5803.0, 1899.0), (5799.0, 1896.0), (5790.0, 1897.0), (5776.0, 1908.0), (5761.0, 1911.0), (5753.0, 1911.0), (5748.0, 1906.0), (5743.0, 1910.0), (5730.0, 1908.0), (5717.0, 1898.0), (5703.0, 1900.0), (5681.0, 1920.0), (5670.0, 1921.0), (5666.0, 1917.0), (5659.0, 1919.0), (5639.0, 1916.0), (5624.0, 1924.0), (5616.0, 1915.0), (5597.0, 1909.0), (5592.0, 1911.0), (5587.0, 1907.0), (5572.0, 1910.0), (5564.0, 1918.0), (5554.0, 1916.0), (5537.0, 1909.0), (5534.0, 1905.0), (5532.0, 1907.0), (5519.0, 1896.0), (5513.0, 1896.0), (5495.0, 1878.0), (5485.0, 1880.0), (5473.0, 1873.0), (5458.0, 1887.0), (5449.0, 1887.0), (5441.0, 1895.0), (5432.0, 1894.0), (5424.0, 1902.0), (5417.0, 1900.0), (5409.0, 1903.0), (5396.0, 1891.0), (5390.0, 1896.0), (5381.0, 1889.0), (5372.0, 1894.0), (5362.0, 1886.0), (5340.0, 1896.0), (5322.0, 1897.0), (5310.0, 1893.0), (5293.0, 1877.0), (5286.0, 1882.0), (5276.0, 1880.0), (5262.0, 1884.0), (5249.0, 1875.0), (5237.0, 1878.0), (5228.0, 1868.0), (5223.0, 1869.0), (5218.0, 1864.0), (5193.0, 1863.0), (5179.0, 1872.0), (5172.0, 1872.0), (5166.0, 1868.0), (5161.0, 1873.0), (5151.0, 1871.0), (5145.0, 1876.0), (5134.0, 1875.0), (5125.0, 1883.0), (5117.0, 1883.0), (5113.0, 1977.0)], 'image': 'f5aa2125-017b-4f29-bdca-81de75f8c03a', 'type': 'text_line', 'worker_version': '02bcc248-c065-46aa-ac3d-96d17eb6e196'}And got those logs
File "/home/valentin/dev/backend/arkindex/documents/serializers/elements.py", line 478, in create zone = Zone \ File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/query.py", line 429, in get raise self.model.DoesNotExist( arkindex.images.models.Zone.DoesNotExist: Zone matching query does not exist. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) psycopg2.errors.CheckViolation: new row for relation "images_zone" violates check constraint "zone_polygon_size" DETAIL: Failing row contains (5a530085-2135-4137-8afa-bf3f6feffc20, 2021-04-19 11:31:15.206287+00, 2021-04-19 11:31:15.206306+00, f5aa2125-017b-4f29-bdca-81de75f8c03a, 0102000000A60000000000000000F9B3400000000000E49E40000000000003B4...). The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/contrib/staticfiles/handlers.py", line 76, in __call__ return self.application(environ, start_response) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 133, in __call__ response = self.get_response(request) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/base.py", line 130, in get_response response = self._middleware_chain(request) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/exception.py", line 49, in inner response = response_for_exception(request, exc) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/exception.py", line 103, in response_for_exception response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info()) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/exception.py", line 138, in handle_uncaught_exception return debug.technical_500_response(request, *exc_info) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django_extensions/management/technical_response.py", line 40, in null_technical_500_response raise exc_value.with_traceback(tb) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner response = get_response(request) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/core/handlers/base.py", line 181, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/views/generic/base.py", line 70, in view return self.dispatch(request, *args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch response = self.handle_exception(exc) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception self.raise_uncaught_exception(exc) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception raise exc File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch response = handler(request, *args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/drf_spectacular/utils.py", line 369, in wrapped_method return method(self, request, *args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/generics.py", line 190, in post return self.create(request, *args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/mixins.py", line 19, in create self.perform_create(serializer) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/mixins.py", line 24, in perform_create serializer.save() File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/rest_framework/serializers.py", line 205, in save self.instance = self.create(validated_data) File "/home/valentin/dev/backend/arkindex/documents/serializers/elements.py", line 488, in create zone = image.zones.create(polygon=polygon) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/fields/related_descriptors.py", line 677, in create return super(RelatedManager, self.db_manager(db)).create(**kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/query.py", line 447, in create obj.save(force_insert=True, using=self.db) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/base.py", line 753, in save self.save_base(using=using, force_insert=force_insert, File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/base.py", line 790, in save_base updated = self._save_table( File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/base.py", line 895, in _save_table results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/base.py", line 933, in _do_insert return manager._insert( File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/query.py", line 1254, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql cursor.execute(sql, params) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/debug_toolbar/panels/sql/tracking.py", line 198, in execute return self._record(self.cursor.execute, sql, params) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/debug_toolbar/panels/sql/tracking.py", line 133, in _record return method(sql, params) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django_extensions/management/debug_cursor.py", line 48, in execute return utils.CursorWrapper.execute(self, sql, params) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/utils.py", line 90, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/valentin/.virtualenvs/backend/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) django.db.utils.IntegrityError: new row for relation "images_zone" violates check constraint "zone_polygon_size" DETAIL: Failing row contains (5a530085-2135-4137-8afa-bf3f6feffc20, 2021-04-19 11:31:15.206287+00, 2021-04-19 11:31:15.206306+00, f5aa2125-017b-4f29-bdca-81de75f8c03a, 0102000000A60000000000000000F9B3400000000000E49E40000000000003B4...).Edited by Valentin Rigal mentioned in issue #718 (closed)
After a lot of investigation with @vrigal & @erwanrouchet we are faced with the prod database behaving differently on the same polygon (inserted now, vs already in the DB):
arkindex_prod=# select id, pg_column_size(polygon), st_memsize(ST_normalize(polygon)), st_summary(polygon), st_memsize(polygon), st_memsize(st_geomfromtext(st_astext(polygon))) from images_zone where id in ('628f3404-946a-4eb5-86c5-f91294b507c8', 'a43f6d05-235f-45f6-9379-00b5ea2361de'); id | pg_column_size | st_memsize | st_summary | st_memsize | st_memsize --------------------------------------+----------------+------------+-------------------------------+------------+------------ 628f3404-946a-4eb5-86c5-f91294b507c8 | 770 | 2656 | LineString[B] with 164 points | 2664 | 2656 a43f6d05-235f-45f6-9379-00b5ea2361de | 769 | 2656 | LineString[B] with 164 points | 2656 | 2656 (2 rows)The memory size detected by postgis is
2664instead of2656(8 extra bytes).As we can't find a reasonable way to detect this behaviour (nor its source), we have to lower the maximum number of points for new polygons to 164, without touching the existing constraints: this will give us a bit more of error margin.
mentioned in issue frontend#623 (closed)
mentioned in issue tasks#105 (closed)
mentioned in merge request !1314 (merged)
mentioned in commit 4d09bdfa
Please register or sign in to reply