Skip to content

PostGIS integration

Erwan Rouchet requested to merge postgis into master

Requires !866 (merged), !910 (merged), !913 (merged), frontend!661 (merged), architecture#14, and architecture!123
Closes #163 (closed)
Please make a base-postgis tag
Before merging, please remove the WIP commit that adds base builds to this MR (for dev.arkindex.teklia.com).

TODO

  • Add PostGIS to the database Dockerfile
  • Add PostGIS dependencies to the backend Dockerfiles
  • Polygon migration
  • Zone deduplication
  • Unique constraint
  • Unique constraint without MD5
  • ST_IsClosed constraint
  • Integer coordinates constraint
  • Drop arkindex.project.polygon
  • Update the entire codebase
  • Add unit tests
  • Update js/helpers/polygon.js in the frontend
  • Mitigate ST_Normalize issue
  • Benchmarks
  • Update dev setup docs

Possible follow-ups

  • #372 — This still allows "bowtie polygons"; we could use a ST_IsRing constraint for that, but it requires extra work due to a lot of existing data not matching this constraint.
  • images/consumers.py could maybe be removed; this would require some benchmarking on the ST_Scale function #413 (closed)
  • Create an arkindex cleanup command to remove unused zones, unused images and unused image servers: there are over 17 million useless zones in prod and 10 million in preprod. This would be similar in code complexity to the provided deduplicate_zones command. The transcription simplification would make this command even easier #414 (closed)

Migrating

Migrate normally. If images.0005 fails and complains about duplicated polygons, run arkindex deduplicate_zones, then migrate again.

Edited by Erwan Rouchet

Merge request reports

Loading