Incorrect resizing during data extraction (max_resize)
The max_resize function does not have the right behavior.
With --max_width 1000 and --max_height 2000, the current behavior is:
- images with
width > max_widthorheight > max_heightare resized so that their height is equal to 2000, keeping the aspect ratio (seemax_resize)
It does not work for in the following cases:
- an image of size (1200, 1500) cannot be extracted because we attempt to extract it with a height larger than its original height (403 error). Expected size: (1000, 1250).
- an image of size (4000, 4000) will be resized to (2000, 2000), so its width will end up being larger than
max_width. Expected size: (1000, 1000).
teklia-dan dataset extract hugin-munin-10k-20230927-151230.sqlite --output humu-10k/ --train-folder 45ea3bc5-af40-424c-98ea-f57f2744802f --val-folder 45ea3bc5-af40-424c-98ea-f57f2744802f --test-folder 45ea3bc5-af40-424c-98ea-f57f2744802f --element-type paragraph --max-width 1250 --max-height 2000
Extracting data from page (8d3932de-8a0f-444e-bd95-54fc471dff50) for split (train): 100%|██████████████████████████████████████████████████████████████████████████████████| 73/73 [00:00<00:00, 259.22it/s]
Extracting data from page (8d3932de-8a0f-444e-bd95-54fc471dff50) for split (val): 100%|████████████████████████████████████████████████████████████████████████████████████| 73/73 [00:00<00:00, 266.18it/s]
Extracting data from page (8d3932de-8a0f-444e-bd95-54fc471dff50) for split (test): 100%|███████████████████████████████████████████████████████████████████████████████████| 73/73 [00:00<00:00, 286.70it/s]
Downloading images: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 309/309 [01:27<00:00, 3.54it/s]
2023-09-27 19:42:44,823 ERROR/dan.datasets.extract.extract: Failed to download 15 image(s).
https://demo.arkindex.org/uploads/iiif/2/18df389f-e158-4d46-a9dc-a288baae9c46/164,137,1935,1328/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/18df389f-e158-4d46-a9dc-a288baae9c46/164,137,1935,1328/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/95db3b03-036e-42eb-950b-d6d02931b475/252,154,1659,1380/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/95db3b03-036e-42eb-950b-d6d02931b475/252,154,1659,1380/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/9d6d2105-84a9-4806-991e-7af1a4d80cc7/150,178,1970,1300/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/9d6d2105-84a9-4806-991e-7af1a4d80cc7/150,178,1970,1300/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/03277bf6-0a67-4068-9fa7-c5181baf8718/614,297,1606,1171/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/03277bf6-0a67-4068-9fa7-c5181baf8718/614,297,1606,1171/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/2d5c5f5f-0018-42b4-acc1-9c222e05aadc/156,251,1930,1023/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/2d5c5f5f-0018-42b4-acc1-9c222e05aadc/156,251,1930,1023/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/18df389f-e158-4d46-a9dc-a288baae9c46/164,137,1935,1328/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/18df389f-e158-4d46-a9dc-a288baae9c46/164,137,1935,1328/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/9d6d2105-84a9-4806-991e-7af1a4d80cc7/150,178,1970,1300/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/9d6d2105-84a9-4806-991e-7af1a4d80cc7/150,178,1970,1300/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/95db3b03-036e-42eb-950b-d6d02931b475/252,154,1659,1380/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/95db3b03-036e-42eb-950b-d6d02931b475/252,154,1659,1380/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/03277bf6-0a67-4068-9fa7-c5181baf8718/614,297,1606,1171/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/03277bf6-0a67-4068-9fa7-c5181baf8718/614,297,1606,1171/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/2d5c5f5f-0018-42b4-acc1-9c222e05aadc/156,251,1930,1023/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/2d5c5f5f-0018-42b4-acc1-9c222e05aadc/156,251,1930,1023/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/18df389f-e158-4d46-a9dc-a288baae9c46/164,137,1935,1328/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/18df389f-e158-4d46-a9dc-a288baae9c46/164,137,1935,1328/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/95db3b03-036e-42eb-950b-d6d02931b475/252,154,1659,1380/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/95db3b03-036e-42eb-950b-d6d02931b475/252,154,1659,1380/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/9d6d2105-84a9-4806-991e-7af1a4d80cc7/150,178,1970,1300/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/9d6d2105-84a9-4806-991e-7af1a4d80cc7/150,178,1970,1300/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/03277bf6-0a67-4068-9fa7-c5181baf8718/614,297,1606,1171/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/03277bf6-0a67-4068-9fa7-c5181baf8718/614,297,1606,1171/,2000/0/default.jpg
https://demo.arkindex.org/uploads/iiif/2/2d5c5f5f-0018-42b4-acc1-9c222e05aadc/156,251,1930,1023/,2000/0/default.jpg: 403 Client Error: Forbidden for url: https://demo.arkindex.org/uploads/iiif/2/2d5c5f5f-0018-42b4-acc1-9c222e05aadc/156,251,1930,1023/,2000/0/default.jpgEdited by Solene Tarride