Skip to content

AttributeError: 'NoneType' object has no attribute 'shape'

Trying to finetune doc-ufcn-generic-historical-line_model.pth with my own data. I followed the docs in creating the dataset.

(env-py3.10) incognito@DESKTOP-NHKR7QL:~/doc-ufcn$ ls -al train/doc-ufcn-samaritan-v1/
total 32
drwxr-xr-x 5 incognito incognito 4096 Jun 27 22:37 .
drwxr-xr-x 3 incognito incognito 4096 Jun 25 21:56 ..
-rwxr-xr-x 1 incognito incognito 1588 Jun 27 21:48 alto2labels_json.py
-rw-r--r-- 1 incognito incognito 1091 Jun 27 22:00 config.json
-rw-r--r-- 1 incognito incognito  520 Jun 27 22:37 experiments.csv
drwxr-xr-x 5 incognito incognito 4096 Jun 27 22:34 test
drwxr-xr-x 5 incognito incognito 4096 Jun 27 22:35 train
drwxr-xr-x 5 incognito incognito 4096 Jun 27 22:35 val
(env-py3.10) incognito@DESKTOP-NHKR7QL:~/doc-ufcn$ ls -al train/doc-ufcn-samaritan-v1/train/
total 28
drwxr-xr-x 5 incognito incognito  4096 Jun 27 22:35 .
drwxr-xr-x 5 incognito incognito  4096 Jun 27 22:37 ..
drwxr-xr-x 2 incognito incognito 12288 Jun 27 22:35 images
drwxr-xr-x 2 incognito incognito  4096 Jun 25 21:57 labels
drwxr-xr-x 2 incognito incognito  4096 Jun 27 22:35 labels_json
(env-py3.10) incognito@DESKTOP-NHKR7QL:~/doc-ufcn$ ls -al train/doc-ufcn-samaritan-v1/train/labels_json/|head -n 4
total 9820
drwxr-xr-x 2 incognito incognito   4096 Jun 27 22:35 .
drwxr-xr-x 5 incognito incognito   4096 Jun 27 22:35 ..
-rw-r--r-- 1 incognito incognito 125299 Jun 27 22:35 bcuf_ms_l_2057004.json
(env-py3.10) incognito@DESKTOP-NHKR7QL:~/doc-ufcn$ ls -al train/doc-ufcn-samaritan-v1/train/images/|head -n 4
total 186724
drwxr-xr-x 2 incognito incognito   12288 Jun 27 22:35 .
drwxr-xr-x 5 incognito incognito    4096 Jun 27 22:35 ..
-rw-r--r-- 1 incognito incognito 1850666 Jun 27 22:05 bcuf_ms_l_2057004.jpg

Example of an image file: bcuf_ms_l_2057004

Example of a json file: bcuf_ms_l_2057004.json

My experiments.csv file: experiments.csv

My experiments_config.json file: experiments_config.json

When running the training task I get this error:

(env-py3.10) incognito@DESKTOP-NHKR7QL:~/doc-ufcn$ ./run_dla_experiment.sh -c experiments.csv
2024-06-27 23:03:22,981 - INFO - Retrieved 1 experiment configuration from experiments.csv

Running experiment 1 with ./tmp/1_doc-ufcn-samaritan-v1.json
2024-06-27 23:03:25,573 - INFO - Saved configuration in /home/incognito/doc-ufcn/runs/doc_ufcn_samaritan_v1/doc-ufcn-samaritan-v1.json
Computing parameters (prog):   2%|▏         | 3/200 [00:00<00:10, 19.07it/s]
Traceback (most recent call last):
  File "/home/incognito/doc-ufcn/doc_ufcn/train/cli.py", line 50, in <module>
    main()
  File "/home/incognito/doc-ufcn/doc_ufcn/train/cli.py", line 46, in main
    run(config)
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/doc_ufcn/train/experiment.py", line 232, in run
    run_experiment(config=config, num_workers=num_workers, mlflow_logging=False)
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/doc_ufcn/train/experiment.py", line 242, in run_experiment
    normalization_params(
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/doc_ufcn/train/normalization_params.py", line 45, in run
    for data in tqdm(loader, desc="Computing parameters (prog)"):
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/tqdm/std.py", line 1182, in __iter__
    for obj in iterable:
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 630, in __next__
    data = self._next_data()
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
    return self._process_data(data)
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
    data.reraise()
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/_utils.py", line 694, in reraise
    raise exception
AttributeError: Caught AttributeError in DataLoader worker process 1.
Original Traceback (most recent call last):
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/incognito/doc-ufcn/env-py3.10/lib/python3.10/site-packages/doc_ufcn/train/utils/preprocessing.py", line 132, in __getitem__
    if len(image.shape) < 3:
AttributeError: 'NoneType' object has no attribute 'shape'
Edited by Teodor Bors