Memory leak: Catch error and retry (!466) · Merge requests · Automatic Text Recognition / DAN

Merged Manon Blanco requested to merge memory-leak-catch-error-and-retry into main 5 months ago

Nov 12, 2024
- Apply suggestions · bc926a7c
  Manon Blanco authored 4 months ago
  
  bc926a7c
Nov 06, 2024
- Apply suggestions · cc0082d4
  Manon Blanco authored 4 months ago
  
  cc0082d4
Oct 16, 2024
- Fix diff · 6f1733f3
  Manon Blanco authored 4 months ago
  
  6f1733f3
- Display Cuda error · 361c991f
  Manon Blanco authored 4 months ago
  
  361c991f
- Check exit code · d8be0314
  Manon Blanco authored 4 months ago
  
  d8be0314
- Do not use recursive function · 7aec5e2e
  Manon Blanco authored 4 months ago
  
  7aec5e2e
Oct 14, 2024
- Init W&B in the subprocess (not in the parent process) · d6a8a31c
  Manon Blanco authored 4 months ago
  
  d6a8a31c
Oct 10, 2024
- Nits · 359ce276
  Manon Blanco authored 5 months ago
  
  359ce276
- Some cleanup · 6cf8e620
  Manon Blanco authored 5 months ago
  
  6cf8e620
- start training in subprocess · ff57bd1c
  Manon Blanco authored 5 months ago and Manon Blanco committed 5 months ago
  
  ff57bd1c
- Try something else · 895a7d1f
  Manon Blanco authored 5 months ago
  
  895a7d1f
- Store memory snapshots · 96eb3024
  Manon Blanco authored 5 months ago
  
  96eb3024
- Restart full training · a48a7895
  Manon Blanco authored 5 months ago
  
  a48a7895
- Store memory snapshots · 44217c0e
  Manon Blanco authored 6 months ago
  
  44217c0e
Oct 08, 2024
- Try something else (again) · 87c3cef6
  Manon Blanco authored 5 months ago
  
  87c3cef6
- Try something else · 62b906a6
  Manon Blanco authored 5 months ago
  
  62b906a6
- Memory leak: Catch error and retry on smaller batches · c06bf2e5
  Manon Blanco authored 5 months ago
  
  c06bf2e5

Memory leak: Catch error and retry

Merge request reports

Activity