Skip to content

Support git lfs when building a docker image

Some git repositories use git LFS to store huge ML models.

The docker build task need to : 1. check if there is a .gitattributes file 2. check if that file contain a mention of lfs: (filter|diff|merge)=lfs 3. run a git lfs pull if both previous check are successful

Edit: We can't do the above, as the current download only retrieves the files, not the .git metadatas (so git lfs or any git operation does not work).

So we need to do a propre git clone. Fortunately we already have all the pieces in the task & repository api endpoint:

  • repository HTTP url
  • user's oauth token

To summarize, the download_archive method must be replaced with a clone_repository that does a git clone https://oauth2:<USER_TOKEN>@<REPO_URL through suprocess.

Git LFS must (of course) be added in the base docker build image (depends on #51 (closed))

Edited by Bastien Abadie