Support git lfs when building a docker image
Some git repositories use git LFS to store huge ML models.
The docker build task need to :
1. check if there is a
.gitattributes
file2. check if that file contain a mention of lfs:
(filter|diff|merge)=lfs
3. run a git lfs pull
if both previous check are successful
Edit: We can't do the above, as the current download only retrieves the files, not the .git
metadatas (so git lfs
or any git operation does not work).
So we need to do a propre git clone
. Fortunately we already have all the pieces in the task & repository api endpoint:
- repository HTTP url
- user's oauth token
To summarize, the download_archive
method must be replaced with a clone_repository
that does a git clone https://oauth2:<USER_TOKEN>@<REPO_URL
through suprocess.
Git LFS must (of course) be added in the base docker build image (depends on #51 (closed))