Commit d4eef73
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -165,5 +165,5 @@ cython_debug/ | |||
| 165 | 165 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. | |
| 166 | 166 | .idea/ | |
| 167 | 167 | ||
| 168 | - # model .bin files | ||
| 169 | - docker/auto_docker/*.bin | ||
| 168 | + # downloaded model .bin files | ||
| 169 | + docker/open_llama/*.bin | ||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -7,16 +7,21 @@ | |||
| 7 | 7 | **Note #2:** NVidia GPU CuBLAS support requires a NVidia GPU with sufficient VRAM (approximately as much as the size above) and Docker NVidia support (see [container-toolkit/install-guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)) | |
| 8 | 8 | ||
| 9 | 9 | # Simple Dockerfiles for building the llama-cpp-python server with external model bin files | |
| 10 | - - `./openblas_simple/Dockerfile` - a simple Dockerfile for non-GPU OpenBLAS, where the model is located outside the Docker image | ||
| 11 | - - `cd ./openblas_simple` | ||
| 12 | - - `docker build -t openblas_simple .` | ||
| 13 | - - `docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t openblas_simple` | ||
| 14 | - where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. | ||
| 15 | - - `./cuda_simple/Dockerfile` - a simple Dockerfile for CUDA accelerated CuBLAS, where the model is located outside the Docker image | ||
| 16 | - - `cd ./cuda_simple` | ||
| 17 | - - `docker build -t cuda_simple .` | ||
| 18 | - - `docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t cuda_simple` | ||
| 19 | - where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. | ||
| 10 | + ## openblas_simple - a simple Dockerfile for non-GPU OpenBLAS, where the model is located outside the Docker image | ||
| 11 | + ``` | ||
| 12 | + cd ./openblas_simple | ||
| 13 | + docker build -t openblas_simple . | ||
| 14 | + docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t openblas_simple | ||
| 15 | + ``` | ||
| 16 | + where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. | ||
| 17 | + | ||
| 18 | + ## cuda_simple - a simple Dockerfile for CUDA accelerated CuBLAS, where the model is located outside the Docker image | ||
| 19 | + ``` | ||
| 20 | + cd ./cuda_simple | ||
| 21 | + docker build -t cuda_simple . | ||
| 22 | + docker run -e USE_MLOCK=0 -e MODEL=/var/model/<model-path> -v <model-root-path>:/var/model -t cuda_simple | ||
| 23 | + ``` | ||
| 24 | + where `<model-root-path>/<model-path>` is the full path to the model file on the Docker host system. | ||
| 20 | 25 | ||
| 21 | 26 | # "Open-Llama-in-a-box" - Download a MIT licensed Open Llama model and install into a Docker image that runs an OpenBLAS-enabled llama-cpp-python server | |
| 22 | 27 | ``` | |
0 commit comments