It is possible to compile and run a recent llama.cpp with gcc 8.5 and nvcc 10.2 (latest supported CUDA compiler from Nvidia for the 2019 Jetson Nano) that also supports the use of the GPU.
- Prerequisites
- Procedure - 5 minutes, plus 85 minutes for the compilation in the last step
- Benchmark
- Compile llama.cpp for CPU mode - 24 minutes
- Install prerequisites
- Choosing the right compiler