how to download local model?

[toinou180]

please

[Harald B]

The go-to place is HuggingFace[huggingface.co]. What model to pick is a never-ending debate and also depends on what you can run. For some popular basic models to start with, I'd suggest trying Stheno 3.2[huggingface.co] if you've got a 6GB or 8GB card or UnslopNemo 4.1[huggingface.co] if you've got a 12GB or 16GB card.
Start with a Q4 quant, possibly go up a bit depending on what speed you can tolerate and how much context size you want to set. Run GGUF files with KoboldCPP[github.com]. Don't forget to set the appropriate template in settings in-game (Llama3 for Stheno or Metharme for UnslopNemo).

[agrimes]

Here is a good catalog of LLMs: https://localai.io/gallery.html The game requires one that is a non-CoT (currently) model that is fairly close to an instruct model, all else is taste and what you can run...