Mastodon Feed: Post

lloydmeta ("Lloyd") wrote:

Been running #LLaMA2 locally using #llm-rs.

* Completely crushes my 2018 Intel Mac wrt CPU usage...600% easily when it's doing stuff. And this is with a quantised model.
* Already caught it in a lie 😂

Neither are surprising (my Mac is from 2018 2.2 GHz 6-Core Intel Core i7) but it actually made me fact check whether EPFL and/or Lightened were acquired by IBM in 2022 (spoiler: they haven't)!

Attachments:

• Terminal windows with 2 panes. The one on the left is running an llm-rs repl chat against an LLaMA2 model. The one on the left is running htop and showing ~600% cpu usage.