I have an RX6800XT and I use KoboldCPP to run models I download off of Huggingface.

I’m not sure how many tokens per second it generates, probably about 10?

If you want to try it yourself here’s a link to the Github page: https://github.com/LostRuins/koboldcpp

I don’t know what your - and your kid’s - situation is, but I worry pushing Linux onto someone would be counterproductive to getting them to like it.

I only use it because I genuinely like and appreciate it. I’d probably start by getting him interested in it. If he likes it enough then he’ll try and learn more by himself.

I recently got an LLM running locally on an AMD GPU. This was only possible on Linux. Depending on your son, something like that could be a cool way to get him interested.

How do you mean teach?

Just getting them to use it or teaching them terminal commands?

If you’re willing to self host Nextcloud then that could be a solution.

Currently I use it to sync my calendar on my phone and an instance of MagicMirror2.

There’s a small amount of telemetry going on.

Also, Pop_OS makes running an Nvidia GPU less painful.

Idk, it seems to be picking up steam. It’s what I use unless I’m trying to use something super lightweight.

For me it has the stability of Ubuntu without having to use Ubuntu.

Haven’t tried Debian yet though.

Let’s forget Linux and praise Linus then I guess.