I distrohopped so much after each previous distro eventually broke and me clearly not being smart enough to recover. I'm honestly kinda sick of it, even if the immutable nature also annoys the shit out of me.
My GPU is a 6650 XT, which should in principle work with ROCm.
Which model specifically are you recommending? Llama-3.1-8B-Lexi-Uncensored-V2-GGUF? Because the original meta-llama ones are censored to all hell and Huggingface is not particularly easy to navigate, on top of figuring out the right model size & quantization being extremely confusing.
I just can't get ROCm / gpu generation to work on Bazzite, like at all. It seems completely cursed.
I tried koboldcpp through a Fedora distrobox and it didn't even show any hardware options. Tried through an Arch AUR package through distrobox and the ROCm option is there but ends with a CUDA error. lol
The Vulkan option works but seems to still use the CPU more than the GPU and is consequently still kinda slow and I struggle to find a good model for my 8GB card. Fimbulvetr-10.7B-v1-Q5_K_M for example was still too slow to be practical.
Tried LM Studio directly in Bazzite and it also just uses the CPU. It also is very obtuse on how to connect to it with SillyTavern, as it asks for an API key? I managed it once in the past but I can't remember how but it also ended up stopping generating anything after a few replies.
Krita's diffusion also only runs on the CPU, which is abysmally slow, but I'm not sure if they expect Krita to be build directly on the system for ROCm support to work.
I'm not even trying to get SDXL or something to run at this point, since that seems to be still complicated enough even on a regular distro.
I tried LM Studio (directly in Bazzite) and it has the same issue of not running through the GPU. It also always seem to end up stopping generating anything after a few moments when I use it with SillyTavern.
Tried the koboldcpp AUR package through Distrobox, and when I select the ROCm option it crashes with a CUDA error. lol
Using the Vulkan option it still seems to run through the CPU for some reason.
It's not so much a matter of the region itself, I live in a pretty decent area of my city. It's more that the houses here are all apartment buildings so in most cases the packages will get placed in the hallways instead of the front door of the building. There's just not really a need to ever have them drop it off there.
I agree it is an idiotic design for a cab, but I'd argue the Cybertruck is a way dumber design even by visuals alone. That thing would at least make for a half decent sports car (again, visually).
In Germany you have to give permission in the tracking menu to have them drop the package off somewhere else. I live in an apartment building so I usually tell them to drop it in the hallway of the building, since there's usually someone at home to at least open the door downstairs if I'm not at home. I've seen packages outside the front door only once here and it is not a place I'd ever choose.
The story is an overblown nothingburger and would make Firefox still the massively better choice. This is just agenda pushing at this point. But hey, keep using Chrome if you think they preserve your privacy and the web in a better way. No one is forcing you to use a Firefox based browser.
Sadly some webpages are made only thinking in chromium, thus working worse in other browsers.
YT absolutely works in Firefox. I use it daily for many many years. The only thing that annoys me is that the fullscreen toggle is kinda sluggish but that's not anything I'd use Chrome for.
Well, anything remotely raunchy gets a "I cannot participate in explicit content" default reply.
I am using the rocm install of koboldcpp but as said, the ROCm option errors out with a CUDA error for some reason.