Ha, you may be underestimating your survival instinct. Look at historical events like sieges. Resource shortage, dangerous environment, social collapse - all that can be expected in post-apocalyptic world. And people did survive. Eating whatever they can find or catch, leaving in cold, and filth, getting sick, witnessing death and other horrors... People survived.
Unless you get killed, or shoot yourself, you would wish you came prepared for that kind of times...
People are downvoting you, but I like your idea to draw analogy with heating, because it is something most of us rely on, and if LLMs and related technology will keep evolving as they do, probably most of us will rely on it more or less, sooner or later. Regardless of what AI haters would say.
But your wood fire/central heating analogy is bad. I would compare large LLM vendors to hot water heating utility common in Eastern Europe, and small LLMs to various heating devices.
Utility companies can set prices, and decide who gets connected to hot water pipe, and set water temperature. There are regulations that limit the power of such utility companies, allow customers to choose the supplier, etc. Same should happen with LLM providers - competition and anti-monopoly laws should protect customers who choose to use them.
Alternatively, customers may choose not to use utility-supplied heating. They can purchase space heaters, hand warmers, install split systems, burn wood - they are free to pick technology, power source, size, appearance of such devices. They can take responsibility of heating their homes, willing to invest their time and money in order to be independent of central heating utility.
Small LLMs are like that - people can run their own, with capabilities dependent on investment, or they can pay smaller providers or resellers to get more flexibility and/or privacy and avoid capital investments. They could spend time tuning small models and harnesses to do some simple tasks, and they wouldn't need to "buy intelligence" from OpenAI and others.
Slate looks interesting, but I wonder how that mid $20k price is possible with US based manufacturer... Also, not clear if they will collect data just like everyone else or not.
Yeah, I didn't consider Leaf because of these issues... Thanks for sharing about e-golf.
And I'm wary about quality of early Chevy and Ford EVs. Even newer models are crappy. I actually own a Ford Escape PHEV, and cannot charge it at all because of a recall after recall that come with a warning "not to charge due to the risk of fire" and take many months to be fixed.
Edit: corrected my car's model
Yeah... Too bad that all EVs are spyware on the wheels. When car makers will be forced to stop or significantly reduce the amount of data they are collecting, I may consider buying one. Probably not going to happen in my lifetime.
I started from zero with Forgejo, so Gitea is not a consideration. I just don't need latest and greatest as long as security is covered. But I finally noticed that v15 is the new LTS, so that's what I hope to run during the next couple of years.
Well, before your response, v11 was the latest LTS release, received security patches and did all I wanted it to do. Now, I actually did look at the new release and found out that 15 is a new LTS. So, thank you! I guess it's time to review the breaking changes and get prepared.
Wow, I didn't think you were running 176GB worth of GPUs on a consumer board! I don't have an extra board, and my gaming PC that has 9070XT is not a good basis for multi GPU build - it has a cheap mATX motherboard with too few slots and lanes.
So it's going to be a new build. Used EPYC boards look interesting for that.
Thanks! I was running some models on my RX 9070XT, but only Ollama works flawlessly. I couldn't make llama.Cpp to run Gemma4 or the newer Qwen - maybe I'm hitting that incompatibility, but probably it's the skill issue.
P40 doesn't look very appealing. 32 GB V100 costs about the same as 2xP40, less VRAM in total, but it's faster, will use less power.
But I'm not sure if I follow you on the PCIe... If I run a model that spans multiple GPUs, doesn't PCIe bandwidth matter?
Interesting. What kind of network do you have between the two machines?