Thanks. That's very helpful.
- Posts
- 13
- Comments
- 254
- Joined
- 3 yr. ago
- Posts
- 13
- Comments
- 254
- Joined
- 3 yr. ago
United Kingdom @feddit.uk Burnham to unveil £8.4bn nuclear submarine investment in major boost for UK defence
cats @lemmy.world Cuddles, the family cat
Technology @lemmy.world Comparing x265 to AV1 with Grain Synthesis
cats @lemmy.world The largest of yawns
Progressive Politics @lemmy.world Reform local media ban is a glimpse of what may happen with Farage in power
cats @lemmy.world Picture of Poppy, my brother's cat ❤️
Technology @lemmy.world GlobalX, Airline for Trump’s Deportations, Hacked
Actually Infuriating @lemmy.world The UK government is planning massive cuts to disability welfare.
Technology @lemmy.world Why are Google's Assistant(s) so bad nowadays?
Technology @lemmy.world Photosphere and all cameras now work on Pixel 9 devices via AGC9.2.14 V14
Technology @lemmy.world I benchmarked 6 different metal USB sticks
memes @lemmy.world Caption this
Technology @lemmy.world Handbrake custom LapSharp values


Having run models locally, RAM use seems to be almost directly proportional to number of parameters. 8 Billion parameters requires approx 8GB of VRAM at 1/4 precision.
Therefore, if this pattern holds you somehow need 10 Terabytes of VRAM at 4K and 40 Terabytes at full precision.
I think I saw some estimates that Claude's Opus models may be and Opus model equivalents may be at around 100B parameters (100-400GB VRAM).
TLDR its clear why RAM is so expensive.