Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)M

marauding_gibberish142

@ marauding_gibberish142 @lemmy.dbzer0.com

Posts
9
Comments
164
Joined
1 yr. ago

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Thank you, and that highlights the problem - I don't see any affordable options (around $200 or so for a motherboard + CPU combo) for a lot of PCIe lanes other than purchasing Frankenstein boards from Aliexpress. Which isn't going to be a thing for much longer with tariffs, so I'm looking elsewhere

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Thank you, but which consumer motherboard + CPU combo is giving me 32 lanes of PCIe Gen 4 neatly divided into 2 x16 slots for me to put 2 GPUs in? I only asked this question because I was going to buy used computers and stuff a GPU in each.

    Your point about networking is valid, and I'll be hesitant to invest in 25Gbe right now

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Thanks but I'm not going to run supercomputers. I just want to run 4 GPUs separately because of inadequate PCIe lanes in a single computer to run 24B-30B models

  • How to use GPUs over multiple computers for local AI?

    Jump

  • I'm not going to do anything enterprise. I'm not sure how people seem to think of it this way when I didn't even mention it.

    I plan to use 4 GPUs with 16-24GB VRAM each to run smaller 24B models.

  • How to use GPUs over multiple computers for local AI?

    Jump

  • I don't understand your point, but I was going to use 4 GPUs (something like used 3090s when they get cheaper or the Arc B580s) to run the smaller models like Mistral small.

  • How to use GPUs over multiple computers for local AI?

    Jump

  • It is because modern consumer GPUs do not have enough VRAM to load the 24B models. I want to run Mistral small locally.

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Thank you, I'll take a look

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Thank you for the links. I will go through them

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Aren't Epyc boards really expensive? I was going to buy 3-4 used computers and stuff a GPU in each.

    Are there motherboards on the used market that can run the E5-2600 V4 series CPUs and have multiple PCIe Xi slots? The only ones I found were super expensive/esoteric.

  • How to use GPUs over multiple computers for local AI?

    Jump

  • Never heard of it. What is it about?

  • Best ‘simple’ budgeting app

    Jump

  • I just use a spreadsheet with LibreOffice, and with their viewer on Android, I can just sync the spreadsheet and viola.

  • How and where should I keep backups of system configurations?

    Jump

  • I don't like big corporations so I wouldn't do that, but sure

  • Trump cuts funding to FOSS projects.

    Jump

  • The problem is that no other country has done anything of the sort. I'm sure the Swiss, the Dutch, the French, the other german-speaking countries and especially the Nordics have enough money to give a few million here and there to FOSS. But they didn't, because their leaders are just as corrupt and vile. Everybody is acting as if America is the ultimate villain: the fact of the matter is, give any of those European leaders power over America and sooner or later they would turn out exactly the same. It is in their nature. And I say this having lived in Europe for a while.

    The German fund staying is pure luck since the hawks at EU are stupid. They'll come after it once they're done with Chat control.

  • Trump cuts funding to FOSS projects.

    Jump

  • If only there was a decentralised tracker which would track project funding and give us metrics like which project is dangerously close to shutting down

  • Trump cuts funding to FOSS projects.

    Jump

  • Man, ReThink is such a lifesaver. It's so good people at Graphene recommend it. I might give Qubes some bucks too, they are awesome

  • How and where should I keep backups of system configurations?

    Jump

  • I suggest SOPS

  • Trump cuts funding to FOSS projects.

    Jump

  • With the new leadership I am doubtful on how long that will last. One of the better things that came of Scholz's era

  • Trump cuts funding to FOSS projects.

    Jump

  • Which other projects caught your attention? I was going to donate to Graphene, EFF and some TOR operators

  • How and where should I keep backups of system configurations?

    Jump