Skip Navigation

troed

@ troed @fedia.io

Posts
8
Comments
325
Joined
3 yr. ago

HW/FW security researcher & Demoscene elder.

I started having arguments online back on Fidonet and Usenet. I'm too tired to care now.

  • Field note: measuring your context budget before a run, not after

    Jump

  • When I start llama-server I point it to the models-config that have unique max context sizes per model - and they're allocated at their max size as soon as the server starts so since it comes up I will be able to use that context size too.

    I'm actually a bit unsure as to how you run it since you get OOMs during usage :)

    I also use the DCP plugin for Opencode to help manage the context cache and have less of a disruption as it gets compressed, but I wouldn't need to for the above to work. When I hit the context limit the context would still get compressed.

  • llama rpc exists and works really well

    Jump

  • Regular 1Gbit/s with three switches in total between them - distance about 40m (different buildings).

  • Stop Using OpenCode

    Jump

  • Yeah it was quite obvious from all the "clanker" that this is a typical run-of-the-mill AI-hater, but I found this funny anyway:

    The weight count is too low to reproduce the training set verbatim

    That's not what anyone wants. That's not how LLMs gain "intelligence", at all. The whole point of training on large datasets is to NOT internalize training data verbatim.

  • Qwen 27B Q4 at usable speed on 16GB VRAM

    Jump

  • The trick in itself (the FFN tensors selectively being off GPU) shouldn't be.

  • Qwen 27B Q4 at usable speed on 16GB VRAM

    Jump

  • Not sure I understand but I'm on Linux fwiw.

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    Qwen 27B Q4 at usable speed on 16GB VRAM

    www.reddit.com /r/LocalLLM/comments/1uxg871/qwen_36_27b_q4_96k_context_running_on_16gb_vram/
  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    My llama-server suddenly started error 400 on the chat template - this fixed it

    huggingface.co /froggeric/Qwen-Fixed-Chat-Templates

  • InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation

    Jump

  • Our framework outpaces orbital velocity by 9 times on a consumer GPU

    I'm sure that nomenclature is obvious with their intended audience but am I confused right now

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    GPU bifurcation - options

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    Opencode llama-server prefill/generation stats plugin

    codeberg.org /troed/oc-ls-stats
  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    North Mini Code v1.0 - a Qwen 3.6 35B MoE alternative

    huggingface.co /unsloth/North-Mini-Code-1.0-GGUF

  • Don't skimp on the quant when using MoE

    Jump

  • 15t/s is workable IMHO. What's your system specs? I have 96GB DDR5 but never thought about going to an ever higher MoE.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • Of course it's not necessary. I'm a way-beyond-senior dev who laughed at LLMs up until a few months ago when trusted friends, whose competence is not in question, told me they got good usage out of them.

    I decided to challenge my convictions and sat down and took the time to learn how to use LLM assistants (I tried everyting from full vibe coding to manual gatekeeping of suggestions).

    Now I use them for my own personal projects, and I'm much more productive (for various reasons - but one is that the initial friction of oh yet another thing I have to learn just to do X is much lower. I have no boss telling me what to do, and I select my projects myself. If they didn't bring any benefits I wouldn't use them.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • That's my personal view as well and so all apps I write are "licensed" as CC0. Other maintainers are taking a different view.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • Having a policy against bad apps is fine - but that's not what this policy says.

    It's trivial to write a good app using LLM aids - but that new app won't have such a history.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • That's ... incredibly stupid. It will also make Flathub completely irrelevant shortly since app developers won't change what they're doing just because Flathub throws a hissy fit.

    /had an LLM-assisted PR merged into an open source project yesterday

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    Don't skimp on the quant when using MoE

    unsloth.ai /docs/models/qwen3.6
  • Deleted

    Permanently Deleted

    Jump

  • It's similar to being an assembler coder when higher level languages with compilers came. No need for management purging, you'll simply be competing for a smaller segment of assigments.

    I don't know of a single developer that has actually used LLM aids say there's no benefit to them. Those that refuse do so for some other convictions and don't really know the difference between LLM aiding in tasks and full on yolo vibe coding.

  • What is your policy on letting a child use the internet?

    Jump

  • When they figure it out and become capable of reading and writing. Tablets, phones and computers are not locked down. Parental guidening and open communication means they know what it is, that there's good and there's bad content and people etc.

    Working great.

    /Swedish

  • Kamala Harris surges in new 2028 poll

    Jump

  • Haven't we decided AOC/Stewart is the strongest ticket?

    /European

  • Which open models are actually good at agentic coding?

    Jump

  • I run a quant of Qwen 35B A3B (Qwen3.6-35B-A3B-GGUF:UD_Q4_K_XL) at the moment, using Opencode and llama.cpp. I'm getting useful work out of it - but it's of course not Claude. My hardware is a 5060Ti with 16GB VRAM and then ~20GB or so of system mem is getting used as well.

    It's important to put boundaries on less capable models though, so I have two plugins in Opencode as well that really makes a big difference to the results: @tarquinen/opencode-dcp@latest and superpowers@git+https://github.com/obra/superpowers.git.

    I want to work in small steps with good control over what the models do so it's not very similar to what you describe with just having them run away for half an hour and do everything.

  • does anyone have exercises to learn how to program?

    Jump

  • Best I can do is just point to source code written by an old colleague and friend of mine from back in the day: https://sourceforge.net/p/qbubblemp/code/HEAD/tree/

    I wrote system apps and UI frameworks and it's 25 years ago so I don't want to pretend to remember too much :D

  • does anyone have exercises to learn how to program?

    Jump

  • ex-Symbian employee/developer here:

    Symbian/Epoc C++ is vastly different from modern C++. Enjoy yourself, as long as you are ok with learning a "dead" eco system where a lot of the things you learn aren't useful on other more modern platforms.

  • NSFW

    What are some of the interesting ways you use your AI interface?

    Jump

  • I have since many years back documented most tasks in a digital note taker app. Over the years I've transferred them between systems, right now they're in Nextcloud.

    I also have configured a local AI on the server and since Nextcloud has built in support for using local AI it has now indexed all my notes so I can pose queries to a local AI that has all that info I've ever written down backing it.

    "When is it time to re-apply for the garage building permit" and "Which one of our kids was the tallest at birth" are examples of questions I can now do.

  • Those who have ever taken medications for depression or anxiety, how long before it started working from your experience?

    Jump

  • Depression: SSRIs, within a few days Anxiety: Promethazine, within a few days

  • Pool spare GPU capacity to run LLMs at larger scale

    Jump

  • That's really interesting. Only macOS instructions though? Seems like something that would easily run on Linux as well.

    (I'd love to hook my server's GPU into local LLM workloads otherwise only offloaded to the CPU from my main workstation when needing too much VRAM)

  • Proton @lemmy.world
    troed @fedia.io

    Are you still here Proton?