Skip Navigation

troed

@ troed @fedia.io

Posts
8
Comments
328
Joined
3 yr. ago

HW/FW security researcher & Demoscene elder.

I started having arguments online back on Fidonet and Usenet. I'm too tired to care now.

  • Qwen3.8-27B Weights Released

    Jump

  • Got an answer - there were indeed issue with some quants

    Actually edit - UD-Q4_K_XL and all UD-* has our corrected chat teaplate

    We need to update the non UD-* ones with our chat template - stay tuned

  • Qwen3.8-27B Weights Released

    Jump

  • Having issues with it (Unsloth's gguf).

     
        
0.27.136.404 I srv  proxy_reques: proxying request to model unsloth/Qwen3.8-27B-GGUF:Q4_K_M on port 55741
[55741] 0.14.424.772 W srv    operator(): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 106, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}

  • Field note: measuring your context budget before a run, not after

    Jump

  • When I start llama-server I point it to the models-config that have unique max context sizes per model - and they're allocated at their max size as soon as the server starts so since it comes up I will be able to use that context size too.

    I'm actually a bit unsure as to how you run it since you get OOMs during usage :)

    I also use the DCP plugin for Opencode to help manage the context cache and have less of a disruption as it gets compressed, but I wouldn't need to for the above to work. When I hit the context limit the context would still get compressed.

  • llama rpc exists and works really well

    Jump

  • Regular 1Gbit/s with three switches in total between them - distance about 40m (different buildings).

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    llama rpc exists and works really well

    github.com /ggml-org/llama.cpp/blob/master/tools/rpc/README.md

  • Stop Using OpenCode

    Jump

  • Yeah it was quite obvious from all the "clanker" that this is a typical run-of-the-mill AI-hater, but I found this funny anyway:

    The weight count is too low to reproduce the training set verbatim

    That's not what anyone wants. That's not how LLMs gain "intelligence", at all. The whole point of training on large datasets is to NOT internalize training data verbatim.

  • Qwen 27B Q4 at usable speed on 16GB VRAM

    Jump

  • The trick in itself (the FFN tensors selectively being off GPU) shouldn't be.

  • Qwen 27B Q4 at usable speed on 16GB VRAM

    Jump

  • Not sure I understand but I'm on Linux fwiw.

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    Qwen 27B Q4 at usable speed on 16GB VRAM

    www.reddit.com /r/LocalLLM/comments/1uxg871/qwen_36_27b_q4_96k_context_running_on_16gb_vram/
  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    My llama-server suddenly started error 400 on the chat template - this fixed it

    huggingface.co /froggeric/Qwen-Fixed-Chat-Templates

  • InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation

    Jump

  • Our framework outpaces orbital velocity by 9 times on a consumer GPU

    I'm sure that nomenclature is obvious with their intended audience but am I confused right now

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    GPU bifurcation - options

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    Opencode llama-server prefill/generation stats plugin

    codeberg.org /troed/oc-ls-stats
  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    North Mini Code v1.0 - a Qwen 3.6 35B MoE alternative

    huggingface.co /unsloth/North-Mini-Code-1.0-GGUF

  • Don't skimp on the quant when using MoE

    Jump

  • 15t/s is workable IMHO. What's your system specs? I have 96GB DDR5 but never thought about going to an ever higher MoE.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • Of course it's not necessary. I'm a way-beyond-senior dev who laughed at LLMs up until a few months ago when trusted friends, whose competence is not in question, told me they got good usage out of them.

    I decided to challenge my convictions and sat down and took the time to learn how to use LLM assistants (I tried everyting from full vibe coding to manual gatekeeping of suggestions).

    Now I use them for my own personal projects, and I'm much more productive (for various reasons - but one is that the initial friction of oh yet another thing I have to learn just to do X is much lower. I have no boss telling me what to do, and I select my projects myself. If they didn't bring any benefits I wouldn't use them.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • That's my personal view as well and so all apps I write are "licensed" as CC0. Other maintainers are taking a different view.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • Having a policy against bad apps is fine - but that's not what this policy says.

    It's trivial to write a good app using LLM aids - but that new app won't have such a history.

  • Flathub Now Rejects AI-Assisted Apps and Submissions

    Jump

  • That's ... incredibly stupid. It will also make Flathub completely irrelevant shortly since app developers won't change what they're doing just because Flathub throws a hissy fit.

    /had an LLM-assisted PR merged into an open source project yesterday

  • LocalLLaMA @sh.itjust.works
    troed @fedia.io

    Don't skimp on the quant when using MoE

    unsloth.ai /docs/models/qwen3.6
  • Deleted

    Permanently Deleted

    Jump

  • It's similar to being an assembler coder when higher level languages with compilers came. No need for management purging, you'll simply be competing for a smaller segment of assigments.

    I don't know of a single developer that has actually used LLM aids say there's no benefit to them. Those that refuse do so for some other convictions and don't really know the difference between LLM aiding in tasks and full on yolo vibe coding.

  • What is your policy on letting a child use the internet?

    Jump

  • When they figure it out and become capable of reading and writing. Tablets, phones and computers are not locked down. Parental guidening and open communication means they know what it is, that there's good and there's bad content and people etc.

    Working great.

    /Swedish

  • Kamala Harris surges in new 2028 poll

    Jump

  • Haven't we decided AOC/Stewart is the strongest ticket?

    /European

  • Which open models are actually good at agentic coding?

    Jump

  • I run a quant of Qwen 35B A3B (Qwen3.6-35B-A3B-GGUF:UD_Q4_K_XL) at the moment, using Opencode and llama.cpp. I'm getting useful work out of it - but it's of course not Claude. My hardware is a 5060Ti with 16GB VRAM and then ~20GB or so of system mem is getting used as well.

    It's important to put boundaries on less capable models though, so I have two plugins in Opencode as well that really makes a big difference to the results: @tarquinen/opencode-dcp@latest and superpowers@git+https://github.com/obra/superpowers.git.

    I want to work in small steps with good control over what the models do so it's not very similar to what you describe with just having them run away for half an hour and do everything.

  • does anyone have exercises to learn how to program?

    Jump

  • Best I can do is just point to source code written by an old colleague and friend of mine from back in the day: https://sourceforge.net/p/qbubblemp/code/HEAD/tree/

    I wrote system apps and UI frameworks and it's 25 years ago so I don't want to pretend to remember too much :D

  • does anyone have exercises to learn how to program?

    Jump

  • ex-Symbian employee/developer here:

    Symbian/Epoc C++ is vastly different from modern C++. Enjoy yourself, as long as you are ok with learning a "dead" eco system where a lot of the things you learn aren't useful on other more modern platforms.

  • Proton @lemmy.world
    troed @fedia.io

    Are you still here Proton?