Skip Navigation

InitialsDiceBearhttps://github.com/dicebear/dicebearhttps://creativecommons.org/publicdomain/zero/1.0/„Initials” (https://github.com/dicebear/dicebear) by „DiceBear”, licensed under „CC0 1.0” (https://creativecommons.org/publicdomain/zero/1.0/)F

FrankLaskey

@ FrankLaskey @lemmy.ml

Posts
3
Comments
41
Joined
6 yr. ago

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • Yes, I do local host several models. Mostly the Qwen3 family stuff like 30b a3b etc. Have been trying GLM 4.5 a bit through OpenRouter and I’ve been liking the style pretty well. Interesting to know I could just pop in some larger RAM dimms potentially and run even larger models locally. The thing is OR is so cheap for many of these models and with zero data retention policies I feel a bit stupid for even buying a 24 GB VRAM GPU to begin with.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • Is this Grok Code fast 1? I’ve noticed it’s hitting tops on OR for programming as of recently. I was going to try it out but it won’t respect my zero data retention preference unsurprisingly.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • Honestly it has been good enough until recently when I’ve been struggling specifically with docker networking stuff and it’s been on the struggle bus with that. Yes, I’m using OpenRouter via OpenWebUI. I used to run a lot of stuff locally (mostly 4-b it quant 32b and smaller since I only have a single 3090) but lately I’ve been trying more larger models out on OpenRouter since many of the non proprietary ones are super cheap. Like fractions of a penny for a response.. Many are totally free to a point as well.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • I tried that! I literally told it to be concise and to limit its response to a certain number of words unless strictly necessary and it seemed to completely ignore both.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • I definitely have been looking out for this for a while. Wanting to replicate GPT deep research but not seeing a great way to do this. I did see that there was a OWUI tool for this but it didn’t seem particularly battle-tested so I hadn’t checked it out yet. I’ve been curious about how the new Tongyi Deep Research might be..

    That said, specifically for troubleshooting somewhat esoteric (or at least quite bespoke in terms of configuration) software problems, I was hoping the larger coder focused models would have enough built-in knowledge to suss out the issues. Maybe I should be having them consistently augment their responses with web searches if this isn’t the case? I have not been clicking that button typically.

    I do generally try to paste in or link as much of the documentation for whatever software I’m troubleshooting though.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • The coder model (480B). I initially mistakenly said the 235b one but edited that. I didn’t know you could customize quant on OpenRouter (and I thought the differences between most modern 4 bit quants and 8-bit was minimal as well..) I have tried GPT OSS 120 a bunch of time and though it seems quote unquote ‘intelligent’ enough it is just too talkative and verbose for me (plus I can’t remember the last time it responded without somehow working an elaborate comparison table into the response) and it makes it too hard to parse through things.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • Appreciate you sharing your experience. With this being the case and it being an order of magnitude more $$$ than Qwen3 coder, I think I’ll mostly steer clear for now. Not sure why this model seems to have such mindshare and dominance with programmers these days honestly. Other than many in the west seem somewhat biased against Chinese models.

  • Best LLMs for PC/Tech Troubleshooting?

    Jump

  • Even if this isn’t going to solve the issue of the quality of the LLM’s advice and help, it would massively simplify my current workflow which is copy/pasting logs and command responses and everything into the OWUI window. I’ll check it out. Can you use OpenRouter with VSCode to have access to more models or?

  • What local, small models are you all using?

    Jump

  • In my opinion, Qwen3-30B-A3B-2507 would be the best here. Thinking version likely best for most things as long as you don’t mind a slight penalty to speed for more accuracy. I use the quantized IQ4_XS models from Bartowski or Unsloth on HuggingFace.

    I’ve seen the new OSS-20B models from OpenAI ranked well in benchmarks but I have not liked the output at all. Typically seems lazy and not very comprehensive. And makes obvious errors.

    If you want even smaller and faster the Qwen3 Distill of DeepSeek R1 0528 8B is great for its size (esp if you’re trying to free up some VRAM to use larger context lengths)

  • wanderer v0.17.0 released — Federation support is here

    Jump

  • Interesting project. Is it actually possible to track workouts using your phone or smartwatch without needing proprietary third-party apps like Strava or Garmin Connect though?

  • Removed

    TIL on 2 May 2025 a vessel carrying international human rights activists and humanitarian aid destined for the Gaza Strip, was attacked by drones in international waters off the coast of Malta.

    Jump

  • Thanks! now I feel a bit lazy since there was a whole wiki article on their relations that I could have just looked up. Sorry it’s early in the morning here.. appreciate it

  • Removed

    TIL on 2 May 2025 a vessel carrying international human rights activists and humanitarian aid destined for the Gaza Strip, was attacked by drones in international waters off the coast of Malta.

    Jump

  • Upvoted for the info about Greek support for Israel. I didn’t know that was a thing. Any other context for why that could be?

  • OpenWebUI Release v0.6.0

    Jump

  • Looks like it now has Docling Content Extraction Support for RAG. Has anyone used Docling much?

  • Consumer GPUs to run LLMs

    Jump

  • Oh and I typically get 16-20 tok/s running a 32b model on Ollama using Open WebUI. Also I have experienced issues with 4-bit quantization for the K/V cache on some models myself so just FYI

  • Consumer GPUs to run LLMs

    Jump

  • It really depends on how you quantize the model and the K/V cache as well. This is a useful calculator. https://smcleod.net/vram-estimator/ I can comfortably fit most 32b models quantized to 4-bit (usually KVM or IQ4XS) on my 3090’s 24 GB of VRAM with a reasonable context size. If you’re going to be needing a much larger context window to input large documents etc then you’d need to go smaller with the model size (14b, 27b etc) or get a multi GPU set up or something with unified memory and a lot of ram (like the Mac Minis others are mentioning).

  • Perplexity open sources R1 1776, a version of the DeepSeek R1 model that CEO Aravind Srinivas says has been “post-trained to remove the China censorship”.

    Jump

  • I think we can all agree that modifications to these models which remove censorship and propaganda on behalf of one particular country or party is valuable for the sake of accuracy and impartiality, but reading some of the example responses for the new model I honestly find myself wondering if they haven’t gone a bit further than that by replacing some of the old non-responses and positive portrayals of China and the CPC with a highly critical perspective typified by western governments which are hostile to China (in particular the US). Even the name of the model certainly doesn’t make it sound like neutrality and accuracy is their primary aim here.

  • Linux's Sole Wireless/WiFi Driver Maintainer Is Stepping Down - Phoronix

    Jump

  • I used to daily drive Ubuntu some years ago for work/personal use but have been back on Win 10 primarily for the last 4-5 years. I was considering trying to go back due to how much Windows sucks (despite some proprietary software only being available on it) but remembering the trouble I had with some networking/printer drivers and troubleshooting those issues and then seeing this article Is definitely making me reconsider..

  • Deleted

    Permanently Deleted

    Jump
  • Deleted

    Permanently Deleted

    Jump

  • Is this what the Leonard Cohen song is about?