litchralee

Very interesting! Im no longer pursuing Meshtastic -- I'm changing over my hardware to run MeshCore now -- but this is quite a neat thing you've done here.

As an aside, if you later want to have full networking connectivity (Layer 2) using the same style of encoding the data as messages, PPP is what could do that. If transported over Meshtastic, PPP could give you a standard IP network, and on top of that, you could use SSH to securely access your remote machine.

It would probably be very slow, but PPP was also used for dial-up so it's very accommodating. The limiting factor would be whether the Meshtastic local mesh would be jammed up from so many messages.

This answer is going to go in multiple directions.

If you're looking for practice on using C to implement ways to talk to devices and peripherals, the other commenter's suggested to start with an SBC (eg Raspberry Pi, Orange Pi) or with a microcontroller dev kit (eg Arduino, MSP430, STM32) is spot-on. That gives you a bunch of attached peripherals, the datasheet that documents the register behavior, and so you can then write your own C functions that fill in and read those registers. In actual projects, you would probably use the provided libraries that already do this, but there is educational value in trying it yourself.

However, just because you write a C function named "put_char_uart0()", that isn't enough to prepare for writing full-fledged drivers, such as those in the Linux and FreeBSD kernel. This next step is more about software design, where you structure your C code so that rather than being very hardware-specific (eg for the exact UART peripheral in your microcontroller) you have code which works for a more generic UART (which abstracts general details) but is common-code to all the UARTs made by the same manufacturer. This is about creating reusable code, about creating abstraction layers, and about writing extensible code. Not all code can be reusable, not every abstraction layer is desirable, and you don't necessarily want to make your code super extensive if it starts to impact your core requirements. Good driver design means you don't ever paint yourself into a corner, and the best way to learn how to avoid this is through sheer experience.

For when you do want to write a full-and-proper driver for any particular peripheral -- maybe one day you'll create one such device, such as by using an FPGA attached via PCIe to a desktop computer -- then you'll need to work within an existing driver framework. Linux and FreeBSD drivers use a framework so that all drivers have access to what they need (system memory, I/O, helper functions, threads, etc), and then it's up to the driver author to implement the specific behavior (known in software engineering as "business logic"). It is a learned skill -- also through experience -- to work within the Linux or FreeBSD kernels. So much so that both kernels have gone through great lengths to enable userspace drivers, meaning the business logic runs as a normal program on the computer, saving the developer from having to learn the strange ways of kernel development.

And it's not like user space drivers are "cheating" in any way: they're simply another framework to write a device driver, and it's incumbent on the software engineer to learn when a kernel or user space driver is more appropriate for a given situation. I have seen kernel drivers used for sheer computational performance, but have also seen userspace drivers that were developed because nobody on that team was comfortable with kernel debugging. Those are entirely valid reasons, and software engineering is very much about selecting the right tool from a large toolbox.

Sadly, I'm not familiar enough with Nginx Proxy Manager to know. But I would imagine that there must be a different way to achieve the same result.

BTW, when I read "NPM", I first think of Node.JS Package Manager. The title of your post may be confusing, and you might consider editing it to spell out the name of Nginx Proxy Manager.

I'll take a stab at the question. But I'll need to lay some foundational background information.

When an adversarial network is blocking connections to the Signal servers, the Signal app will not function. Outbound messages will still be encrypted, but they can't be delivered to their intended destination. The remedy is to use a proxy, which is a server that isn't blocked by the adversarial network and which will act as a relay, forwarding all packets to the Signal servers. The proxy cannot decrypt any of the messages, and a malicious proxy is no worse than blocking access to the Signal servers directly. A Signal proxy specifically forwards only to/from the Signal servers; this is not an open proxy.

The Signal TLS Proxy repo contains a Docker Compose file, which will launch Nginx as a reverse proxy. When a Signal app connects to the proxy at port 80 or 443, the proxy will -- in the background -- open a connection to the Signal servers. That's basically all it does. They ostensibly wrote the proxy as a Docker Compose file, because that's fairly easy to set up for most people.

But now, in your situation, you already have a reverse proxy for your selfhosting stack. While you could run Signal's reverse proxy in the background and then have your main reverse proxy forward to that one, it would make more sense to configure your main reverse proxy to directly do what the Signal reverse proxy would do.

That is, when your main proxy sees one of the dozen subdomains for the Signal server, it should perform reverse proxying to those subdomains. Normally, for the rest of your self hosting arrangement, the reverse proxy would target some container that is running on your LAN. But in this specific case, the target is actually out on the public Internet. So the original connection comes in from the Internet, and the target is somewhere out there too. Your reverse proxy simply is a relay station.

There is nothing particularly special about Signal choosing to use Nginx in reverse proxy mode, in that repo. But it happens to be that you are already using Nginx Proxy Manager. So it's reasonable to try porting Signal's configuration file so that it runs natively with your Nginx Proxy Manager.

What happens if Signal updates that repo to include a new subdomain? Well, you wouldn't receive that update unless you specifically check for it. And then update your proxy configuration. So that's one downside.

But seeing as the Signal app demands port 80 and 443, and you already use those ports for your reverse proxy, there is no way to avoid programming your reverse proxy to know the dozen subdomains. Your main reverse proxy cannot send the packets to the Signal reverse proxy if your main proxy cannot even identify that traffic.

Since that whole vibe-coded Cloudflare Matrix nonsense and associated attempted retcon -- see here for context -- I am looking forward to a talk on how Matrix actually works.

Specifically, I'd like to know what aspects of a secure, decentralized message platform are particularly hard. That's in the context of whether Matrix can ever grow into a bona fide Signal competitor (nb: Signal remains the gold standard), and also whether Matrix would function well as a Discord replacement, even if it doesn't have as strong of group chat privacy and encryption protections.

There can be, although some parts may still need to be written in assembly (which is imperative, because that's ultimately what most CPUs do), for parts like a kernel's context switching logic. But C has similar restrictions, like how it is impossible to start a C function without initializing the stack. Exception: some CPUs (eg Cortex M) have a specialized mechanism to initialize the stack.

As for why C, it's a low-level language that maps well to most CPU's native assembly language. If instead we had stack-based CPUs -- eg Lisp Machines or a real Java Machine -- then we'd probably be using other languages to write an OS for those systems.

The other commenters correctly opined that encryption at rest should mean you could avoid encryption in memory.

But I wanted to expand on this:

I really don't see a way around this, to make the string searchable the hashing needs to be predictable.

I mean, there are probabilistic data structures, where something like a Bloom filter will produce one of two answers: definitely in the set, or possibly in the set. In the context of search tokens, if you had a Bloom filter, you could quickly assess if a message does not contain a search keyword, or if it might contain the keyword.

A suitably sized Bloom filter -- possibly different lengths based on the associated message size -- would provide search coverage for that message, at least until you have to actually access and decrypt the message to fully search it. But it's certainly a valid technique to get a quick, cursory result.

Though I think perhaps just having the messages in memory unencrypted would be easier, so long as that's not part of the attack space.

Upvoting because the FAQ genuinely is worthwhile to read, and answers the question I had in mind:

7.9 Why not just use a subset of HTTP and HTML?

I don't agree with their answer though, since if the rough, overall Gemini experience:

is roughly equivalent to HTTP where the only request method is "GET", the only request header is "Host" and the only response header is "Content-type", plus HTML where the only tags are
<p>
,
<pre>
,
<a>
,
<h1>
through
<h3>
,
<ul>
and
<li>
and
<blockquote>

Then it stands to reason -- per https://xkcd.com/927/ -- to do exactly that, rather than devise new protocol, client, and server software. Instead, some of their points have few or no legs to stand on.

The problem is that deciding upon a strictly limited subset of HTTP and HTML, slapping a label on it and calling it a day would do almost nothing to create a clearly demarcated space where people can go to consume only that kind of content in only that kind of way.

Initially, my reply was going to make a comparison to the impossibility of judging a book by its cover, since that's what users already do when faced with visiting a sketchy looking URL. But I actually think their assertion is a strawman, because no one has suggested that we should immediately stop right after such a protocol has been decided. Very clearly, the Gemini project also has client software, to go with their protocol.

But the challenge of identifying a space is, quite frankly, still a problem with no general solution. Yes, sure, here on the Fediverse, we also have the ActivityPub protocol which necessarily constrains what interactions can exist, in the same way that ATProto also constrains what can exist. But even the most set-in-stone protocol (eg DICT) can be used in new and interesting ways, so I find it deeply flawed that they believe they have categorically enumerated all possible ways to use the Gemini protocol. The implication is that users will never be surprised in future about what the protocol enables, and that just sounds ahistoric.

It's very tedious to verify that a website claiming to use only the subset actually does, as many of the features we want to avoid are invisible (but not harmless!) to the user.

I'm failing how to see how this pans out, because seeing as the web is predominantly client-side (barring server side tracking of IP address, etc), it should be fairly obvious when a non-subset website is doing something that the subset protocol does not allow. Even if it's a lay-in-wait function, why would a subset-compliant client software honor that?

When it becomes obvious that a website is not compliant with the subset, a well-behaved client should stop interacting with the website, because it has violated the protocol and cannot be trusted going forward. Add it to an internal list of do-not-connect and inform the user.

It's difficult or even impossible to deactivate support for all the unwanted features in mainstream browsers, so if somebody breaks the rules you'll pay the consequences.

And yet, Firefox forks are spawning left and right due to Mozilla's AI ambitions.

Ok, that's a bit blithe, but I do recognize that the web engines within browsers are now incredibly complex. Even still though, the idea that we cannot extricate the unneeded sections of a rendering engine and leave behind the functionality needed to display a subset of HTML via HTTP, I just can't accept that until someone shows why that is the case.

Complexity begats complexity, whereas this would be an exercise in removing complexity. It should be easier than writing new code for a new protocol.

Writing a dumbed down web browser which gracefully ignores all the unwanted features is much harder than writing a Gemini client from scratch.

Once again, don't do that! If a subset browser finds even one violation of the subset protocol, it should halt. That server is being malicious. Why would any client try to continue?

The error handling of a privacy-respecting protocol that is a subset of HTML and HTTP would -- in almost all cases -- assume the server is malicious, and to disconnect. It is a betrayal of the highest order. There is no such thing as a "graceful" betrayal, so we don't try to handle that situation.

Even if you did it, you'd have a very difficult time discovering the minuscule fraction of websites it could render.

Is this about using the subset browser to look at regular port-80 web servers? Or is this about content discovery? Only the latter has a semblance of logic behind it, but that too is an unsolved problem to this day.

Famously, YouTube and Spotify are drivers of content discovery, based in part due to algorithms that optimize for keeping users on those platforms. Whereas the Fediverse eschews centralized algorithms and instead just doesn't have one. And in spite of that, people find communities. They find people, hashtags, images, and media. Is it probably slower than if an algorithm could find these for the user's convenience? Yes, very likely.

But that's the rub: no one knows what they don't know. They cannot discover what they don't even imagine could exist. That remains the case, whether the Gemini protocol is there or not. So I'm still not seeing why this is a disadvantage against an HTTP/HTML subset.

Alternative, simple-by-design protocols like Gopher and Gemini create alternative, simple-by-design spaces with obvious boundaries and hard restrictions.

ActivityPub does the same, but is constructed atop HTTP, while being extensible to like-for-like replace any existing social media platform that exists today -- and some we haven't even thought of yet -- while also creating hard and obvious boundaries which forment a unique community unlike any other social media platform.

The assertion that only simple protocols can foster community spaces is belied by ActivityPub's success; ActivityPub is not exactly a simple protocol either. And this does not address why stripping down HTML/HTTP wouldn't also do the same.

You can do all this with a client you wrote yourself, so you know you can trust it.

I sure as heck do not trust the TFTP client I wrote at uni, and that didn't even have an encryption layer. The idea that every user will write their own encryption layer to implement the mandatory encryption for Gemini protocol is farcical.

It's a very different, much more liberating and much more empowering experience than trying to carve out a tiny, invisible sub-sub-sub-sub-space of the web.

So too would browsing a sunset of HTML/HTTP using a browser that only implements that subset. We know this because if your reading this right now, you're either viewing this comment through a web browser frontend for Lemmy, or using an ActivityPub client of some description. And it is liberating! Here we all are, on this sub sub sub sub space of the Internet, hanging out and commenting about protocols and design.

But that doesn't mean we can't adapt already-proven, well-defined protocols into a subset that matches an earlier vision of the internet, while achieving the same.

The full-blown solution would be to have your own recursive DNS server on your local network, and to block or redirect any other DNS server to your own, and possibly blocking all know DoH servers.

This would solve the DNS leakage issue, since your recursive server would learn the authoritative NS for your domain, and so would contact that NS directly when processing any queries for any of your subdomains. This cuts out the possibility of any espionage by your ISP/Google/Quad9's DNS servers, because they're now uninvolved. That said, your ISP could still spy in the raw traffic to the authoritative NS, but from your experiment, they don't seem to be doing that.

Is a recursive DNS server at home a tad extreme? I used to think so, but we now have people running Pi-hole and similar software, which can run in recursive mode (being built atop Unbound, the DNS server software).

"It was DNS" typically means that name resolution failed or did not propagate per its specification. Whereas I'm of the opinion that if DNS is working as expected, then it's hard to pin the blame on DNS. For example, forgetting to renew a domain is not a DNS problem. And setting a bad TTL or a bad record is not a DNS problem (but may be a problem with your DNS software). And so too do I think that DNS leakage is not a DNS problem, because the protocol itself is functioning as documented.

It's just that the operators of the upstream servers see dollar-signs by selling their user's data. Not DNS, but rather a capitalism problem, IMO.

/</minor nitpick>

I loaded True Nas onto the internal SSD and swapped out the HDD drive that came with it for a 10tb drive.

Do I understand that you currently have a SATA SSD and a 10TB SATA HDD plugged into this machine?

If so, it seems like a SATA power splitter that divides the power to the SSD would suffice, in spite of the computer store's admonition. The reason for splitting power from the SSD is because an SSD draws much less power than spinning rust.

Can it still go wrong? Yes, but that's the inherent risk when pushing beyond the design criteria of what this machine was originally built for. That said, "going wrong" typically means "won't turn on", not "halt and catch fire".

IMO, circular buffers with two advancing pointers are an awesome data structure for high performance compute. They're used in virtualized network hardware (see virtio) and minimizing Linux syscalls (see io_uring). Each ring implements a single producer, single consumer queue, so two rings are usually used for bidirectional data transfer.

It's kinda obscure because the need for asynchronous-transfer queues doesn't show up that often unless dealing with hardware or crossing outside of a single CPU. But it's becoming relevant due to coprocessors (ie small ARM CPUs attached to a main CPU) that process offloaded requests and then quickly return the result when ready.

Alas, there is no such option on my TV. The TV takes about maybe 10-15 seconds to turn on, which suggests that it's actually booting up its embedded processor.

I used to have one such power strip for my computer, but it quickly became infeasible for the following use-case: computer goes into sleep mode, power strip shuts off power to other accessories, those accessories drop off the USB bus, the computer wakes up from a change in USB state, the power strip turns everything back on.

The Bose speakers were one such USB device that caused this behavior, and it's hopefully the last such device that needs its own power.

If I understand the Encryption Markdown page, it appears the public/private key are primarily to protect the data at-rest? But then both keys are stored on the server, although protected by the passphrase for the keys.

So if the protection boils down to the passphrase, what is the point of having the user upload their own keypair? Are the notes ever exported from the instance while still being encrypted by the user's keypair?

Also, why PGP? PGP may be readily available, but it's definitely not an example of user-friendliness, as exemplified by its lack of broad acceptance by non-tech users or non-government users.

And then, why RSA? Or are other key algorithms supported as well, like ed25519?

Please explain what you mean by "rotate". The thermostat is physically turning in-place, as though a wall clock?

One way to make this more Pythonic -- and less C or POSIX-oriented -- is to use the pathlib library for all filesystem operations. For example, while you could open a file in a contextmanager, pathlib makes it really easy to read a file:

    
from pathlib import Path
...

config = Path("/some/file/here.conf").read_text()

This automatically opens the file (which checks for existence), reads out the entire file as a string (rather than bytes, but there's a method for that too), and then closes up the file. If any of those steps go awry, you get a Python exception and a backtrace explaining exactly what happened.

To many of life's either-or questions, we often struggle when the answer is: yes. That is to say, two things can hold true at the same time: 1) LLMs can result in job redundancies, and 2) LLMs hallucinate results.

But if we just stopped the analysis there, we wouldn't have learned anything. To use this reality to terminate any additional critical thinking is, IMO, wholly inappropriate for solving modern challenges, and so we must look into the exact contours of how true these statements are.

To wit, LLM-induced job redundancies could come from skills which have been displaced by the things LLMs can do well. For example, typists lost their jobs when businesspeople were expected to operate a typewriter on their own. And when word processing software came into existence for the personal computer, a lot of typewriter companies folded or were consolidated. In the case of LLMs, consider that people do use them to proofread letters for spelling and grammar.

Technologically, we've had spell-check software for a while, but grammar was harder. In turn, an industry appeared somewhere in the late 2000s or early 2010s to develop grammar software. Imagine how the software devs at these companies (eg Grammarly) might be in a precarious situation, if an LLM can do the same work. At least with grammar checking, even the best grammar software still struggles with some of the more esoteric English sentence constructions, so if an LLM isn't 100% perfect, that's still acceptable. I can absolutely see the fortunes of grammar software companies suffering due to LLMs, and that means those software devs are indeed threatened by what LLMs can do.

For the second statement, it is trivial to find examples of LLMs hallucinating, sometimes spectacularly or seemingly ironic (although an LLM would be hard-pressed to simulate the intention of irony, I would think). In some fields, such hallucinations are career-limiting moves for the user, such as if an LLM was used to advise on pharmaceutical dosage, or used to draft a bogus legal appeal and the judge is not amused. This is very much a FAFO situation, where somehow the AI/LLM companies are burdened with none of the risk and all of the upside. It's like how autonomous driving automotive companies are somehow allowed to do public road tests of their beta-quality designs, but the liability for crashes still befalls the poor sod seated behind the wheel. Thoss companies just keep yapping about how those crashes are all "human error" and "an autonomous car is still safer".

But I digress.

My point is that LLMs have quite a lot of capabilities, and people make a serious mistake when they assume its incompetence in one capacity reflects its competency in another. This is not unlike how humans assess other humans, such as how a record-setting F1 driver would probably be a very good chauffeur for a limousine company. But whereas humans have patterns that suggest they might be good (or bad) at something, LLMs are a creature unlike anything else.

I personally am not bullish on additional LLM improvements, and think the next big push will require additional academic research, being nowhere near commercialization. But even I have to recognize that some very specific tasks are decent using today's availabile LLMs. I just don't think that's good enough for me to consider using them, given their subscription costs, the possibility of becoming dependent, and being too niche.

Using an MSP430 microcontroller, I once wrote an assembly routine that (ab)used its SPI peripheral in order to stream a bit pattern from memory out to a GPIO pin, at full CPU clock rate, which would light up a "pixel" -- or blacken it -- in an analog video signal. This was for a project that superimposed an OSD onto the video feed of a dashcam, so that pertinent vehicle data would be indelibly recorded along with the video. It was for one heck of a university project car.

To do this, I had to study the MSP430 instruction timings, which revealed that a byte could be loaded from SRAM into the SPI output register, then a counter incremented, then a comparison against a limit value in a tight loop, all within exactly 8 CPU cycles. And the SPI completes an 8-bit transfer every 8 SPI clock cycles, but the CPU and SPI blocks can use the same clock source. In this way, I can prepare a "frame buffer" of bits to write to the screen -- plenty of time during the vertical blanking interval of analog video -- and then blast it atop the video signal.

I think I ended up running it at 8 MHz, which gave me sufficient pixel resolution on a 480i analog video signal. Also related was the task of creating a set of typefaces which would be legible on-screen but also be efficient to store in the MSP430's limited SRAM and EEPROM memories. My job was basically done when someone else was able to use printf() and it actually displayed text over the video.

This MSP430 did not have a DMA engine, and even if it did, few engines permit an N-to-1 transaction to write directly to the SPI output register. Toggling the GPIO register directly was out of the question, due to taking multiple clock cycles to toggle a single bit and load the next value. Whereas my solution was a sustained 1 bit per clock cycle at 8 MHz. All interrupts disabled too, except for the vertical and horizontal blanking intervals, which basically dictated the "thinking time" available for the CPU.

For a single password, it is indeed illogical to distribute it to others, in order to prevent it from being stolen and misused.

That said, the concept of distributing authority amongst others is quite sound. Instead of each owner having the whole secret, they only have a portion of it, and a majority of owners need to agree in order to combine their parts and use the secret. Rather than passwords, it's typically used for cryptographically signing off on something's authenticity (eg software updates), where it's known as threshold signatures:

Imagine for a moment, instead of having 1 secret key, you have 7 secret keys, of which 4 are required to cooperate in the FROST protocol to produce a signature for a given message. You can replace these numbers with some integer t (instead of 4) out of n (instead of 7).

This signature is valid for a single public key.

If fewer than t participants are dishonest, the entire protocol is secure.

Weather station, terrestrial/satellite TV DVR (TVHeadend), Git repository (Forgejo for a nice web UI, cgit for a classic UI), DNS resolver.

litchralee

@ litchralee @sh.itjust.works

Posts

4
Comments

240
Joined

3 yr. ago

litchralee

I made a way to remotely control my homelab without any internet access required

Help learning low-level dev

Can you help me adapt the Signal TLS Proxy to be used behind Nginx Proxy Manager?

Can you help me adapt the Signal TLS Proxy to be used behind Nginx Proxy Manager?

FOSDEM 2026, one of the world's largest software meeting will start in 2 days in Brussels 🇧🇪. This edition features 1176 speakers and 1063 events

How useful are functional programming languages?

Best Practices for Encrypted Search

Going Dark: Looking for the End of the Internet, Part 3: The Gemini Project (2020)

solved (sort of): How are people discovering random subdomains on my server?

Options to Expand Sata HDD Drives on a Lenovo Computer

What are some cool and obscure data structure you know of?

Audited my home electricity consumption; found two "phantom" loads

Audited my home electricity consumption; found two "phantom" loads

Finally implemented encryption in Jotty <3

is it normal for smart thermostat to rotate very often for no appearant reason?

Kinda proud of this python script I wrote

Permanently Deleted

what's the coolest thing you have ever programmed?

Password managers...

Fun/interesting things to self host?