Check out BookOrbit if you find Grimmory too hefty in the RAM reqs. It uses very little RAM for me. I think it also has import from Grimmory as well.
As for the database Q, if you’re using docker/Podman it’ll be very easy to point the database folder (if included in the image, like Jellyfin does) to your local filesystem and the media to your NAS. Same idea if the container uses a Postgres container, just spread across two containers in that case.
Interested in what you divine, I’m switching from a USB drive with the key in it to one of those fancy things.
Oh damn this could replace the bookmakers. I have Linkding with the Linkding Injector extension, but this would be next level.
You can view the saved text too, nice. Would it be possible to attach an HTML file from like single file for sites that have heavy image content as part of the “view” button? That’d completely replace Linkding/Karakeep/Linkwarden use cases for me
Understandable if not, that’s ancillary to the text search focus
Choose Debian. Real secret reason? Debian means you have to upgrade to new versions less. Ubuntu LTS lasts as long as Debian (5 years) but they crank out a new LTS every 2 years. If you have 3rd party sources, I’ve run into they only support the two latest LTSes - well you have one year left on your 5 year support… but do you really? And now you can put on your sad face because you get to do two LTS upgrades. All while Debian was over there on one version and everyone will target that one version and its previous version no problem.
Debian just makes better sense for a server.
(And Ubuntu is stupid with its “ooh extra patches just register your machine with our central authority” ok bud my Debian machine just says I have mail for some reason every time I boot it up, it doesn’t make it weird and gimmicky and based on a “free” deal that could change whenever they want to jerk you around)
I use Authentik - works well - but I’m prepping to switch to Authelia for the config-based setup.
I updated Authentik across 2 versions (they did 2 month-long supported tags, missed one; now they do 3 month-long supported tags) and it destroyed itself. Had to recover the DB from a backup (and then step through the version I tried to skip), and while I was doing that I was like “wait why does this have only a DB? Should just be a config file cause that’s all the depth I do with it” and lo that’s what Authelia is.
Authentik is audited and Authelia has not been. Initially while I chose Authentik. But config-file robustness in the face of Authentik’s GUI-setup DB imploding swayed me not to care, it’d take so long and be so tedious to redo all my proxies and auths through the GUI. Plus I always forget what to click in the GUI when I come back to add some new program in 4 months.
I added examples so you don't have to dig as far if you want to give it a try!
I approach it the same as I did with Docker, one user per container.
I started with rootless but networking within Podman is moot with multiple users per container. And one user for all containers to get networking has to lead to subUID clashes (and thus escape vectors) - unless someone can explain how not…
But root Podman is just as secure anyway, and easier, so I just roll with UserNS=auto and use idmap= on the volumes to enable writing as the specified user for the container. And networking in Podman works because it’s one user space. By default UserNS=auto gives 1024 subUIDs to a container. I had to up that to the expected max of 65535 for Frigate to work (example bit I use with idmap, UserNS with the size and group mapping, and GroupAdd):
Volume=/mnt/camraz:/media/frigate:rw,noexec,nosuid,nodev,Z,idmap=uids=@1111-0-1;gids=@1111-0-1#1020-1020-1
UserNS=auto:size=65535,uidmapping=105:@105:1,gidmapping=105:@105:1
GroupAdd=105
So what's going on here is on the Volume= I have the camera folder mounted to where Frigate wants it, and I specify that it accesses that volume with idmap= and frigate's UID/GID 1111 and the camera group I made at GID 1020.
The order for the idmap is @${hostUID}-${containerUID}-${num2map}. The @ symbol means that 1111 is an absolute host UID; drop that and 1111 is pulled from the containers subUID/subGID list. Most containers run as root (0) so you just map to that user.
Side quest: For containers that have one of those mini-hypervisors like LinuxServers images and their "S6" thing, I worked around them starting with a root user but then requiring moving to a non-root user by specifying PUID=1 and PGID=1 then setting idmap=uids=@1999-0-2;gids=@1999-0-2 and reserving 1999 as username containerNameStartup and 2000 as containerName. This idmap= maps host 1999 to container 0 but maps two UID/GIDs consecutively, so it also maps host 2000 to container 1 keeping everything lined up perfectly - while LinuxServer images still don't get host root like they so desperately want.
GroupAdd=105 adds in the group 105 to the container, which is notably not the host group 105. UserNS is needed to complete that. This just makes that group exist in the container.
UserNS=auto:size=65535,gidmapping=105:@105:1 accomplishes the subUID/subGID size allocation to the expected default max of 65535 (because Frigate needs it) and the group connection from host 105 to container 105. Note that this syntax helpfully uses the opposite order as idmap=. It goes ${containerUID}:@${hostUID}:${num2map} and uses colons instead of dashes.
Every other container is cool with the default 1024 (just UserNS=auto). The subUIDs are pulled from a non-existent user named containers that you need to enable for Podman root to work with UserNS, and it has like 2 million billion or something with their recommended setup, so it’s good on subUIDs/subGIDs.
Command to set up root Podman with UserNS=auto:
sudo echo "containers:2147483647:2147483648" >> /etc/subuid
sudo echo "containers:2147483647:2147483648" >> /etc/subgid
You can also have UserNS=auto and run the container as a specific host user directly instead of "nobody" with:
UserNS=auto:uidmapping=0:@1111:1,gidmapping=0:@1111:1
Same idea as how I mapped in the host group 105 above. Where the root user 0 is mapped to the host (via @) user 1111 so the container runs as the user. I don't use that because the idmap= works perfectly and then the container doesn't even run as the user that controls the files directly. But maybe there will be a use for that sometime(?).
And I do have a fuckton of users; Debian once complained it ran out of numbers or something after like 20 users, so I just ran the first thing I found to make the UID limit some really big number, and I never thought about it again! No idea what it was..
**Edit: ** I cleaned it up and added examples
Also I mostly work with UIDs not users per-say, so I specify UID 1111 has access to Frigate files. I just make a user named frigate to make it a bit easier for me. When you do ls -l it says frigate instead of 1111; life is easier.
Overall, it has been a lot of work to divine these things because the documentation and useful examples for Podman are confusing and non-existent. I think I've gotten most of this idmap= and UserNS= stuff working from reading GitHub issues that drop little tidbits. But after I get it working as I want it, it's solid, and I can now rinse-and-repeat in the future. No Docker daemon to fuck with or get hacked, and it's reliable since I just let systemd start these quadlets up.
I'm not 100% done, but I'm close. Outstanding issue right now is that on server restart sometimes some Podman networks don't actually start up correctly and then containers that want them fail to start because of that (I have to restart the Podman network via systemctl to get them to exist in Podman). But eventually I'll find the stupid GitHub issue where someone mentions something relevant for that, and then I'll be golden!
Thanks for the write up, again! I def gotta up my NetworkManager game…
This is great, I’d love to see the config files - especially the WiFi pass through stuff. No experience with that! But the rest all fitting together would be rad as hell.
Peeps would be able to get the 4GB banana zero and just rock it up quick full service!
I know that’s a big ask, sharing your cursed config files is work… if you feel up for it tho - hell yes
I have this setup. Upfront, I would not recommend Proxmox, the update methods are annoying. The better way is straight Debian with Incus installed, then you get straightforward stable Debian updates automatically - they won't break anything and you're secure. Sometime I'll redo it - I haven't because, of course, it is my router and when its down I don't have internet! So foreboding and on the back burner.
Also also Proxmox's GUI leaves a lot to be desired (for me, it looks like ass and is confusing), Incus is nicer for VM control and Cockpit is nicer for host control. After typing all that I realize I'm a hater at this point
I haven't really noticed downtime issues cause of Proxmox updates cause I just do it when nothing is happening. And Proxmox hasn't bricked itself, though I am wary of it because that has happened to others due to their rolling release update style.
I've got a Dell Wyse 5070 Extended with a 2 port Intel NIC in it. I pass both ports through leaving the built-in port for managing Proxmox.
Here are my notes:
Set NIC PCIe Passthrough for Network Card
nano /etc/default/grub
intel_iommu=on to getGRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
update-grub
nano /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
update-initramfs -u -k all
reboot
Click on 2nd level thing named router on the left side vertical bar hierarchy thing and then click in the top right the blue Create VM button.
OPNsense checked 1 15local-lvm 15 checked checked4 host {makes it not moveable between diff CPU types but will theoretically allow for more speed}2048 512checkedHardware tab in the 2nd left vertical list on the browser page and click add PCI Device ...01:00.0 I350 Gigabit... & ...01:00.1 I350 Gigabit... (1st & 2nd ones) checkedGo to the Console tab in the 2nd left vertical list on the browser page and hit enter to get to a command line in the OPNsense VM
!Add expand storage via command line!
And lastly, during setup I have these notes
It will choose wrong (WAN gets igb1 and LAN gets igb0 -> we want WAN gets igb0 and LAN gets igb1)
Default User: root, PW: opnsense (they don't tell you anywhere, you don't have internet b/c this is your new router, fuck em)
**Access at 192.168.1.1 via pluging an ethernet cable into the 1st port in a set of forwarded ports**
*Note that we will move it so the 1st port is the WAN (can't access OPNsense from the WAN port for safety), so after following this you access via 2nd port*
So watch out for those things. Not sure quite what I mean by the 1st and 2nd port things, may be related to on setup it had the order of the ports I wanted wrong so they're switched till setup is complete and it reboots.
I don't remember doing this at this point, but maybe this info dump will help!
Pangolin has a lot more going on than I expected, I thought it was just a mesh system. That might be a one-stop shop, thanks for sharing!!
I do see Authentik can apparently act as a reverse proxy, so it can sit at the very front. But I’d lose out on caddy + crowdsec then…
I’ll have to do some reading if caddy can actually just forward to caddy, and where the TLS is terminated and all that.
And I do use the internal setting now, but I need it off if I want to publish the port on the LAN so that the VM can see the ports on the LAN. But if I can have WAN caddy do the auth check and forward along good stuff to the LAN caddy, then I’ll only need to publish LAN caddy’s port and that’s not the worst at all.
Thanks for the ideas, I’ll try to cook “caddy (DMZ) -> auth OIDC (DMZ) -> caddy (LAN) -> services (LAN)”!
Rootful to get the cross-user networkingEdit: cross-user via running the containers with userNS & user or volume idmap from root Podman. Containers can be run by different users and share networks, but the containers don't have root access
Question WRT secure networking with Podman/Docker stack and a reverse proxy in a VM "DMZ"
Containers lower the bar since the developer doesn’t need to make their program work on every system - just the container’s system.
Price we pay for more programs. And they bring boons like read-only, rootless, limited capabilities, and constrained perf limits (esp if you use Podman with Quadlets).
And don’t feel trapped - the Dockerfile is a recipe to build that program. Probably want to do it in an LXC container since it’ll want to use /data for its data or something. But the LXC container can also be run as a user but the program thinks it’s root. Plenty of security abounds!
I think it’s worth the price and you’re not trapped. They’re trapped with you and your robust Quadlet files
You can slam semicolons at the ends of lines in Python, interpreter doesn’t care
P sure you can do multiple lines in one line by slamming semicolons too, though idr for sure and I can’t be arsed to check
Do you know of how it compares to the option that’s been around for a while?https://github.com/christiaangoossens/hass-oidc-auth
I see they say “seamless”; the extant one requires a different landing page and it doesn’t remember logged in browser well. So on the face of it, this sounds better.
But the one linked has had many more eyes and is made by the person who made a big stink on the forums https://community.home-assistant.io/t/open-letter-for-improving-home-assistants-authentication-system-oidc-sso/494223
For the ease teased, I’ll prob check it out though
F in the chat for your savings, least you’ve got the peak of home NASes. Pretty fuckin cool and I hold out hope when the drop comes in a… 6 months to 3 years…? that I’ll be able to afford full SSD NAS life. The power savings, the speed, the no worries of shock or vibrations, the silence - jealous
It is a gamble, fuck the AI bozos for speculating us into economic uncertainty
For power on and off automatically, I just rely on Linux’s spin down timer. Which I guess is built in - not sure of anything more specific!
IPv6 & Opnsense & Not Exposing Machine-Specific IPv6s to Corpos
The docker compose file is great, clear once you get used to all its little sections. For volumes I only use what they call “bind mounts” which are where you have a folder on your system connected to the folder in the container. As opposed to the “docker volumes” which are internal to docker.
And with those it’s super easy to just be like in the volume subsection:
- /path/to/local/drive:/container/database:rw,noexec,nosuid,nodev,Z- /path/to/network/drive:/container/media:rw,noexec,nosuid,nodev,ZThe
:rw,noexec,nosuid,nodev,Zat the end is a great extra security thing that never causes problems. rw means read-write, you can switch it to ro for read-only if you’ve got something you want the container to only read from but not be able to modify. I use that for jellyfin’s media since I don’t want it doing anything but reading it. The noexec means don't let executables be run from the folder, never should happen so it just prevents a sick hack from being put in the folder and run. I forget what the others do but they’ve never been a bother. And Z means only one process can access the volume, you can switch it to lower case z to let multiple processes access the volume - and I’m not sure it does anything without SELinux going which I think only fedora does by default right now.Enjoy the info dump!