I have much more powerful hardware than you but Jellyfin shouldn’t take more than a few seconds to load directories.
That has been my experience as well with accessing it via the web interface. I read up on the kodi addons a bit more. As I understand it the jellyfin addon integrates the jellyfin library and has to sync it. I think this causes the delay for me. I think I will give Jellycon another try, since it accesses the libraries themselves. I will take a look at my media management and refine the folder structure to make it work.
TBH I find Jellyfin works best with media that can be scraped or that I’m willing to create nfo files for.
Metadata is actually an aspect I have little trouble with. There are solutions for TV and movies and pinchflat actually provides metadata for the youtube videos in a format jellyfin can use.
Beets can organize your music library. It uses musicbrainz IDs to identify songs and albums, it can move/copy the files into folders. You can also convert files right away. I created additional mp3 files and moved Thema into a separate folder for use on my phone to save space.
There are plugins to normalize the volume. Loads of dstuff. You just have to set up a config file to your needs and you are good to go.