Have you considered using a podcast player which can play local files?
They are specifically built with annotations and notes and should help better in this regard. I can't give specific recommendation but you can find a good one on fdroid by searching and using some.
Maybe, switching to Linux would've been more beneficial.