If you're concerned about data retention, you'd want to select an inference provider that is listing 'ZDR' (zero data retention) as a feature.
For example, a lot of the Chinese open weight models have become quite capable and because their weights are available end up like generic vs brand name medicine where there's multiple providers serving them with different production conditions.
Because enterprise use will often be worried about data retention or sending data to China, the alternative providers usually offer things like US-only inference or zero data retention.
If you're not going to use it all that often, a la carte API use is going to be way cheaper than a subscription, probably better results than a free plan with a closed model provider, and give you more control over the process.
Probably less so.
The hardware to run it locally would be fairly expensive and would require using a very simple model compared to alternatives. Also much more wasteful if you were only using that hardware for AI use, as you'd be distributing the hardware out rather than centralizing so it'd be often idle and when replaced create more waste than a centralized server rack.
Also, additional per user post-training seems to me both wasteful and not necessary. In context learning is often much more powerful but frequently overlooked.