I have two VPNs. One for privacy (e.g. general browsing) and one to reach my home NAS (e.g. to reach my selfhosted image backups). Is there a way to set the system up so that when I use applications X, Y and Z, it should use the privacy one, and if I use A, B or C, it should use the private one?
Yes! In Android use work profiles. You can use the f Droid app shelter to do this.
Then set a different VPN for your work profile, and a different VPN for your main profile. Any work profile app will use the work profile VPN
If you have Tasker, you can do that. Not sure if you are using Wireguard, but that makes it simpler to integrate IMHO with it. With it, you set a condition basically if App XYZ is open then use VPN ABC Otherwise, If app XYZ is not open, then use BCA VPN.
The private one is using Wireguard, buuut I’d also like for it to work even when the apps are backgrounded.
I’m beginning to think that the cleverer solution would be to set something up in my home Linux environment to make the routing decisions and then just have an always on VPN home.
It just seems like it should be a solved problem on my mobile OS already.
With Linux, you need to have the awareness of what is running on your phone 24/7 and AFAIK, there’s nothing like that except maybe Home Assistant, but that’s pushing it. I have a buddy which has an always on VPN which he uses 100% of the time at home, never any issues with him. As for the phone, I have Wireguard for when I am not connected to my WiFi, and then disconnect when I’m back on my WiFi network. This way, I am certain to be on my home network 99.99% of the time. I use Tasker for that.
With that said, you will need an automation of sorts on the Android to be 100% sure you are on your appropriate VPN. Whether through Tasker, Macrodroid or a few other apps.
You might consider a more elegant approach to accomplish your goals.
For example, I run Tailscale on all of my devices. They are accessible to each other (at all times) through the encrypted “Tailnet” while each has its own public internet provider (my home ISP, my cellular provider, my VPS host, etc).
They all route their DNS requests through my home server which is running Adguard (for DNS ad blocking on every device). If I wanted I could route all their traffic (not just DNS) through the home server, and I could have the home server’s internet-facing interface connect through a commercial VPN to then hide all that egress traffic, across all my devices.
You could connect to your privacy VPN from your home server, then set up your home VPN to route all external traffic through the privacy VPN. From your phone, your home VPN is now both your home and privacy VPN.
Trying to do it on a per-app basis, you will very likely end up with whichever app is in the background trying to use the wrong connection.