I guess I'm not explaining myself well, so I made a picture.
Both the NAT and the VPN have a public IP and can access the Internet directly.
On the firewall, I have a certain set of rules for the NAT and a certain set of rules for the VPN server. For the NAT, I allow all outgoing connections, but no incoming connections. For the VPN server, I allow incoming connections only on the OpenVPN ports, and no outgoing connections. All of this, of course, under the assumption that the stateful firewall is smart enough to allow connections once they were established.
When an OpenVPN client connects to the VPN server, its incoming connection goes through (1).
When the VPN server wants to make an outgoing connection, it goes through (2) and (3), not exposing its own IP.
If, on the VPN server, I set the firewall IP address as gateway, the OpenVPN connections work fine. It comes in through (1) and goes out through (1). However, I can't establish any outgoing connections as they will also be routed directly through the firewall over (1), which will block the packets.
If, on the VPN server, I set the NAT IP address as gateway, outgoing connections work fine. They go out through (2) and (3) and come back through (3) and (2). However, incoming OpenVPN connections don't get their response, as they will come through (1), but the response packets will be routed through (2) and (3). The client never accepts these packets coming from an unkown IP (the NAT IP address), so it appears as if the server times out / doesn't respond.
What I want is to route all traffic through (2) and (3), unless there are incoming packets on (1). In this case, the return traffic for those packets should also go back through (1). As (1) only allows incoming packets on OpenVPN ports, it would only apply to VPN traffic, but I guess you could also generalize it.