OpenVPN and IP cameras - Mobotix

Muse95

Hello
I have a situation where a person I am trying to help is behind an ISP double NAT. I have seen two variations on this. One is that it is a private range address on the home router WAN address, and the other is that it is a public range address but different than the WAN address reported by "What's my IP". The second layer of NAT is always in the ISP equipment and not accessible for port forwarding. The ISP also blocks all unsolicited incoming requests. DDNS does not work.
I want to be able to access an IP camera with a webserver.
We have set up a droplet with an OpenVPN server. We have put a client file on my home router (Asus RT-AC66U with Merlin) and another client file on my android phone (OpenVPN Connect). Client to client is set up with a ccd on the server that redirects addresses that are "LAN" type matching the Router (ie 172.16.xxx.xxx).
We have gotten this to work. I have typed in my router address and seen the router sign in page. I have also seen it work with a more basic IP camera with a built in webserver - I got to the camera sign-in page.
I have not been able to make this work with a Mobotix brand camera. Does anyone have any ideas?
I figure there must be a security setting but I cannot find it. Mobotix has the ability to set up OpenVPN right into it, but I already have this other method working and it is one client file for a router full of cameras versus individual client files for each camera.

Muse95

I have a theory.
If you use a Mobotix camera, you notice that when you log in, the login window is a separate popup.
I suspect that the popup is being blocked by something. The possible options: VPN configuration either in the router or the server. Or maybe the browser settings. I have tried to turn on allowing popups in the browser, but it has not worked. So I'm back to the VPN.
Any ideas?

Muse95

OK, still working on this. Tried a different tack Trying to connect the camera directly to the server. There are OpenVPN connection setup options right in the camera, but I am possibly suspecting that it is using an older version of OpenVPN. The main issue I am having is with TLS handshake errors.
They don't allow you to load a client config file, they have a screen that sort of mimics creating a config file, but I don't see options for a TA key.
Here is the screen for loading in certificates:

Here is the screen for setting up the camera as a client

There is no place to specifically load a TLS key. And I always get a TLS handshake failed, TLS key negotiation failed to occur.

Any help?
Here is my cleaned server config file, it shows a few different settings we've tried - things that are commented out and replaced.

#Set TLS server mode and device - no do not use TLS
#mode server
# following line from Ray commented out
#tls-server
dev tun0

# Listen on UDPv4 1194 only
port 1194
proto udp4

# Set network topology and IP addressing for client endpoints
topology subnet
server 10.8.0.0 255.255.255.248

# Root and Server certificates and Server private key
ca ca.crt
cert ServerMx1.crt
key ServerMx1.key

# Disable Diffie-Helman since we are using ECC?
# dh none
# trying Diffie-Hillman instead
dh 02.pem

# Maintain record of client to virtual IP address associations
ifconfig-pool-persist ipp.txt

# enable client-to-client communication
client-to-client

# Set keepalive timers, the mutual authentication key and what cipher to use
keepalive 10 120
# try commenting out because we are not using TLS with Mobotix
#tls-auth secret.key 0
#push "compress lz4-v2"
cipher AES-256-CBC

# Enable compression
# compress lz4-v2
# push "compress lz4-v2"
# Try older compression
comp-lzo

# Limit number of concurrent client connections and Reduce daemon privileges
max-clients 6
persist-key
persist-tun
user nobody
group nobody

# Set logging verbosity, 0 is silent, 4 is good for production, higher for debugging
verb 3

# Notify client when server restarts so it can automatically reconnect
explicit-exit-notify 1

# uncomment line below to fix TCP-MSS.
;mssfix 1440

TinCanTech

Some things to try:

Increase the log verb to 4 in your client (camera) setup in order to resolve any problems.
--cipher must match on client and server - AES-256-CBC is preferred over Blowfish-CBC
If your client can not use a TLS auth key then you cannot use --tls-auth
This is not a deal breaker but it is preferred ..
Ensure you use the same --comp-lzo on both client and server
--comp-lzo can be pushed from the server which makes this easier to maintain.
Check to see what the latest version of openvpn your camera uses.
Also check for upgrades for your camera which may have more up to date openvpn.

Muse95

Sorry, I showed the default screen in the camera without the setup. I should have clarified, because I didn't want to remove some of my other settings.
I am using
UDP
AES-256-CBC
LZO compression is enabled
And logging is at Verb 4
Here is some of the log info, just cutting and pasting some pieces that don't have identifying info:
...
VERIFY EKU OK
++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Validating certificate extended key usage
VERIFY KU OK
...
Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-CBC,auth SHA1,keysize 256,key-method 2,tls-server'
Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-CBC,auth SHA1,keysize 256,key-method 2,tls-client'
Data Channel MTU parms [ L:1622 D:1450 EF:122 EB:406 ET:0 EL:3 ]
Control Channel MTU parms [ L:1622 D:1212 EF:38 EB:0 ET:0 EL:3 ]
LZO compression initializing
Re-using SSL/TLS context
Restart pause, 5 second(s)
SIGUSR1[soft,tls-ERROR] received, process restarting
TCP/UDP: CLOSING socket
TLS ERROR: TLS handshake FAILED
TLS ERROR: TLS key negotiation FAILED to occur within 60 seconds (check your network connectivity)

TinCanTech

You need to check your server log, again at verb 4, for errors.

Muse95

With verb 4 in the server config, as well, now
systemctl status -1 ...
gives the following error:
TLS Error: cannot locate HMAC in incoming packet from [AF_INET]xxx.xxx.xxx.xxx:yyyyy

TinCanTech

Muse95 wrote TLS Error: cannot locate HMAC in incoming packet

Which means your server is still using --tls-auth ..

Muse95

OK. Thank you.
Do you think this is related to how we generated the keys using the easyrsa script and our vars file?
Do we need to redo that with some different variables? Or am I looking in the wrong place to root out the TLS stuff?
I must admit (it is probably obvious) but I have been doing a bit of monkey copying, without understanding fully the underlying actions happening. I need to dig deeper, rather than randomly changing things and trying to accidentally stumble on what works.

TinCanTech

HMAC is --tls-auth only .. so remove it from the server config and restart the server ..

Muse95

I had it commented out. Is that not good enough?

TinCanTech

It could also be --auth .. which is SHA1 by default.

Please post your complete logs at verb 4

Muse95

When we generated the keys, we were given a vars file to edit as needed
This was the default given us
set_var EASYRSA "/etc/ openvpn / easyrsa "
set_var EASYRSA_PKI " $EASYRSA /pki"
set_var EASYRSA_DN "org "
set_var EASYRSA_REQ_COUNTRY "CA"
set_var EASYRSA_REQ_PROVINCE " myprovince "
set_var EASYRSA_REQ_CITY " mycity "
set_var EASYRSA_REQ_ORG " companyname "
set_var EASYRSA_REQ_EMAIL " MY_EMAIL_ADDRESS "
set_var EASYRSA_REQ_OU " $DEPARTMENT "
# Use Elliptic Curve instead of RSA
set_var EASYRSA_ALGO ec
set_var EASYRSA_CURVE secp521r1
# RootCA expires in 10 years
set_var EASYRSA_CA_EXPIRE 3650
# issued certificates expire in 2 years
set_var EASYRSA_CERT_EXPIRE 730
# Do not support Netscape extensions
set_var EASYRSA_NS_SUPPORT "no"
# Use sha512 cryptographic digest
set_var EASYRSA_DIGEST " sha512 "

I modified this recently to try to correct the issues and used the following instead:
set_var EASYRSA "/etc/ openvpn / easyrsa "
set_var EASYRSA_PKI " $EASYRSA /pki"
set_var EASYRSA_DN "org "
set_var EASYRSA_REQ_COUNTRY "CA"
set_var EASYRSA_REQ_PROVINCE " myprovince "
set_var EASYRSA_REQ_CITY " mycity "
set_var EASYRSA_REQ_ORG " companyname "
set_var EASYRSA_REQ_EMAIL " MY_EMAIL_ADDRESS "
set_var EASYRSA_REQ_OU " DEPARTMENT "
set_var EASYRSA_KEY_SIZE 2048
# Use RSA
set_var EASYRSA_ALGO rsa
# RootCA expires in 10 years
set_var EASYRSA_CA_EXPIRE 3650
# issued certificates expire in 10 years
set_var EASYRSA_CERT_EXPIRE 3650
# Do not support Netscape extensions
set_var EASYRSA_NS_SUPPORT "no"
# Use sha256 cryptographic digest
set_var EASYRSA_DIGEST " sha256 "

Then we ran
easyrsa init-pki
and
easyrsa build-ca

Is there something you would change?

OK, the logs from the camera. I think these are not in order because I think it outputs some stuff then it waits and outputs another blurb at the top of the log window, etc. This is just copy and pasted out of the log window.
SIGTERM[soft,exit-with-notification] received, process exiting
TCP/UDP: CLOSING socket
SIGTERM received, sending exit notification to peer
event_wait : Interrupted system call (code=4)
VERIFY OK: depth=0, C=CA, ST=..., L=..., O=..., OU=..., CN=..., emailAddress=...
VERIFY EKU OK
++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Validating certificate extended key usage
VERIFY KU OK
VERIFY OK: depth=1, C=CA, ST=..., L=...n, O=..., OU=..., CN=..., emailAddress=...
TLS: Initial packet from [AF_INET]123.123.123.123:1194, sid=56fdf206 e373ecde
UDP link remote: [AF_INET]123.123.123.123:1194
UDP link local: (not bound)
Socket Buffers: R=[110592->110592] S=[262144->262144]
TCP/UDP: Preserving recently used remote address: [AF_INET]123.123.123.123:1194
Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-CBC,auth SHA1,keysize 256,key-method 2,tls-server'
Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-CBC,auth SHA1,keysize 256,key-method 2,tls-client'
Data Channel MTU parms [ L:1622 D:1450 EF:122 EB:406 ET:0 EL:3 ]
Control Channel MTU parms [ L:1622 D:1212 EF:38 EB:0 ET:0 EL:3 ]
LZO compression initializing
Re-using SSL/TLS context
Restart pause, 5 second(s)
SIGUSR1[soft,tls-ERROR] received, process restarting
TCP/UDP: CLOSING socket
TLS ERROR: TLS handshake FAILED
TLS ERROR: TLS key negotiation FAILED to occur within 60 seconds (check your network connectivity)
VERIFY OK: depth=0, {same as above}
VERIFY EKU OK
++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Validating certificate extended key usage
VERIFY KU OK
VERIFY OK: depth=1, {same as above}
TLS: Initial packet from [AF_INET]123.123.123.123:1194, sid=77af0f6b 50187f63
UDP link remote: [AF_INET]123.123.123.123:1194
UDP link local: (not bound)
Socket Buffers: R=[110592->110592] S=[262144->262144]
TCP/UDP: Preserving recently used remote address: [AF_INET]123.123.123.123:1194
Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-CBC,auth SHA1,keysize 256,key-method 2,tls-server'
Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-CBC,auth SHA1,keysize 256,key-method 2,tls-client'
Data Channel MTU parms [ L:1622 D:1450 EF:122 EB:406 ET:0 EL:3 ]
Control Channel MTU parms [ L:1622 D:1212 EF:38 EB:0 ET:0 EL:3 ]
LZO compression initializing
library versions: OpenSSL 1.0.2k 26 Jan 2017, LZO 2.03
OpenVPN starting up...
auth_user_pass_file = '[UNDEF]'
pull = ENABLED
client = ENABLED
tls_crypt_file = '[UNDEF]'
tls_auth_file = '[UNDEF]'
tls_exit = DISABLED
push_peer_info = DISABLED
single_session = DISABLED
transition_window = 3600
handshake_window = 60
renegotiate_seconds = 3600
renegotiate_packets = 0
renegotiate_bytes = -1
tls_timeout = 2
ssl_flags = 0
remote_cert_eku = 'TLS Web Server Authentication'
remote_cert_ku = 0
remote_cert_ku = 65535
ns_cert_type = 0

I do not know where all this TLS stuff is coming from.

When the server is started, I see:
openvpn@xyz.service - OpenVPN Robust And Highly Flexible Tunneling Application On xyz
Loaded: loaded (/usr/lib/systemd/system/openvpn@.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2018-05-22 12:33:32 CST; 5s ago
Main PID: 19588 (openvpn)
Status: "Initialization Sequence Completed"
CGroup: /system.slice/system-openvpn.slice/openvpn@xyz.service
└─19588 /usr/sbin/openvpn --cd /etc/openvpn/ --config xyz.conf

May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=722714 Data Channel MTU parms [ L:1622 D:1450 EF:122 EB:406 ET:0 EL:3 ]
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723488 Socket Buffers: R=[212992->212992] S=[212992->212992]
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723586 UDPv4 link local (bound): [AF_INET][undef]:1194
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723784 UDPv4 link remote: [AF_UNSPEC]
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723803 GID set to nobody
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723816 UID set to nobody
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723840 MULTI: multi_init called, r=256 v=256
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723875 IFCONFIG POOL: base=10.8.0.2 size=4, ipv6=0
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723894 IFCONFIG POOL LIST
May 22 12:33:32 VPN-test-pilot openvpn[19588]: Tue May 22 12:33:32 2018 us=723976 Initialization Sequence Completed

Then when I enable the camera, I get the HMAC error reported above in a previous post.

TinCanTech

Can you verify the openvpn version on both server and client ?

$ openvpn --version

Muse95

I don't know if I can tell on the camera. The only guide I ever got from them said that it supported OpenVPN from version 4.2.1.43 on, which makes no sense. Document was copyrighted in 2014.
The server is using
OpenVPN 2.4.5 x86_64-redhat-linux-gnu [Fedora EPEL patched] [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Mar 1 2018
library versions: OpenSSL 1.0.2k-fips 26 Jan 2017, LZO 2.06

Muse95

The 4.2.1.43 refers to the Mobotix firmware, I guess. Not a problem. I am up to date.
In the opensource listing it mentions
******* openssl *******
Copyright (c) 1998-2011 The OpenSSL Project. All rights reserved.

TinCanTech

Muse95 wrote The server is using
OpenVPN 2.4.5 x86_64-redhat-linux-gnu [Fedora EPEL patched] [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Mar 1 2018
library versions: OpenSSL 1.0.2k-fips 26 Jan 2017, LZO 2.06

ok.

Muse95 wrote I don't know if I can tell on the camera. The only guide I ever got from them said that it supported OpenVPN from version 4.2.1.43 on, which makes no sense. Document was copyrighted in 2014.

Maybe the client only has an early version of 2.3 .. you could downgrade your server:
https://openvpn.net/index.php/open-source/downloads.html

Try building openvpn version 2.3.18 from source on your server.

Version 2.3 with 2.4 probably ought to work in all but the most extreme corner cases ..
You really need to post the complete server & client logs at verb 4

Muse95

I don't know what else I could post of the logs?
I posted everything except server IP address, removed any ref to the name of the config file, and the specific CN type var details. Why would you need to see any of that. Is there something specific you are looking for?

TinCanTech

I suggest you contact the camera support channel because without the full log (with your private details removed, that is ok) I cannot help any further.

smith78

I really agree to your viewpoint.The informations are illustrative and brief.I have tried out that .I have following two queries in this trail that I have setup my router to create a OpenVPN connection and I have installed the OpenVPN app on my Samsung Android phone. Uptil this it worked out fine.
Now I just want to view my cameras using the iVMS-4500 app on my phone, while I'm on the OpenVPN connection. But I don't know anything about what information I have to enter into the app.I have thought that I can choose between HiDDNS - IP/Domain - IP Server, but I think I have to use IP/Domain.
But,when I selecting the IP/Domain, I entered the IP of my camera (10.0.0.100) in the adresse field, and the portnumber in the port field, and then username and password. But it does not work.And secondly,I am unable to setup VPN server In my Arlo pro 2.Suggest us what to do regarding these