FINALLY GOT IT!My mistake was idiotic, I was not including the gmt_unix_time bytes as part of the Client/Server Random and that was messing everything up. Now the keyblocks are calculated correctly.
As is suggest quite strongly by the payload lengths (and common sense) the actual control channel data is encrypted using AES256, not public key. Public key or diffie helman is still used only to negotiate a block key.
FYI for other learners, it's possible to get OpenVPN to return quite a bit of useful information about what's going on "under the hood" as OpenSSL does its thing in the background. I ended up inserting the following logic:
void MYdebug(SSL * s)
{
struct gc_arena gc = gc_new();
if (NULL != s->session)
{
if (NULL != s->session->master_key) // this master key is the result of applying TLS PRF once to the premaster key. so only apply the second round of TLS PRF (i.e., using "key expansion" label. For OpenVPN with AES256 the block size will be 136 bytes)
{
dmsg (D_TLS_DEBUG,"MY: session master key %s",format_hex(s->session->master_key,s->session->master_key_length,0,&gc));
}
if (NULL != s->session->cipher)
{
dmsg (D_TLS_DEBUG,"MY: key cipher name %s",s->session->cipher->name);
}
if (NULL != s->s3)
{
dmsg (D_TLS_DEBUG,"MY: Client Random %s",format_hex(s->s3->client_random,SSL3_RANDOM_SIZE,0,&gc));
dmsg (D_TLS_DEBUG,"MY: Server Random %s",format_hex(s->s3->server_random,SSL3_RANDOM_SIZE,0,&gc));
}
if ((void *)&(s->s3->tmp)) // check the keyblock value. it will exist only briefly but WILL be caught during one of the calls to write_ciphertext()
{
dmsg (D_TLS_DEBUG,"MY: key block %s",format_hex(s->s3->tmp.key_block,s->s3->tmp.key_block_length,0,&gc));
}
}
gc_free(&gc);
}
And insert MyDebug in the following functions, after the ASSERT() logic: key_state_write_plaintext,key_state_write_ciphertext