Friday, November 6, 2015

Playing with Certificates (from a Researcher's Perspective)

I often face a problem that I need to test several TLS servers. In order to make the tests consistent, I want to deploy the same keys and certificates on each server. However, this is not that easy, since there are several key formats and generation mechanisms. Deploying the same key to an OpenSSL and JSSE servers is thus a huge pain...
In the following, I will give a brief overview on basic certificate types and on few conversion possibilities.

Types and Formats

First of all, we need to have a short summary of the relevant standards...

ASN.1

Abstract Syntax Notation One (ASN.1) is an ancient format from 1988 which defines data in an abstract way. It is a basis for further standards described here. What you need to remember is: ASN.1 is complicated, do not try to understand it or implement it (unless you are a researcher or penetration tester starting a big fuzzing project).

DER

Distinguished Encoding Rules (DER) is a standard based on ASN.1. It defines how to export private and public keys, in a binary format.
Typical file extensions: .der, .cer

PEM

Privacy-Enhanced Mail (PEM) is an ASCII encoded DER (it uses Base64). You can easily recognize these files since they typically start with "-----BEGIN CERTIFICATE-----" (if they contain a certificate) or with "-----BEGIN PRIVATE KEY-----" (if they contain a private key)
Typical file extensions: .pem, .crt, .key

X.509

X.509 is a huge standard describing PKI (Public Key Infrastructure). This standard also defines a certificate structure, thus we are often talking about "X.509 certificates". The X.509 format relies on DER.

PKCS#12

"Container" format for storing cryptographic objects in a single encrypted (and signed) file. Typically, it is used to store a key and a certificate, for example in Microsoft products. It is defined by RFC7292.
Typical file extensions: .p12, .pfx

JKS

Similarly to PKCS#12, Java KeyStore (JKS) is a format for storing keys and certificates. It is typically used in Java applications. In order to create it, you can use the keytool command provided with the JDK (or, if you like GUIs, you can try Portecle).
Typical file extensions: .jks, .keystore

From JKS to PEM

In the following we will assume we would like to generate an RSA key using a Java keytool and then export this key to a PEM certificate. You can proceed as follows:

Generate a 2048 bit key pair, valid for 2 years (we are going to be asked for some command line inputs):
keytool -keystore rsa.jks -genkeypair -alias alias -validity 730 -keysize 2048 -keyalg rsa

Export the key to PKCS#12 (we need to perform this step since we cannot convert JKS directly to PEM):
keytool -importkeystore -srckeystore rsa.jks -destkeystore rsa.p12 -srcstoretype jks -deststoretype pkcs12

Export to PEM:
openssl pkcs12 -in rsa.p12 -out rsa.pem -nodes
The nodes parameter ensures we export both the certificate and the private key (you can verify this by opening the PEM file in your text editor). If we want to export only one of them, we can use (for certificate): 
openssl pkcs12 -in rsa.p12 -out rsacert.pem -nokeys

or (for a key):
openssl pkcs12 -in rsa.p12 -out rsakey.pem -nocerts

If we want to remove the password from the PEM file, we can simply use the rsa command and import/export the key again (Do NOT do this for real servers). This is useful if we need to start the test server again and again, so we do not need to provide the password:
openssl rsa -in rsakey.pem -out rsakey.pem

I experienced some compatibility problems when parsing such keys without passwords. In particular, the Botan library does not like these keys and complains about BER encoding. If you experience the same problems, consider to use the genpkey command (see below).

In case you wanted to use an elliptic curve key pair, it is also very easy. In the first step, you just need to generate a key using an ec algorithm:
keytool -keystore ec256.jks -genkeypair -alias alias -validity 730 -keysize 256 -keyalg ec

Afterwards, you proceed as in the previous steps.

From PEM to JKS

Now, we assume we would like to do this differently and first generate a key pair using openssl. Afterwards, we want to convert this key pair to a JKS file.

First, we need to generate a key pair and a certificate file:
openssl req -new -newkey rsa:2048 -x509 -keyout rsa.key -out rsa.crt

Even though now you have two files with different extensions, they are actually in a PEM format (just open them in your editor and you can  see "-----BEGIN CERTIFICATE-----"). If you want to have them in a single file, just copy-paste them into a new file, e.g. rsa.pem. In Linux you just need to:
cat rsa.key rsa.crt > rsa.pem

Afterwards, we need to convert these files into a PKCS#12 format:
openssl pkcs12 -export -out rsa.p12 -inkey rsa.key -in rsa.crt

Finally, we are ready to create a JKS file:
keytool -importkeystore -srckeystore rsa.p12 -srcstoretype pkcs12 -destkeystore rsa.jks -deststoretype jks

Further Useful Commands

Read and display a key (from a PEM file)

openssl rsa -in rsa.key -text

Read and display a certificate

openssl x509 -in rsa.pem -text

Read and display a DER certificate

openssl x509 -inform DER -in rsa.pem -text

Read and display a JKS file

keytool -list -v -keystore rsa.jks

Convert DER to PEM

openssl x509 -inform DER -in rsa.der -outform PEM -out rsa2.pem

Convert PEM to DER

openssl x509 -inform PEM -in rsa.pem -outform DER -out rsa.der

Verify a certificate issued by a given certificate authority

openssl verify -CAfile <ca-cert.crt> <cert.crt>

Generate a private without password directly

If you want to create a private key  without password directly, you can use the genpkey command. Afterwards, you generate the certificate:
openssl genpkey -algorithm RSA -out rsa2048.pem -pkeyopt rsa_keygen_bits:2048 
openssl req -key rsa2048.pem -new -x509 -days 365 -out rsa2048.crt

References and further Information

With the above given commands, you are able to generate keys for your Tomcat or OpenSSL servers.
If you are missing any information or further commands, you can take a closer look at the sources I used. There you can definitely find other useful commands:

Beliebte Posts