In latest project we came across the problem of securing communication, where peers don’t share backend language. Our goal was to securely generate shared secret between Java server and .NET client. Each language supports all features needed for the key agreement, but these features aren’t always compatible, due to different encoding or data representation.

This article should be quick and easy solution how to agree on shared secret between such server and client with hand-on examples. Each subchapter contains concrete example on how to send, receive and process data needed for respected operation. Authentication and signature scheme are similar to guides published internet-wide, the main topic- key agreement is presented with our insights and comments on how to make it work.

The reason I have included all the code snippets even if they are “straight from internet” is that I wanted to simply group all these schemes to one place, so that you don’t have to find examples on Oracle, CodeRanch, StrackOverflow, MSDN Microsoft documentation, … Honestly it was a little bit frustrating for me, so there you go 🙂

If you are interested just in key agreement, feel free to skip right to the key agreement example, as I would like to discuss authentication and signature schemes first in order to prevent attacks on anonymous key agreement.

Please beware of incautious use of the published code. Your protocol might require different configuration and by simple copy-pasting you may introduce some security issues or errors to it.

Certificate based authentication

Consider each peer to have certificate signed by some root CA.

In my examples I’ll be working mostly with byte arrays as it is most universal way to represent data.

In most cases whole certificate chain is transferred to other side. Using one certificate (self-signed) is just simplification of process below.

.NET authentication

Java peer can transform certificate into byte array using Certificate.getEncoded() method. Byte array is then sent to counterpart.

.NET peer parses certificate chain data and stores parsed certificates into List<byte[]> to ease processing.

Code below shows the authentication process. Process differs from standard validation in a way, that we are sending only certificate chain. From the chain peers certificate is extracted. From the rest of received certificate chain we create subchain and check if we were able to create complete chain from peer to Certificate Authority.

For simplicity revocation checks and certificate usage field (OID of the field – 2.5.29.15) are being ignored. In real environment you should definitely check this list and field, but in this article it is not really necessary.

/**
* receivedCertificateChain previously transformed byte array containing whole certificate chain. This chain is transformed
*                          into list of separate certificates from chain 
*/
public bool certificateChainValidation(List<byte[]> receivedCertificateChain)
{
  var otherSideCertificate = new X509Certificate2(receivedCertificateChain[0]);
  var chain = new X509Chain { ChainPolicy = { RevocationMode = X509RevocationMode.NoCheck, VerificationFlags = X509VerificationFlags.IgnoreWrongUsage } };
  chain.ChainPolicy.ExtraStore.AddRange(receivedCertificateChain.Skip(1).Select(c => new X509Certificate2(c)).ToArray());

  return chain.Build(otherSideCertificate));
}

Root CA certificate must be trusted by your system to be successfully validated.

Java authentication

.NET peer can transform each certificate into byte array using X509Certificate2.RawData property and send it to counterpart.

First, Java peer needs to define validation parameters and Trust anchor. In this case it’s root CA.

After that, you validate received certificate chain. If method validate(CertPath, PKIXParameters) is processed without any exception, you can consider received certificate chain correctly validated.

/**
* certificatePath received data of certificate chain from counterpart 
*/
public boolean certificateChainValidation(byte[] certificatePath) {
  try {
    //generate Certificate Path from obtained byte array
    CertificateFactory cf = CertificateFactory.getInstance(“X.509”);
    CertPath received = cf.generateCertPath(new ByteArrayInputStream(certificatePath));

    //get all trusted entities from dedicated truststore
    Set<TrustAnchor> trustedCAs = getAnchors();

    //set the list of trusted entities and turn off the revocation check
    //revocation is by default set to true, so if you don't turn it off explicitly, exceptions will bloom
    PKIXParameters params = new PKIXParameters(trustedCAs);
    params.setRevocationEnabled(false);

    CertPathValidator cpv = CertPathValidator.getInstance("PKIX");
    //if no exception is thrown during validation process, validation is successful
    cpv.validate(received, params);
  } catch (CertificateException | NoSuchAlgorithmException | InvalidAlgorithmParameterException | CertPathValidatorException e ) {
    throw new CertificateException("Could not validate certificate", e);
    return false;
  }
  return true;
}

getAnchors() is a method which simply loads all entries from dedicated truststore and returns them as Set<TrustAnchors>.

Signatures

Without correct signature scheme whole authentication scheme would be compromised, as any attacker would be able to replay intercepted certificate chain. Therefore while sending outgoing certificate chain, each peer has to include signature to it. It is sufficient for such signature to be reasonably short (resp. long) nonce. This nonce should be freshly generated by the communication counterpart and can be sent before authentication attempt (or as an authentication challenge). Nonce could be any-random value or  static counter, but must be definitely unique.

During the key agreement phase, signatures are used to sign public keys that are exchanged between peers to ensure the integrity of data.

.NET signature generation

First you need to use certificate with included private key – PKCS#12 certificates.

Flags as PersistKeySet and Exportable have to be included to certificate loading, as without them you are not able to obtain private key for signatures from the keystore. PersistKeySet flag ensures that while importing certificate from .pfx or .p12 file, you import private key with it. Exportable flag means that you can export imported keys (mainly private key).

As mentioned in this comment we need a little workaround to force crypto provider to use SHA256 hashes.

public byte[] signData(byte[] dataToSign)
{
  X509Certificate2 cert = new X509Certificate2(PATH_TO_CERTIFICATE, PASSWORD, X509KeyStorageFlags.PersistKeySet | X509KeyStorageFlags.Exportable);
  RSACryptoServiceProvider privateKey = (RSACryptoServiceProvider)cert.PrivateKey;
 
  RSACryptoServiceProvider rsaClear = new RSACryptoServiceProvider();
  // Export RSA parameters from 'privateKey' and import them into 'rsaClear'
  // Workaround to force RSACryptoServiceProvider use SHA256 hash
  rsaClear.ImportParameters(privateKey.ExportParameters(true));
  return rsaClear.SignData(dataToSign, "SHA256");
}

Java signature generation

Straight-forward example presented internet-wide.

/**
* data       data to be signed
* privateKey private key imported from key pair storage
*/
public byte[] signData(byte[] data, PrivateKey privateKey) {
  try {
    Signature signature = Signature.getInstance(signatureType);
    signature.initSign(privateKey);
    signature.update(data);
    return signature.sign();
  } catch (NoSuchAlgorithmException | SignatureException | InvalidKeyException e) {
    throw new SignatureException(“Could not create signature”, e);
    return null;
  }
}

.NET signature verification

First thing to mention is, that you shouldn’t create X509Certificate2 object directly from byte arrays (5th tip in this article). Depending on use of your protocol it might result into stalling and slowing whole protocol, what you definitely don’t want. But again, for simplicity I’m creating object from byte array directly. Mitigation of mentioned problem can be seen in linked article.

/**
* signature    signature of received data
* receivedData data received from counterpart. These data were also signed by counterpart to ensure integrity
*/
public bool verifySignature(byte[] signature, byte[] receivedData)
{
  X509Certificate2 cert = new X509Certificate2(PATH_TO_CERTIFICATE);

  RSACryptoServiceProvider publicKey = (RSACryptoServiceProvider)cert.PublicKey.Key;
  return publicKey.VerifyData(receivedData, "SHA256", signature);
}

Java signature verification

Straight-forward example presented internet-wide.

/**
* receivedData data received from counterpart
* signature    signature of receivedData
* publicKey    public key associated with privateKey object from "Java signature generation"
*/
public boolean verifyData(byte[] receivedData, byte[] signature, PublicKey publicKey) {
  try {
    Signature signature = Signature.getInstance(“SHA256withRSA”);
    signature.initVerify(publicKey);
    signature.update(receivedData);
    return signature.verify(signature);
  } catch (NoSuchAlgorithmException | SignatureException | InvalidKeyException e) {
    throw new SignatureException(“Could not validate signature”, e);
    return false;
  }
}

 

Key pair generation

Both, key pair generation and key agreement use BouncyCastle library (BC) for the crypto operations. During key generation phase, you need to specify key agreement algorithm. In this project we used elliptic curve Diffie-Hellmann (“ECDH”) key agreement. Elliptic curves were used to generate key pair mainly due to their advantages (performance, key size).

.NET key pair generation

The only difference from standard key generation is encoding of generated public key in a way that Java will accept the key and is able to create PublicKey object.

private static AsymmetricCipherKeyPair generateKeypair()
{
  var keyPairGenerator = GeneratorUtilities.GetKeyPairGenerator("ECDH");
  var ellipticCurve = SecNamedCurves.GetByName(ELLIPTIC_CURVE_NAME);
  var parameters = new ECDomainParameters(ellipticCurve.Curve, ellipticCurve.G, ellipticCurve.N, ellipticCurve.H, ellipticCurve.GetSeed());
  keyPairGenerator.Init(new ECKeyGenerationParameters(parameters, new SecureRandom()));
  return keyPairGenerator.GenerateKeyPair();
}

Now you need to send public key to other peer. Unfortunately, it is necessary to format public key in following way, otherwise Java peer will not be able to correctly create PublicKey object and create shared secret.

AsymmetricCipherKeyPair keyPair = generateKeypair();
byte[] publicKeyData = SubjectPublicKeyInfoFactory.CreateSubjectPublicKeyInfo(keyPair.Public).GetDerEncoded();

Java key pair generation

public static KeyPair generateKeyPair() {
  try {
    Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
 
    ECNamedCurveParameterSpec ecNamedCurveParameterSpec = ECNamedCurveTable.getParameterSpec(ELLIPTIC_CURVE_NAME);
    KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("ECDH", PROVIDER);
    keyPairGenerator.initialize(ecNamedCurveParameterSpec);
    return keyPairGenerator.generateKeyPair();
  } catch (NoSuchAlgorithmException | InvalidAlgorithmParameterException | NoSuchProviderException e) {
    throw new EncryptionException("Could not generate key pair", e);
  }
}

 

Key agreement

.NET key agreement

At this step, you need to beware the issue in C# BoucyCastle library, which results into failed key agreement even if you have correct keys (correct means same data, but different indentation => 0x0059534F4654 is different from 0x59534F4654).

Problem is that BC library creates shared secret as BigInteger. BigIntegers trims all leading zeroes and at conversion from it to byte array these zeroes aren’t included into array.

Unfortunately until publication day issue wasn’t still fixed in BC and you need to check key lengths by yourselves.


private static byte[] DeriveKey(AsymmetricCipherKeyPair myKeyPair, byte[] otherPartyPublicKey)
{
  IBasicAgreement keyAgreement = AgreementUtilities.GetBasicAgreement(KEY_AGREEMENT_ALGORITHM);
  keyAgreement.Init(myKeyPair.Private);

  //check otherPartyPublicKey length

  //shared secret generation
  var fullKey = keyAgreement.CalculateAgreement(PublicKeyFactory.CreateKey(otherPartyPublicKey)).ToByteArrayUnsigned();  
  return fullKey;
}

Java key agreement

Straight-forward example presented internet-wide.

public SecretKey secretDerivation(PublicKey receivedPublicKey, PrivateKey myPrivateKey) {
  try {
    KeyAgreement keyAgreement = KeyAgreement.getInstance(KEY_AGREEMENT_ALGORITHM, PROVIDER);
    keyAgreement.init(privateKey);
    keyAgreement.doPhase(publicKey, true);
    return keyAgreement.generateSecret(“AES”);
  } catch (NoSuchAlgorithmException | InvalidKeyException | NoSuchProviderException e) {
    throw new EncryptionException("Could not generate secret", e);
  }
}

 

From these examples You might gain bad feeling as all examples are rather brute and messages aren’t sent/received in no standard way- only as byte arrays. Best way is to exchange protocol messages in more standardized way. By doing so, you start heavily relying on third party library.

More standardized version of protocol should use Cryptographic Message Syntax. We will have a closer look on this option in next blog article.

While choosing the right security layer for messaging via ZeroMQ, there were two main considerations: built-in security CurveZMQ or usage of more conventional TLS backend?

First option is ZeroMQ’s proprietary security protocol based on elliptic curve cryptography and Daniel’s Bernstein NaCl cryptographic library. CurveZMQ utilizes not only Bernstein’s Curve25519 elliptic curve, but also other ciphers, designed by him, as well- f.e. Salsa20 and poly1305. Other NaCl’s ciphers can be found on its web page under Public-key and Secret-key cryptography chapters.

From the beginning this option was really hot candidate to use, as according to papers about performance of Salsa20 and performance of Curve25519, NaCl crypto functions are faster than standard crypto functions. Also the fact that it was already included in ZeroMQ was big plus. Unfortunately NaCl library and its functions were created in years 2008-2010 and aren’t used widely as f.e. AES or SHA are, therefore aren’t matured enough as other libraries, what brings a little bit panic to our mind and ultimately was the cause of not using it.

Second option was the usage of TLS backend- TLSZMQ. This is a demo project of Ian Barber to secure ZeroMQ communication using OpenSSL library. Unfortunately TLS isn’t built into ZeroMQ, so for now this is the only option how to use TLS and ZeroMQ together 🙁

Implementations comparison

To find the best approach we had few requirements to check. Mainly it was the speed of messaging and library and crypto functions maturity, but other requirements such as support and liveliness of implementation, licensing, quality of code also weighted in decision.

From this set of metrics only 2 appeared to be ultimately decisive:

  • Library and crypto functions maturity
  • Speed of messaging

While the maturity of NaCl library was strong argument why not to use CurveZMQ, performance of NaCl mentioned before spoke strongly in favor of it.

To finally resolve which approach to use and determine the total overhead of each implementation I have created performance tests to find out if TLSZMQ could be used in our environment. The test suite for each implementation was designed to be a block of 20 separate tests running for 10 minutes each, to be sure of reliable test outcomes. Even the message sizes were chosen to reflect the size of messages to be really sent in environment.

total persec

At this point, results clearly spoke strongly in favor of CurveZMQ. On the other hand, there was a conflict with maturity of crypto functions used in CurveZMQ and use of single elliptic curve to generate keys. This could cause many problems, if the curve or other functions would be compromised in the future.

At this point I realized that tests were done with OpenSSL’s by default chosen cipher suite RSA-AES256-SHA384 (OpenSSL version 1.0.1p). There is nothing wrong with this cipher suite, but we have to admit that these algorithms were unnecessarily secure (understand with high overhead).

Yes, I know what I just said and that it sounds really bad, but we must also consider the environment of messaging (its encryption) and performance of the system as well. Only after thorough examination we should accommodate ciphers to real requirements and not just use cipher suites with unnecessarily high security (and overhead) to overkill it.

Therefore, tests were rerun with more appropriate (weaker but still strong) cipher suites. The requirement was to get as much close to CurveZMQ performance as possible:

  • elliptic curves used everywhere possible
    • key exchange [ECDH]
    • authentication [ECDSA]

Any EC could be used, not just one.

  • AES doesn’t have to use 256b key as 128b long keys are still considered pretty strong.

From few acceptable cipher suites, this one specific peaked out: ECDSA-AESGCM128-SHA256.

This new finding has shown that new cipher suite increased the messaging performance considerably (sometimes effectively doubling the speed) [see graphs].

From the results one can see, that TLSZMQ isn’t that much inferior to CurveZMQ and due to TLS’s maturity it is a better choice. The final argument is that test results of TLSZMQ were worse, but sufficient in YSoft SafeQ environment, so the choice of implementation was pretty straight-forward.

Truth is, that NaCl and Curve25519 are really fun (and much easier) to play with and I encourage you to at least have a look at them, but in the end TLS backend is more flexible in terms of crypto primitives (ciphers, key generation) and doesn’t bring that much security concerns. And that’s what we are looking for in secured ZeroMQ messaging.