In latest project we came across the problem of securing communication, where peers don’t share backend language. Our goal was to securely generate shared secret between Java server and .NET client. Each language supports all features needed for the key agreement, but these features aren’t always compatible, due to different encoding or data representation.

This article should be quick and easy solution how to agree on shared secret between such server and client with hand-on examples. Each subchapter contains concrete example on how to send, receive and process data needed for respected operation. Authentication and signature scheme are similar to guides published internet-wide, the main topic- key agreement is presented with our insights and comments on how to make it work.

The reason I have included all the code snippets even if they are “straight from internet” is that I wanted to simply group all these schemes to one place, so that you don’t have to find examples on Oracle, CodeRanch, StrackOverflow, MSDN Microsoft documentation, … Honestly it was a little bit frustrating for me, so there you go 🙂

If you are interested just in key agreement, feel free to skip right to the key agreement example, as I would like to discuss authentication and signature schemes first in order to prevent attacks on anonymous key agreement.

Please beware of incautious use of the published code. Your protocol might require different configuration and by simple copy-pasting you may introduce some security issues or errors to it.

Certificate based authentication

Consider each peer to have certificate signed by some root CA.

In my examples I’ll be working mostly with byte arrays as it is most universal way to represent data.

In most cases whole certificate chain is transferred to other side. Using one certificate (self-signed) is just simplification of process below.

.NET authentication

Java peer can transform certificate into byte array using Certificate.getEncoded() method. Byte array is then sent to counterpart.

.NET peer parses certificate chain data and stores parsed certificates into List<byte[]> to ease processing.

Code below shows the authentication process. Process differs from standard validation in a way, that we are sending only certificate chain. From the chain peers certificate is extracted. From the rest of received certificate chain we create subchain and check if we were able to create complete chain from peer to Certificate Authority.

For simplicity revocation checks and certificate usage field (OID of the field – 2.5.29.15) are being ignored. In real environment you should definitely check this list and field, but in this article it is not really necessary.

/**
* receivedCertificateChain previously transformed byte array containing whole certificate chain. This chain is transformed
*                          into list of separate certificates from chain 
*/
public bool certificateChainValidation(List<byte[]> receivedCertificateChain)
{
  var otherSideCertificate = new X509Certificate2(receivedCertificateChain[0]);
  var chain = new X509Chain { ChainPolicy = { RevocationMode = X509RevocationMode.NoCheck, VerificationFlags = X509VerificationFlags.IgnoreWrongUsage } };
  chain.ChainPolicy.ExtraStore.AddRange(receivedCertificateChain.Skip(1).Select(c => new X509Certificate2(c)).ToArray());

  return chain.Build(otherSideCertificate));
}

Root CA certificate must be trusted by your system to be successfully validated.

Java authentication

.NET peer can transform each certificate into byte array using X509Certificate2.RawData property and send it to counterpart.

First, Java peer needs to define validation parameters and Trust anchor. In this case it’s root CA.

After that, you validate received certificate chain. If method validate(CertPath, PKIXParameters) is processed without any exception, you can consider received certificate chain correctly validated.

/**
* certificatePath received data of certificate chain from counterpart 
*/
public boolean certificateChainValidation(byte[] certificatePath) {
  try {
    //generate Certificate Path from obtained byte array
    CertificateFactory cf = CertificateFactory.getInstance(“X.509”);
    CertPath received = cf.generateCertPath(new ByteArrayInputStream(certificatePath));

    //get all trusted entities from dedicated truststore
    Set<TrustAnchor> trustedCAs = getAnchors();

    //set the list of trusted entities and turn off the revocation check
    //revocation is by default set to true, so if you don't turn it off explicitly, exceptions will bloom
    PKIXParameters params = new PKIXParameters(trustedCAs);
    params.setRevocationEnabled(false);

    CertPathValidator cpv = CertPathValidator.getInstance("PKIX");
    //if no exception is thrown during validation process, validation is successful
    cpv.validate(received, params);
  } catch (CertificateException | NoSuchAlgorithmException | InvalidAlgorithmParameterException | CertPathValidatorException e ) {
    throw new CertificateException("Could not validate certificate", e);
    return false;
  }
  return true;
}

getAnchors() is a method which simply loads all entries from dedicated truststore and returns them as Set<TrustAnchors>.

Signatures

Without correct signature scheme whole authentication scheme would be compromised, as any attacker would be able to replay intercepted certificate chain. Therefore while sending outgoing certificate chain, each peer has to include signature to it. It is sufficient for such signature to be reasonably short (resp. long) nonce. This nonce should be freshly generated by the communication counterpart and can be sent before authentication attempt (or as an authentication challenge). Nonce could be any-random value or  static counter, but must be definitely unique.

During the key agreement phase, signatures are used to sign public keys that are exchanged between peers to ensure the integrity of data.

.NET signature generation

First you need to use certificate with included private key – PKCS#12 certificates.

Flags as PersistKeySet and Exportable have to be included to certificate loading, as without them you are not able to obtain private key for signatures from the keystore. PersistKeySet flag ensures that while importing certificate from .pfx or .p12 file, you import private key with it. Exportable flag means that you can export imported keys (mainly private key).

As mentioned in this comment we need a little workaround to force crypto provider to use SHA256 hashes.

public byte[] signData(byte[] dataToSign)
{
  X509Certificate2 cert = new X509Certificate2(PATH_TO_CERTIFICATE, PASSWORD, X509KeyStorageFlags.PersistKeySet | X509KeyStorageFlags.Exportable);
  RSACryptoServiceProvider privateKey = (RSACryptoServiceProvider)cert.PrivateKey;
 
  RSACryptoServiceProvider rsaClear = new RSACryptoServiceProvider();
  // Export RSA parameters from 'privateKey' and import them into 'rsaClear'
  // Workaround to force RSACryptoServiceProvider use SHA256 hash
  rsaClear.ImportParameters(privateKey.ExportParameters(true));
  return rsaClear.SignData(dataToSign, "SHA256");
}

Java signature generation

Straight-forward example presented internet-wide.

/**
* data       data to be signed
* privateKey private key imported from key pair storage
*/
public byte[] signData(byte[] data, PrivateKey privateKey) {
  try {
    Signature signature = Signature.getInstance(signatureType);
    signature.initSign(privateKey);
    signature.update(data);
    return signature.sign();
  } catch (NoSuchAlgorithmException | SignatureException | InvalidKeyException e) {
    throw new SignatureException(“Could not create signature”, e);
    return null;
  }
}

.NET signature verification

First thing to mention is, that you shouldn’t create X509Certificate2 object directly from byte arrays (5th tip in this article). Depending on use of your protocol it might result into stalling and slowing whole protocol, what you definitely don’t want. But again, for simplicity I’m creating object from byte array directly. Mitigation of mentioned problem can be seen in linked article.

/**
* signature    signature of received data
* receivedData data received from counterpart. These data were also signed by counterpart to ensure integrity
*/
public bool verifySignature(byte[] signature, byte[] receivedData)
{
  X509Certificate2 cert = new X509Certificate2(PATH_TO_CERTIFICATE);

  RSACryptoServiceProvider publicKey = (RSACryptoServiceProvider)cert.PublicKey.Key;
  return publicKey.VerifyData(receivedData, "SHA256", signature);
}

Java signature verification

Straight-forward example presented internet-wide.

/**
* receivedData data received from counterpart
* signature    signature of receivedData
* publicKey    public key associated with privateKey object from "Java signature generation"
*/
public boolean verifyData(byte[] receivedData, byte[] signature, PublicKey publicKey) {
  try {
    Signature signature = Signature.getInstance(“SHA256withRSA”);
    signature.initVerify(publicKey);
    signature.update(receivedData);
    return signature.verify(signature);
  } catch (NoSuchAlgorithmException | SignatureException | InvalidKeyException e) {
    throw new SignatureException(“Could not validate signature”, e);
    return false;
  }
}

 

Key pair generation

Both, key pair generation and key agreement use BouncyCastle library (BC) for the crypto operations. During key generation phase, you need to specify key agreement algorithm. In this project we used elliptic curve Diffie-Hellmann (“ECDH”) key agreement. Elliptic curves were used to generate key pair mainly due to their advantages (performance, key size).

.NET key pair generation

The only difference from standard key generation is encoding of generated public key in a way that Java will accept the key and is able to create PublicKey object.

private static AsymmetricCipherKeyPair generateKeypair()
{
  var keyPairGenerator = GeneratorUtilities.GetKeyPairGenerator("ECDH");
  var ellipticCurve = SecNamedCurves.GetByName(ELLIPTIC_CURVE_NAME);
  var parameters = new ECDomainParameters(ellipticCurve.Curve, ellipticCurve.G, ellipticCurve.N, ellipticCurve.H, ellipticCurve.GetSeed());
  keyPairGenerator.Init(new ECKeyGenerationParameters(parameters, new SecureRandom()));
  return keyPairGenerator.GenerateKeyPair();
}

Now you need to send public key to other peer. Unfortunately, it is necessary to format public key in following way, otherwise Java peer will not be able to correctly create PublicKey object and create shared secret.

AsymmetricCipherKeyPair keyPair = generateKeypair();
byte[] publicKeyData = SubjectPublicKeyInfoFactory.CreateSubjectPublicKeyInfo(keyPair.Public).GetDerEncoded();

Java key pair generation

public static KeyPair generateKeyPair() {
  try {
    Security.addProvider(new org.bouncycastle.jce.provider.BouncyCastleProvider());
 
    ECNamedCurveParameterSpec ecNamedCurveParameterSpec = ECNamedCurveTable.getParameterSpec(ELLIPTIC_CURVE_NAME);
    KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("ECDH", PROVIDER);
    keyPairGenerator.initialize(ecNamedCurveParameterSpec);
    return keyPairGenerator.generateKeyPair();
  } catch (NoSuchAlgorithmException | InvalidAlgorithmParameterException | NoSuchProviderException e) {
    throw new EncryptionException("Could not generate key pair", e);
  }
}

 

Key agreement

.NET key agreement

At this step, you need to beware the issue in C# BoucyCastle library, which results into failed key agreement even if you have correct keys (correct means same data, but different indentation => 0x0059534F4654 is different from 0x59534F4654).

Problem is that BC library creates shared secret as BigInteger. BigIntegers trims all leading zeroes and at conversion from it to byte array these zeroes aren’t included into array.

Unfortunately until publication day issue wasn’t still fixed in BC and you need to check key lengths by yourselves.


private static byte[] DeriveKey(AsymmetricCipherKeyPair myKeyPair, byte[] otherPartyPublicKey)
{
  IBasicAgreement keyAgreement = AgreementUtilities.GetBasicAgreement(KEY_AGREEMENT_ALGORITHM);
  keyAgreement.Init(myKeyPair.Private);

  //check otherPartyPublicKey length

  //shared secret generation
  var fullKey = keyAgreement.CalculateAgreement(PublicKeyFactory.CreateKey(otherPartyPublicKey)).ToByteArrayUnsigned();  
  return fullKey;
}

Java key agreement

Straight-forward example presented internet-wide.

public SecretKey secretDerivation(PublicKey receivedPublicKey, PrivateKey myPrivateKey) {
  try {
    KeyAgreement keyAgreement = KeyAgreement.getInstance(KEY_AGREEMENT_ALGORITHM, PROVIDER);
    keyAgreement.init(privateKey);
    keyAgreement.doPhase(publicKey, true);
    return keyAgreement.generateSecret(“AES”);
  } catch (NoSuchAlgorithmException | InvalidKeyException | NoSuchProviderException e) {
    throw new EncryptionException("Could not generate secret", e);
  }
}

 

From these examples You might gain bad feeling as all examples are rather brute and messages aren’t sent/received in no standard way- only as byte arrays. Best way is to exchange protocol messages in more standardized way. By doing so, you start heavily relying on third party library.

More standardized version of protocol should use Cryptographic Message Syntax. We will have a closer look on this option in next blog article.

I am trying to setup better go development environment and decided to give vim-go a try (which also resulted in me replacing Vundle with Pathogen, which is much more straightforward). Installing everything was a breeze and I only encountered a problem when I tried to make tagbar work, because tagbar does not work with BSD ctags and requires Exuberant ctags 5.5.

The simplest way how to install exuberant ctags is with brew.

ondra@nb218 ~/Downloads/ctags-5.8
$ brew install ctags
Warning: You are using OS X 10.11.
We do not provide support for this pre-release version.
You may encounter build failures or other breakage.

However brew is still having problems running on 10.11 and many packages fail to build, ctags being on exception. So let’s see how we can deploy ctags into /usr/local without stepping on brew’s toys.

First of all, we need to determine the prefix to use via brew diy:

ondra@nb218 ~/Downloads/ctags-5.8
$ brew diy --name=ctags --version=5.8
--prefix=/usr/local/Cellar/ctags/5.8

Now you can use brew fetch to download the source for ctags or just download it from sourceforge. To get the source using brew, use:

ondra@nb218 ~/Downloads/ctags-5.8
$ brew fetch --build-from-source ctags
==> Downloading https://downloads.sourceforge.net/ctags/ctags-5.8.tar.gz
Already downloaded: /Library/Caches/Homebrew/ctags-5.8.tar.gz
SHA1: 482da1ecd182ab39bbdc09f2f02c9fba8cd20030
SHA256: 0e44b45dcabe969e0bbbb11e30c246f81abe5d32012db37395eb57d66e9e99c7

==> Downloading https://gist.githubusercontent.com/naegelejd/9a0f3af61954ae5a77e7/raw/16d981a3d99628994ef0f73848b6beffc7
Already downloaded: /Library/Caches/Homebrew/ctags--patch-26d196a75fa73aae6a9041c1cb91aca2ad9d9c1de8192fce8cdc60e4aaadbcbb
SHA1: 24c96829dfdc58b215bfccf5445a409efba1ffe5
SHA256: 26d196a75fa73aae6a9041c1cb91aca2ad9d9c1de8192fce8cdc60e4aaadbcbb

Anyhow, when you have the source extracted, run configure && make with the prefix from brew diy:

ondra@nb218 ~/Downloads/ctags-5.8
$ ./configure --prefix=/usr/local/Cellar/ctags/5.8
Exuberant Ctags, version 5.8
Darwin 15.0.0 Darwin Kernel Version 15.0.0: Sun Jul 26 19:48:55 PDT 2015; root:xnu-3247.1.78~15/RELEASE_X86_64 x86_64
checking whether to install link to etags... no

And link ctags via brew using brew link:

ondra@nb218 ~/Downloads/ctags-5.8
$ brew link ctags
Linking /usr/local/Cellar/ctags/5.8... 2 symlinks created

And you are done. When building is fixed for brew under 10.11 simply unlink ctags and use brew install as usual.

Sometimes you may need to setup Continuous Integration server to build VS solutions without installing VisualStudio there. But MSBuild is not able to build all solution, you’ve created on your local machine.

The first error you can start getting will be something like:

The imported project ” C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\WCF\ Microsoft.VisualStudio.ServiceModel.targets” was not found.

Solution is quite simple:

  • Copy targets from your local machine to MSBuild folder on CI server.

Unfortunately this doesn’t solve all problems. MSBuild will keep throwing exceptions. This time it will be about missing dll:

Could not load file or assembly ‘Microsoft.VisualStudio.ServiceModel.Core, Version=11.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35’ or one of its dependencies.

This time solution will require few more steps:

  • Find the missing dll on your local machine and copy it to the same location on your CI server. It can look like “c:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\Microsoft.VisualStudio.ServiceModel.dll”.
  • Install this dll into GAC: open “Developer Command Prompt for VS2013”, go into directory with dll and register it to the GAC using command “gacutil.exe -i Microsoft.VisualStudio.ServiceModel.Core.dll”

gac

When all these steps are done, MSBuild will be able to build your solution successfully.

It’s quite easy to install Docker on Windows. Just download installer and follow instructions.

You might encounter following error message when starting Docker image: VT-X is not available.

hyperv-docker-vbox-problem

The cause of this failure is enabled Hyper-V Windows Feature. We described this issue in previous article.

Here is the quick fix.

Search for “windows feature” and open “Turn Windows features on or off
turn-off-windows-feature

Search for “Hyper-V” in Windows features and uncheck it.

hyper-v-disableClick OK and reboot the machine.

Update: add information about IntelliJ Idea 14

Java is no longer part of Mac OS default installation. When you want to start Idea on new Mac OS Yosemite you’ll get this nice message:

yosemite-java6

You have two options. Install legacy Java 6 from Apple or install new Java 8 from Oracle.

IntelliJ Idea 13

In case of Java 8, just open file “/Applications/IntelliJ IDEA 13.app/Contents/Info.plist” and change JVMVersion from 1.6* to 1.8*:

sudo vim "/Applications/IntelliJ IDEA 13.app/Contents/Info.plist"

IntelliJ Idea 14

Open file “/Applications/IntelliJ IDEA 14.app/Contents/Info.plist” and change JVMVersion from 1.6* to 1.8*:

sudo vim "/Applications/IntelliJ IDEA 14.app/Contents/Info.plist"

intellij-jvm-1.8

Click IntelliJ Idea icon and enjoy your IDE.

IntelliJ_Idea_on_MacOS_X_Yosemite_with_Java_8

You can find more info at JetBrains Support forum.

Do you use git? Then you probably know the basic commands like git pull, git merge and git rebase. These are pretty common, but also complex. Over the time, I have adopted few simple rules which help me to use them effectively.

Git pull considered harmful

You have probably noticed that sometimes your git pull generates automatic merge commits. They do not hold any useful information and if your repo is busy, your history can be literally flooded.

I like to keep my history linear and simple. That is why I recommend to set the default git pull configuration to fast-forward only (linear type of merge which does not create any commits).

With Git 2.0 and newer, you can just update your settings:

git config --global pull.ff only

Older versions do not have this option, but you can use an alias instead:

git config --global alias.up '!git remote update -p; git merge --ff-only @{u}'

@{u}” is a shortcut which reffers upstream of the tracked branch.

Usage:

git up

Rebasing non-linear changes

When the fast-forward merge is not possible, the default git pull behavior would be a three-way merge. But in order to keep the history clean, rebase can be used.

You might have heard, that rebase is evil (git rebase hell), but as long as your rebased commits are only local, you should be safe.

Again, I recommend an alias:

git config --global alias.upr '!git remote update -p; git rebase -p @{u}'

Usage:

git upr

The git pull --rebase does a similar thing, but it currently doesn’t have an option to rebase merge commits, so you can end up with a slightly different history. Git rebase -p will try to replay merge commits and preserve your history.

Clean up your commits before publishing

Before you push your commits to the repository, it’s good to revise them.  You can run the git interactive rebase to squash your commits or modify commit messages.

An alias for the interactive rebase of unpushed commits:

git config --global alias.ri '!git rebase -i @{u}'

Usage:

git ri

The goal is to publish only clean and relevant commits, no experiments or fixing typos. Therefore I usually do the git rebase and push at the end of my programming sessions (typically at the end of the day), so I see all the commits in one place.

Keep in mind that interactive rebase does not preserve merge commits (combining interactive rebase and preserve merge is not recommended), so you might want to do it before you merge a new branch.

Three-way merge between branches

Three-way merge is a non-linear merge with conflict resolution. It should be always used when merging non-local branches:

  • Merge of feature branch to the master
  • When you are working on a feature in a branch and you would like to merge commits from master.

This type of merge generates the merge commit, which serves as a source of information about the merge (branches, commits, time and date, responsible user).

How to force a three-way merge:

git merge --no-ff the_other_branch

Replace the “the_other_branch” with a name of  the source branch for merge to your current branch.

Summary

  • Disable automatic three-way merge with git pull.
  • If you cannot pull with a fast-forward merge and your commits are only local, run the rebase with preserve merges enabled.
  • Use three-way merge for merging between branches or when updating you feature branch.
  • Before push, revise your commits and clean them with interactive rebase if needed.

Avoid

Never rebase any pushed or pulled changes (not branches, nor commits).

Want to know more?

It’s easy to create vertical text block selection in IntelliJ Idea.

Click Mouse wheel and drag.

Mouse wheel click is sometimes mapped to a function in operating system and selection won’t work.

There is alternative shortcut. Press ALT and drag cursor (by left click).

You’ll select vertical text block.

Newer versions of IntelliJ Idea have support for multiple cursors. Each line of selection will contain one cursor.

Common feature in many application is possibility to zoom text content by Ctrl+Mouse Wheel.

IntelliJ Idea has this feature, but it’s disabled by default.

You can enable zoom in few steps. Open Settings (Ctrl+Alt+S), search for “zoom“, select Editor.

idea-enable-mouse-zoom

Check the option “Change font size (Zoom) with Ctrl+Mouse Wheel“. Click OK and enjoy the zoom feature 🙂

VirtualBox is able to host 64bit guest OS on Windows. That’s useful for testing various platforms.

The trouble begins when you turn on Hyper-V Windows Feature. It might happen by installing some software or update. When Hyper-V is enabled then it’s not possible to create new guests with 64bit OS and it’s not possible to boot anything with 64bit kernel created before this feature was turned on.

In order to make 64bit guests working again you’ll need to turn off Hyper-V feature.

Search for “windows feature” and open “Turn Windows features on or off
turn-off-windows-feature

Search for “Hyper-V” in Windows features and uncheck it.

hyper-v-disableClick OK and reboot the machine. 64bit OS guests start working again.

Gradle is great tool for automation in organization. Software developers often stick just with command line or IDE integration. Non-programmers can use Gradle GUI feature.

Just type command (or store it in batch file):

gradle --gui

Gradle gui

You can select task and execute it. There is another handy feature: Favorites. Here you can store useful commands for your Gradle build.

Gradle GUI favorites