This blog post will introduce tool Terraform, which we use for deploying testing environments in YSoft. We will cover following topics:

  • What is Terraform?
  • How does it work?
  • Example of use

What is Terraform?

Terraform is a command line tool for building and changing infrastructure in a safe and efficient matter. It can manage resources on most of the popular service providers. In essence, Terraform is simply a tool that takes configuration files as input and generates an execution plan describing what needs to be done to reach the desired state. Do you need to add another server to your cluster? Just add another module to your configuration. Or redeploy your production environment in a matter of minutes? Than Terraform is the right tool.

How does it work?

Infrastructure as a code

Configuration files that define infrastructure are written using high-level configuration syntax. This basically means that blueprint of your production or testing infrastructure can be versioned and treated as you would normally treat any other code. In addition, since we are talking about code, the configuration can be shared and re-used.

Execution plan

Before every Terraform execution, there is planning step, where Terraform generates an execution plan. The execution plan will show you what will happen when you run an execution (when you apply the plan). This way you avoid surprises when you manipulate with your infrastructure.

Terraform state file

How does Terraform determine the current state of infrastructure? The answer is the state file. State file keeps the information about all the resources that were created by execution of the given configuration file. To assure that the information in state file is fresh and up to date, Terraform queries our provider for any changes of our infrastructure (and modifies state file accordingly), before running any operation, meaning: for every plan and apply, Terraform will perform synchronization of a state file with a provider.

Sometimes this behavior can be problematic, for example querying large infrastructures can take a non-trivial amount of time. In this scenarios, we can turn off the synchronizing, which means the cached state will be treated as the record of truth.

Below you can see the picture of the whole execution process.

Example of use

In our example, we will be working with the azure provider. The example configuration files can be used only for the azure provider (hence the configuration files for different providers may and will differ). In our example, it is also expected, that we have set up terraform on our machine and appropriate endpoints to provider beforehand.

Step 1: Write configuration file

The presented configuration file has no expectations regarding previously created resources and it can be executed on its own, without the need to create any resources in advance.

The configuration file that we will write describes following resources:

Now we create an empty directory on the machine where we have installed terraform and within we create a file with name main.tf. The contents of the main.tf  file:

provider "azurerm" {
  subscription_id = "..."
  client_id       = "..."
  client_secret   = "..."
  tenant_id       = "..."
}

resource "azurerm_resource_group" "test" {
  name     = "test-rg"
  location = "West US 2"
}

resource "azurerm_virtual_network" "test" {
  name                = "test-vn"
  address_space       = ["10.0.0.0/16"]
  location            = "West US 2"
  resource_group_name = "${azurerm_resource_group.test.name}"
}

resource "azurerm_subnet" "test" {
  name                 = "test-sbn"
  resource_group_name  = "${azurerm_resource_group.test.name}"
  virtual_network_name = "${azurerm_virtual_network.test.name}"
  address_prefix       = "10.0.2.0/24"
}

resource "azurerm_network_interface" "test" {
  name                = "test-nic"
  location            = "West US 2"
  resource_group_name = "${azurerm_resource_group.test.name}"

  ip_configuration {
    name                          = "testconfiguration1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "dynamic"
  }
}

resource "azurerm_virtual_machine" "test" {
  name                  = "test-vm"
  location              = "West US 2"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  network_interface_ids = ["${azurerm_network_interface.test.id}"]
  vm_size               = "Standard_DS1_v2"

  delete_os_disk_on_termination = true

  storage_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "16.04-LTS"
    version   = "latest"
  }

  storage_os_disk {
    name              = "myosdisk1"
    caching           = "ReadWrite"
    create_option     = "FromImage"
    managed_disk_type = "Standard_LRS"
  }

  os_profile {
    computer_name  = "hostname"
    admin_username = "testadmin"
    admin_password = "Password1234!"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }
}

 

Step 2: Planning the execution

Now we browse into a directory with our main.tf  file and we run command terraform init, which initializes various local settings and data that will be used by subsequent commands.

Secondly, we run command terraform plan, which will output the execution plan, describing which actions Terraform will take in order to change real infrastructure to match the configuration. The output format is similar to the diff format generated by tools such as Git. If terraform plan failed with an error, read the error message and fix the error that occurred. At this stage, it is likely to be a syntax error in the configuration.

Step 3: Applying the plan

If terraform plan ran successfully we are safe to execute terraform apply. Throughout the whole “apply” process terraform will inform us of progress. Once the terraform is done, our environment is ready and we can easily check by logging in to our virtual machine. Also, our directory now contains file terraform.tfstate file, which is state file that corresponds to our newly created infrastructure.

Conclusion

This example was only a very simple one to show how configuration file might look. Terraform offers more on top of that. Configurations can be packed into modules, self-contained packages that are managed as a group. This way we can create reusable parametrizable components and treat these pieces of infrastructure as a black box. Besides that, Terraform can also perform a provisioning of VM and much more.

Qt Installer Framework is a quite new framework which is currently still in development. The current version contains set of tools and utilities to create installers. The most significant feature is that the framework itself is multiplatform, it supports Windows, Mac OS X and Linux.

Great feature of QT Installer Framework is that it can download the required files from the server. That means it is not required to provide files with the installer. It works with the so called “repositories”. Thanks to this it is also able to update the files without having to download the installer again, because it creates maintenance tool which can update or uninstall the files. However the documentation is not perfect, it is missing a lot of details.

The framework is an open source project. It is built on top of the Qt (it requires static built of Qt).

The framework is really easy to use. For the basic features the user needs to know to work with XML, that is all. For advanced features the knowledge of javascript (QtScript) is required (C++ might be required for the most advanced features).

Usage

Whole installer creation process starts with creating the required folder structure.

Screen Shot 2015-01-28 at 09.16.11

There are two main folders. The config and the packages folder.

Configuration

The config folder contains the installer configuration file (config.xml) and images used in the installer. There are various things that can be configured – Supported Configuration Settings. This is the example what the configuration file could look like:

<?xml version="1.0" encoding="UTF-8"?>
<Installer>
 <Name>YSoft SafeQ Mac Client</Name>
 <Version>4.3.0</Version>
 <Title>YSoft SafeQ Mac Client Installer</Title>
 <Publisher>Y Soft</Publisher>
 <ProductUrl>http://www.ysoft.com</ProductUrl>
 <TargetDir>@HomeDir@/YSoft</TargetDir>
 <AllowSpaceInPath>true</AllowSpaceInPath>
 <InstallerApplicationIcon>ysoft_96</InstallerApplicationIcon>
 <InstallerWindowIcon>ysoft_96_32x32</InstallerWindowIcon>
 <Watermark>modern-wizard.bmp</Watermark>
 <WizardStyle>Aero</WizardStyle>
</Installer>

Packages

The second folder is the package folder. It contains all components that will be installed by the installer. Every component consists of two folders, the data and the meta folder.

The data folder contains all the files that will be installed on the target machine. Installer archives all data to the 7z format and extracts it on installation.

The meta folder contains configuration file(package.xml) for the component and the installation script that will be called when the component is loaded.

This is the example file of component configuration file

<?xml version="1.0" encoding="UTF-8"?>
<Package>
 <DisplayName>CUPS Backend</DisplayName>
 <Description>CUPS Backend</Description>
 <Version>4.3.0</Version>
 <ReleaseDate>2015-01-23</ReleaseDate>
 <Default>true</Default>
 <Script>installscript.js</Script>
 <ForcedInstallation>true</ForcedInstallation>
 <RequiresAdminRights>true</RequiresAdminRights>
</Package>

and here is the full list of possible configuration values – Summary of Package Information File Settings.

The next file is the installation script file (installscript.js). The script is called when the installer is executed and the component is loaded. The script can add new installer wizard pages, prompt user for custom path for the component etc. This is example of a script that extracts component to the /tmp folder and moves it to the Application. Then it adds new item (log out checkbox) to the final page of the installer (the page items or pages have to be designed in Qt designer).

function Component(){
	//Connect signals to functions
	component.loaded.connect(this, componentLoaded);
	installer.finishButtonClicked.connect(this, finishClicked);
}

Component.prototype.createOperationsForArchive = function(archive){
	//Extract and move .app file
	component.addOperation("Extract", archive, "/tmp");
	component.addElevatedOperation("Execute","mv", "/tmp/YSoft\ SafeQ\ Client.app", "/Applications", 
								   "UNDOEXECUTE", "rm", "-rf", "/Applications/YSoft\ SafeQ\ Client.app");
}

componentLoaded = function(){
	//If this is installer load checkbox from .ui file
	if(installer.isInstaller()){
		installer.addWizardPageItem(component, "LogOutCheckBoxForm", QInstaller.InstallationFinished);	
	}
}

finishClicked = function(){
	if(!component.installed)
		return;
	//If the installation was succesful, let the user log out
	if(installer.isInstaller() && installer.status == QInstaller.Success){
		var isLogOutChecked = component.userInterface("LogOutCheckBoxForm").LogOut.checked;
        if (isLogOutChecked) {
			//Todo - logout
		}
	}
}

Here is the documentation for the Component scripting.

The installer is created by executing the Qt Installer tool

binarycreator -c config/config.xml -p packages installer

where -p is path to the packages folder and -c is path to the config.xml file. The last part is the name of the installer. 

This is the look of the final installer on Mac OS X:

Screen Shot 2015-01-28 at 09.48.00