The cloud deployment process typically consists of two basic phases:

  1. Infrastructure deployment
  2. Product deployment

During the infrastructure deployment, the external environment in which the product runs is prepared. First, the underlying networks are created and configured properly, including all necessary routing options. Then, the virtual machines and their network interfaces are created and configured properly. Also, all of the firewall and other security rules are applied.

After the external environment is prepared and the virtual machines are deployed, the product installation can take place. First, the internal environment of the virtual machine is inspected and modified accordingly. This includes configuration of the operating system, internal firewall settings and installation of the tools necessary for the product deployment. After the environment is prepared, the product itself is installed and configured with respect to the internal and external environment (e.g. configuration of the IP addresses of other system components, assigning of the available ports, …).

Components of the deployment

In general, there are three necessary components of the cloud deployment. Each one of these should be interchangeable without affecting the other components:

  1. Infrastructure deployment configuration
  2. Product deployment configuration
  3. Product versioning

Infrastructure deployment configuration

Infrastructure deployment configuration describes everything necessary for the deployment of the environment. This approach is called infrastructure as a code and it allows defining all parameters of the infrastructure in machine-readable configuration files. These can be later reused, modified and versioned to keep track of the infrastructure modifications. It also allows
easier management of the infrastructure, as all configurations are exactly defined and automated.

Product deployment configuration

Product deployment configuration represents the necessary steps for the successful deployment of the product (or a particular component) to a single machine. This can be seen as an automated installation process, ran and controlled remotely by the deployment tools. It also consists of the product configuration, as it is usually necessary to interconnect several components of the product spread across the underlying network.

Product versioning

Product versioning defines the exact versions of all components of the product which are to be deployed (e.g. in form of a manifest file). It is important to separate the versions of the components from the rest of the deployment configurations, as the deployment process should not depend on the particular version of the product. Note here that it is, of course, possible that future versions of the product will require different installation steps or different infrastructure setup. This separation of versions from the configuration aims to provide transparency only within a subset of versions with similar properties (e.g. minor versions within a single major release or snapshot versions between several minor releases).

Chef is an automation platform designed to help the deployment and provisioning
process during software development and in production. Chef can, in cooperation with other deployment tools, transform the whole product environment into 
infrastructure as a code.

DSL

Chef provides a custom DSL that lets its users define the whole environment as a set of resources, together forming recipes, which can be further grouped into cookbooks. The DSL is based on Ruby, which adds a level of flexibility by offering Ruby’s language constructs to help the development. A basic example of a resource is a file with a specified content:
file 'C:\app\app.config' do
    content "server_port = #{port}"
end
Upon executing, Chef will make sure there exists a defined file and has the correct content. If the file with the same content already exists, Chef will finish without updating the resource, letting developers know the environment has already been in a desired stated before the Chef run.

Provisioning

The resources have build-in validations ensuring only the changes in configurations are applied in an existing environment. This lets users execute recipes repeatedly with only minor adjustments and Chef will make the necessary changes in your environment, leaving the correctly defined resources untouched.
This is especially handy in a scenario when an environment is already deployed and developers keep updating the recipes with new resources and managing configurations of deployed components. Here, with correctly defined validations, the recipes will be executed on target machines repeatedly, always updating the environment without modifying the parts of the environment which are already up to date.
This behavior can be illustrated on the following example:
my_tool = maven 'tool.exe' do
    artifact_id     'tool'
    group_id        'com.ysoft'
    version         '1.0.0'
    dest            'C:\utils'
    packaging       'exe'
end

execute 'run tool.exe' do
    command "#{my_tool.dest}\\#{my_tool.name} > #{my_tool.dest}\\tool.output"
    not_if {::File.exist?(#{my_tool.dest}\\tool.output)}
end
In this example, the goal is to download an exe file and run it exactly once (only the first run of this recipe should update the environment). The maven resource internally validates, whether the given artifact has already been downloaded (there would already exist a file C:\utils\tool.exe).
The problem is with the execute resource, as it has no way of checking whether it has been run before, thus potentially executing more times. Users can, however, define restrictions themselves, in this case, the not_if attribute. It will prevent the resource to execute again, as it checks the existence of the tool output from previous runs.

Architecture

To enable environment provisioning, Chef operates in a client-server architecture with a pull-based model.
Chef server represents the storage of everything necessary for deployment and provisioning. It stores cookbooks, templates, data bags, policies and metadata describing each registered node.
Chef client is installed on every machine managed by Chef server. It is responsible for contacting Chef server and checking whether there are new configurations to be applied (hence the pull-based model).
ChefDK workstation is the machine from which the whole Chef infrastructure is operated. Here, the cookbooks are developed and Chef server is managed.
In this example, we can differentiate between the Chef infrastructure (blue) and the managed environment (green). The process of deployment and provisioning is as follows:
  1. A developer creates/modifies a cookbook and uploads it to the Chef server.
  2. Chef client requests the server for changes in the recipes.
  3. If there are changes to be made, Chef server notifies the client.
  4. The client initiates a Chef run with the new recipes.
Note here that in a typical Chef environment, Chef client is set to request the server for changes periodically, to automate the process of configuration propagation.

Serverless deployment

When only the deployment of the environment is necessary (e.g for a simple installation of a product where no provisioning is required), in an offline deployment or while testing, much of the operational overhead of Chef can be mitigated by leaving out the server completely.
Chef client (with additional tools from ChefDK) can operate in a local mode. In such case, everything necessary for the deployment, including the recipes, is stored on the Chef client, which will act as a dummy server for the duration of the Chef run.
 
Here, you can see the architecture of a serverless deployment. The process is as follows:
  1. Chef client deploys a dummy server and points it to cookbooks stored on the same machine.
  2. Chef client from now on acts as the client in the example above and requests the server for changes in the recipes.
  3. Chef Server notifies the client of the changes and a new Chef run is initiated.

Conclusion

Chef is a promising tool that has a potential to help us improve not only the products we offer, but also make the process of development and testing easier.
In combination with infrastructure deployment tools (like Terraform) we are currently researching, automatization of product deployment and provisioning can allow our developers to focus on important tasks instead of dealing with the deployment of testing environments or manually updating configuration files across multiple machines.