Yes, you heard right! Developer testing. It means testing done by developers! And yes, I’m talking about the confirmation testing, which is known as “The changelog” in our R&D department. The result – improvement from 50 % to 95 % of tickets closed at the end of a sprint and all sprint goals completed on time 4 sprints in a row! [1]

Percentage of work planned and really done over sprints

Percentage of work planned and really done over sprints

Our development team has recently been trying very hard to shorten the development cycle of features and fixes. One of the biggest delays we have identified was caused by the tickets waiting in the “To Test” state. It means that the implementation part has been completed and is waiting for a QA Engineer to confirm it’s functionality by testing it. As I was the only tester for 7 developers on the team, the tickets with lower severity had to simply wait, often more than a week. Moreover, the testing activities were concentrated at the end of a sprint. A ticket reopened too late can easily be a cause of a failure in reaching team’s sprint goals.

Generally, it is strongly recommended by the literature to not let developers test their own work. The common reasoning is, that everybody is in love with their creations and would not like to damage them. Also, some defects are approach-related and thus cannot be discovered by the same person with the same approach. Moreover, test planning, test case writing and test data definition need special skills, which developers generally do not possess. Our mindset was changed by our CTO, who saw this as an opportunity to improve efficiency of the development.

In our team, we kept all of the aforementioned risks in mind and tailored the process to negate all of them. We have tried several versions of the process. In a short time we found the most important thing – that we need to differentiate tasks (new development) and defects (bug fixes). You’ll later see why.

Generally, it is much easier to write test cases for known functionality. This is the case for the defects which only fix or slightly modify an already tested feature. Experiences with an existing (sub-)system, where a feature is only updated and testing approaches are well known, help the QA engineer to define a set of all necessary test cases, edge-case scenarios and also expected visual aspects. Therefore, based on the well-defined test cases, a developer should be able to develop the fix and test it in a way, which will ensure it meets quality requirements. Later, a QA engineer interviews the developer to find out his confidence level about the testing and also asks several direct questions about the most critical areas. Based on this information and on the experiences with the particular developer[2], the QA engineer then decides which defects need to be retested and which can be closed right away.

On the other hand, tasks usually represent development of new features. Since the people writing the code in Y Soft are not “coders” but developers, they have to propose solutions for many low-level issues throughout the development process. Therefore, it frequently happens, that some aspects and limitations are discovered later in the sprint, making it very challenging for a QA engineer to define a complete set of test cases in advance. Also, without a proper hands-on the final work, it is also very difficult to define requirements for visual aspects and to judge user friendliness. Therefore, tasks have to always be retested by a QA engineer.

Nevertheless, defining at least some of these tests brings certain benefits that are also common for the defects:

  • A QA engineer can discover inconsistencies between his/her understanding compared to the developer’s understanding of the work to be done. It is generally better to find them before significant time has been spent on development of something undesired and reworking it later.
  • A partial test suite can be very helpful to the developer during the development, as it can be used as a checklist to cover many possible scenarios.
  • Some other scenarios can be discovered during the development and can be added to the test suite. These test cases would otherwise probably not exist.
  • As the developer performs tests himself, many issues are found and fixed in a much shorter time and with less effort than it would when they are found and reported back by the QA department (several days later). This way we can assure higher quality of the developed product in the development phase.
  • Based on the current state of work and human resources, the team can flexibly agree on the extent to which developers will test their work. Either they do extensive tests to help QA engineers, or they perform only a basic set of tests in order to move on to the next development task sooner.

These result in:

  • shorter development cycles (Open to Closed status)
  • less reopened tickets
  • better understanding of the whole solution for all members of the team

The process itself consists of the following steps:

  1. When the sprint backlog is defined:
    1. A QA Engineer creates a set of test cases for each of the tickets (pre-)planned for the sprint;
    2. The test cases are defined in a subtask of each ticket. The subtasks are named “Testing of [ticketID]”;
  2. At the sprint planning meeting:
    1. The QA engineer consults on the technical details of the solution and proposed tests with other team members and the current product manager;
    2. The effort for each ticket is estimated including the testing part;
  3. Developers have to test their work and switch the ticket to status “To Test”:
    1. All defects are tested in a standardized testing environment (ideally prepared by a QA engineer);
    2. Tasks can be tested in a development environment (e.g. running on a developer’s machine built by a development tool);
  4. When a ticket has the “To Test” status, the QA engineer:
    1. Evaluates which defects need to be tested again and which do not;
    2. Retests all tasks;

It is important to note that the aforementioned benefits are only subjectively observed by the members of the team, as none of them has been measured in any systematic way. Doing that would require returning back to the old way of work, in order to make the necessary measurements. Since members of the team are satisfied with the new process , there is no need or motivation to revert back to the old method. A change from less than 50 % to 95 % of closed tickets from the end of Sprint 17 to the end of Sprint 18 and the 100 % fulfillment of sprint goals in the last four sprints presents a sufficiently strong argument to try this process in other teams.

[1] The orange line represents the percentage of developers’ time was planned for a sprint. The time is estimated for each ticket. The green line represents the percentage of that planned estimated time, for which the tickets were closed at the end of the sprint. The first attempt to use developer testing was in Sprint 18. In Sprint 22, we resumed the process. The trend of about 70 % of work planned and more than 90 % finished remains to date.

[2] In order to gain experiences with the developers, there has to be a period of several sprints, where every defect is retested. The QA engineer needs to measure the ratio of closed and reopened defects per developer. During this period the QA engineer can also find out, whether he/she is able to define all necessary test cases beforehand.

The process of forming the team, it’s stages, team roles which should be represented in a team and many interesting issues about team coordination, cooperation, communication and productivity. All of that was discussed during agile community meeting, which Y Soft was hosting last week in Brno office. In this text I want to highlight most interesting issues we have discussed and found ways hoe to handle it.

agilia_01

Introduction

Short introduction by Michal Vallo was about Tuckman’s stages of group development as life-cycle of team construction and Belbin Team Roles as guide to what characters should be included in each team. Later, each participant propose one topic to be discussed, then everybody vote for the one which he likes. I will present three of them which were discussed the most.

agilia_03

Need of Junior/Senior positions

Team consists of different roles and people. One of way how to differentiate team members is usage of Junior and Senior labels. There were many interesting inputs to this topic.

One of the opinion reflected in real company policy was to remove these formal roles. Reasons to do such a thing are more. There is no race in a team to reach senior position. If starting new project and forming new teams, you will prevent situation with low interest in junior positions in a team. But you still have to differ skills and knowledge, reward courage and motivation. It’s should by done by assigning responsibilities, competences, work tasks with different difficulty. And do not forget about money, company benefits and other materialistic stuff.

On the other hand, especially bigger companies has defined career paths, so they need to differ positions on formal level. It also works if these career paths are well defined and they are not limited by project budget or head count calculated on HR. Because there was also one case discussed, where team is limited by budget and even they had chance to hire great senior guy, they couldn’t because of restrictions on number of senior positions.

Low performance of one member in a team

An interesting discussion was about dealing with low performance of one team member, who is lowering teams overall performance. Well working team should identify such a member itself without intervention from higher positions. Move to another team, relocate to another department or fire him are another “simple solutions”.

But what if this member is e.g. external contractor and you cannot get rid of him like this? Again more working solutions were discussed, but think about two types of people. Ones who will get better if you will push them and ones who are too lazy or incompetent for this work type and load. For the perspective ones, motivation is the key. If direct motivation is not successful, try to do it other way. Make them work in pairs with team members which are more productive, make them responsible for something and force them to present work results to the team e.g. on daily stand-ups. They will be shamed maybe, but it should show the difference. For the lazy ones, try to assign them work tasks which are easier, maybe not popular, or just ones that are the cheapest in case they will screw it.

But do not forget. That it’s not only about performance. Team member with lower performance can have e.g. greater social influence for the team, so it’s affecting the team performance also.

Team motivation

How to motivate team and its members to perform better? In the beginning I want to explain difference between motivation and stimulation, because both approaches were discussed. Motivation refers to the will to act, work, create, etc. On the other hand, stimulation deals with encouraging on an initial effort or in supporting an already existing action. Most of discussed motivation practices were:

  • Allowing self-development of team members, based on their will and needs of the team
  • Giving challenging and interesting task to team members
  • Take inputs from team members seriously, try to give valuable feedback
  • Trust the team and let them make decisions, give responsibilities to each team member

If there are some activities in the team like supporting the product or bugfixing, which are not as popular as e.g. new development, distribute these type of tasks over the whole team by rotating engineering role every iteration or distribute it evenly across all team members.

agilia_02

It looked like everybody has took something from this meeting, and I am looking forward to another interesting topics to be discussed in further community meetings.