Automation of Burn-up and Burn-down charts using GScript and Entrance

I have always found that the burn-up and burn-down charts are very informative and fit to the iterative story-based development very well. Every project that I work on, I will try to figure out different ways to generate burn-up and burn-down charts.

Two months ago, I took the job on putting Platform on Sprints. After some consideration, I have decided to follow the setup that I have for the AF team, creating the stories in the form of JIRA issues. However, the chart generation that I had for AF team was still semi-manual, which means that it takes a couple of minutes to download, and a couple of minutes to update the stats every morning. The worst part is that when I get busy or sick, I will forget.

So my first action item was to figure out how to generate the same kind of charts with the push of a button. The idea seems to be easy enough:

  1. Figure out the search criteria to retrieve all the JIRA issues of the backlog
  2. Count the issues that are in different states
  3. Update the data with the counts, and check it into Perforce.
  4. Refresh the chart with the updated data

Number one and two were actually not that hard, because Guidewire GScript has a nice WebServices support. With a few tries, I was able to count the beans.

Here is an example of the data generated. I think you get the idea just looking at it.

Date,Day,Closed,Deferral Requested,In QA,Open Stories,Open New Features,Open Bugs

Number three took less time but a bit of research because Perforce Java library’s API is not exactly straightforward.

It took me a while to figure out how to do the last one. After looking into JFreeChart and Google Chart API, I eventually turned to my dear friend, Tod Landis, who is also my partner at Entrance, and he quickly drafted an entrance script for me. Based on it, I was able to write a template that can be used for all teams within a few hours.

  very light yellow  area,
  light yellow filled circles and line,
  very light orange  area,
  light orange filled circles and line,
  very light red  area,
  light red filled circles and line,
  very light blue area,
  light blue filled circles and line,
  very light gray area,
  dark gray filled circles and line,
  very light green area,
  dark green filled circles and line,
  TITLE "Sprint"
  TITLE X "Days"
  TITLE Y "Points"
  SCALE Y 0 100 0
  no sides
  `Open Bugs`,
  `Open Bugs`,
  `Open New Features`,
  `Open New Features`,
  `Open Stories`,
  `Open Stories`,
  `In QA`,
  `In QA`,
  `Deferral Requested`,
  `Deferral Requested`,
from report;

Please note this is the final PLOT script, there are other SQLs run before this to import the data into the MySQL database, sum up the data to produce a stacked chart, and even out the labels.

And I now have this chart generated automatically every morning with the help of a windows scheduler.

Phoenix First Two XP Sprints

Phoenix has finished its Sprint 5 and 6, which are the first two XP Sprints.

An XP team always goes through four stages, “forming, storming, norming, performing”. The first Sprint felt like a storming stage, where we are trying to figure out the best way to get the code in without spending too much time on upfront design. At the same time, we are also getting used to paired programming.

Even though paired-programming has become an old trick for me, I still feel that my pairing skill has gotten worse during the past three years of working solo. The second Sprint felt a lot better, and I am hoping to keep this trend.

Items that worth noting:

  • We modified the lava lamp to have a green light on when everything is good. Even though it is redundant, it has very positive effect among us. The only thing we might need to watch out is that someone mentioned that they could be fire hazard because the lamp gets very hot at the end of the day. So we are going to turn them off by the end of the day. This is when I found out that the X10 remote controller does not work, so they are back for replacement now.
  • The lava lamps are helping us getting on the habit of treating broken tests as the highest priority. Due to the nature of Phoenix, we got some interesting test breakage already. We got tests that only break on the server, tests that only break on Linux, and a test that hung. One interesting discovery is that each time we are forced to figure out what is wrong and fix them, our tests ended up making better sense and being more like behavior driven, and I was planning on settling for hacks to keep the test passing!
  • At the beginning of the project, we chose to create just enough stories to get us through the first Sprint, then created a few more for the second Sprint. Looking back, I think that is a good choice. The kind of stories that we create now are so much different but better from the earlier ones. I think that is because at the beginning, your system has literally nothing. It would take a very good story writer to come up with a list stories that really fit into the “INVEST” category of the story. I am not saying it is impossible, I just think that two Sprints of bad stories is not a bad price to pay to get the ball rolling as early as possible and avoid lots of hassle to learn and teach and debate about good stories vs bad stories.

Five Sprints into SCRUM

We kicked off Sprint 5 yesterday for Application Framework team.

Guidewire development is following SCRUM methodology. However, through all these years, due to various reason, the ideas behind Sprints are not exactly followed. There are many reasons for this, some of which are actually good reasons. However, that does not mean it was the best decision, and some development teams are trying to bring back meaningful Sprints to the development process, including AF team.

So what have I done differently this time?

We ended up using JIRA to track our stories. There are many reasons for this. I think the first one is the kind of work we are doing right now. We are not yet doing active development, but rather fixing bugs for a point release and run performance testing. Since all the bugs are created in JIRA already, using JIRA to track items that are not bugs makes it easy to track all the items we need to do given any Sprint. On the weekly work-from-home day, which each Guidewire employee can choose freely, it is very convenience to go to JIRA to pick the next work to do.

I am still keeping a Sprint board by writing down the JIRAs on the story cards but it is not as effective as I would like it to be. I think one reason is that QAs are verifying the JIRAs on their own schedule. (And the reason for that is some QAs are not part of AF team, because AF work affects other application teams). I know it sounds strang, but that is the situation right now. We are talking about how to get away from this mode and have a real complete independent development teams but before that happens, we will just have to pull it through.

The purpose for Sprint board now is more for daily Sprint meeting, where we talk about what we have achieved yesterday and are planning to do today. I use it to help the team focus and work on only the blocker JIRA or the JIRAs scheduled for the Sprint. Old habits die hard but we are making progress in that direction. When we schedule too many for the Sprint, which has been the case for all the past Sprints, I use Sprint board to figure out what to push to the next Sprint. I have not bee doing this aggressively. Now that I have an idea of our current velocity, I’ll do more now.

I am also changing the Sprint planning format. I am not going through the JIRAs one by one and ask question on them anymore, because the feedback has been that it takes a long time and becomes uninteresting. I think the first reason is that we are not sharing enough to make it a team conversation. Rather, it is me and whoever owns that part of the system talking with each other and figuring out the tasks to do. Even that, because I cannot pair on each and every JIRA, I am not able to track and check that each JIRA is estimated correctly and each JIRA is done within a Sprint. Without following them up and closing the feedback loop, all the work of creating tasks and track them become rather pointless.

So in the Sprint plannig, I now show the JIRA list scheduled, talk about them briefly in groups by the functional area, and track down the estimate after the meeting. I think I will change the estimation to be before the meeting next time, so that I would know how much to schedule for the Sprint.