Automated tests versus Manual Testing

Right now there is a bit of a hot topic in the Agile community around Automated Testing versus Manual testing. Ron Jeffries just posted a great article, and Keith Ray also posted a response to one of Ron's posts on the Agile Testing mailing list. Ron had said on the list:

The only time not to automate a test *ought* to be if we are never going to want to run it again. But it isn't. The more common time we don't automate a test is because it would be "too hard" to automate it. That is, however, not really a property of the test, though we like to put it that way. It's really a property of our own test automation capability and tools.

And while Keith hits some good points about our own testing tool capabilities, there still another nugget in Ron's post. Any test we don't want to automate because it is "too hard" is a test we *will* run again. And again. And again. In fact, I'd wager to bet that those "too hard" tests are usually some of the most critical tests in your system.

Remember, duplication counts as something to be refactored both in code and in process. If you are doing it twice, automate it. You'll thank yourself later.

How Eclipse Helped Me Understand My Frustrations

When I first came on board my current employer, apart from the great teams, great benefits, and all the other goody-goody stuff, there were a couple of things that bothered me slightly. One of those was the belief that all of the developer machines should have identical software setups, and that exploring with other software packages was fine - as long as it was on a different system, preferably on a different network. I couldn't pick out what it was that bothered me about that, only that it did.

Throughout the past few months, a couple of other things have come up which have also tweaked me slighltly the wrong way. Again, I couldn't put my finger on why, but they did, and I was beginning to grow resentful of hearing them.

Earlier this week, I came across a blog entry by Jim Shore where he mentioned the Satir Interaction Model, something I had heard of and looked into before. I glanced at his references before I headed to bed that night, but didn't think that much of it.

During the night a startling realization came to me. I cringed when I heard the identical machine setup (specifically around the version of Eclipse) because it was a decision not being made by the team, but by management. In other words, the company had a problem where different teams were using different versions of Eclipse which caused some collision and conflicts because things weren't identical. Rather than pose the problem to the team, and let them work to self-manage a solution, management dictated a solution.

I realized this bothered me because we are normally very good about letting the teams manage the problems, and some really great solutions usually come out of that when they realize it is a problem. But for some reason, in this case, that process was skipped, and that's /really/ what was bothering me.

I just finished reading Dale Emery's Untangling Communication article, and it hits exactly what the problem was - I took in some words, and applied a past experience I didn't fully understand to attach an emotional feeling to them, which muddied my thoughts to not be able to see why it bothered me. But knowing now why I feel like I do, I can more easily approach a solution and look at some other options which will solve versioning problems potentially without necessarily locking us all down to have to be identical.

Being Agile is /not/ easy!

James Shore had a great blog entry about inflated claims, specifically now that agile is starting to cross the chasm into the mainstream.

What I find so interesting about Agile is that in order to be good at it, you have to do a lot of work both to stay disciplined and to become a better developer. In an agile team, your goal, your only goal, is to provide value to your customer. That means that if you take 10 months longer because you don't keep up with the latest industry trends and write your own "Enterprise" Foo, you aren't living up to your promise to deliver the most value for your customer's dollars.

It doesn't just stop there though. Today we had a deployment that in testing we discovered a bug. The bug was that a translator we use wasn't changed when the corresponding code was, so when it pulled it out, it cast it to a String array instead of a List, causing a ClassCastException. The problem wasn't in that we forgot to change it, it was that no test stopped us from moving it up. Why? Because the code was supposed to not change, and was supposed to be temporary, neither of which held true. So we as developers failed our customers because we didn't do our job, fulfill our promise to write the best code we can, test-first, to deliver running, tested features.

Thankfully we *did* have good acceptance tests which caught the scenario, but it should never have made it that far. And that's the trick with being Agile. If you slip, even for a second, it *will* bite you. You have to stay on guard constantly, and also constantly improve your code and yourself. Your customers, and your maintenance programmers, will thank you for it (meaning you can pat yourself on the back 3 months from now when you have to change that temporary, unchanging code without spending 2 hours chasing down a bug).

The Power of the Internet

Today I got an email regarding a tutorial on setting up PPTP with Monowall from a gentleman in Brazil. The email he sent was in Portuguese, but I was able to pick out "Monowall", "porta 5900", "acesso remoto" and some other keywords which led me to believe he was trying to open a hole in his firewall for VNC access. I passed his message through Babelfish (yep, they are still around) which helped a little.

I pulled up my Monowall installation, documented the steps to open a port for VNC, and sent it back. Before I did, I ran the steps through Babelfish to psuedo-translate them to Portuguese. Out of curiousity, I then translated it back to English revealing my steps as:

its console of administration of the fotorreceptora leather strap of Monowall enters

• estale on rulers under the keep-fire


• the action leaves as the ticket, relation so WAN, the protocol how much the TCP, and the source as some


• it has adjusted to the scale of the port of the source to be 5900 the 5900


• the destination has adjusted as a "only host or by the way" and puts it in the address of its user of VNC


• it has adjusted to the scale of the port of the destination to be 5900 the 5900


• a click of the description types inside (something as of "ruler VNC") - excepto


• in the keep-fire it governs the screen, click "apply changes" This must open above of port 5900 in its keep-fire going to move 5900 in what computer you are functioning VNC.

I just got an email back from him, and from what I can tell it worked like a charm. The power of the internet and the services available truly are amazing, and I am thankful for opportunities like this to remind me of how truly powerful it is.

A different way of grouping tests?

Recently on the TDD list , we were discussing the benefits/drawbacks of long test names. I had posted an example to one I use in my Fitnesse talks that looks like public void RemovingLastUserFromRoomRemovesRoomFromAvailableRoomList() {}. Jeff Langr initially suggested something more along the lines of EmptiedRoomIsUnavailable, which I thought was a little too concise to be able to tell at a glance what was going on inside.

After some more discussion, it prompted a different way of thinking about grouping tests. Usually I group them based on the production class I'm testing, basically one test class to one production class. But what if instead of grouping by class, we grouped by behavior? Something like this might come about:

public class RoomRemovedFromListWhen {
  public void lastUserLeaves() {
    //...
  }

  public void roomIsKilledByModerator() {
    //...
  }

  public void lastUserSessionTimesOut() {
    //...
  }
}

The above seems to really capture the behavior we are trying to test, and the conditions that behavior could occur in. It will be interesting to see how this affects my thinking about tests in our test suites.

Hiding variable declarations in Fitnesse

Handy little tip - if your page has a lot of variable declarations like:

!define myVar {1}
!define myOtherVar {2}
!define someOtherVar (blahblah)

You can hide it from view by wrapping it with a collapsable section. So the above would look like:

!*****> Variable Declarations
!define myVar {1}
!define myOtherVar {2}
!define someOtherVar (blahblah)
**********!

and would show up automatically collapsed (thanks to the use of the greater-than sign.)

NUnit, CodePlex, and other news

This weekend was quite busy. First, I'm excited to announce that I've been added as an official NUnit developer. I'll continue working on making sure NUnit builds and tests cleanly in Mono, especially on Linux.

In addition, the NUnit team has announced two new-ish projects. The first is a VSUnit Visual Studio plugin that Charlie Poole has been working on. The second is NUnitLite which will basically be a tiny version of NUnit that can be embedded in your app.

You'll notice that last link isn't to SourceForge. That's because we've been invited to be part of a new Microsoft Open Source community called CodePlex. What is CodePlex?

CodePlex.com is a simple but highly functional online venue for distributed, community-collaborative software development. All projects are 100% public, transparent, and open source. CodePlex's promise is to enable Windows and .NET developers like you to write, share, and consume source code and built binaries with each other and with the rest of the world as fluidly as you can on an intranet or VPN: openly, securely, and freely.

Initially, we envisaged CodePlex as a replacement for gotdotnet Workspaces. During its development, it quickly evolved into something much, much more. CodePlex leapfrogs sites like Workspaces in ways that I think you will find both promising and exciting, in terms of features, client affordances, licensing, and project management. It is not 100% done but I'm sure that you'll discover that its quality, performance, and the direction in which it's headed are clear. Written from the ground up in C# utilizing .NET 2.0 technologies and built on top of Visual Studio Team Foundation Server (VSTFS), CodePlex enables geographically-distributed teams to enjoy the benefits of VSTFS over the Internet: a one-of-a-kind resource.

We're not actually sure how this is all going to work out, but we as the NUnit team decided that if Microsoft wants to try to open up to the Open Source community, we'd at least give it a try. If it doesn't work out, we'll just pull the projects and move them back over to SourceForge. (Charlie does a better job of explaining NUnit's involvement on his blog).

So all in all an exciting weekend, and some projects and sites that it will be interesting to watch.

Pictures from St. Louis Code Camp

While I wasn't able to get pictures of the sessions, mostly because I was speaking in some of them, I did get some shots around the Code Camp:


Registration Desk


MPR1 / Conference Room


Breakfast is served!


MPR2 / Main Room


Rams / Conference Room


Microsoft showing off technologies (note the screen on the XBox is Red vs Blue!)


Lunch Crowd


Brian Button keeping the crowd entertained with his famous agile puppet show


The lunch spread (with one of the giant subs already devoured)


Sean Carley enjoying the lunch spread


Brian showing us why our user story isn't going to make it into this talk


Brian tells the crowd about the free shirts...


...and the crowd responds swiftly...


...and completely. ;)


The coveted XBox 360 door prize...


...someone is going to win...


...and the winner is...


...number 28 (or 00111)

Thanks again to Brian and the XP St Louis Group for a grand time, and I can't wait for the next one!

St. Louis Code Camp Follow up and Slides

Yesterday was the St. Louis Code Camp, and what a great time! I believe around 70-80 people showed up, and we had some really great presentations. On top of the books Apress donated, Microsoft donated some books, goodies, and an XBox 360!

The day consisted of four sessions across four different rooms. The first session I went to was Kyle Cordes's Lua presentation. He did a great job covering how to integrate Lua into an app (even if the app was Delphi), and though there wasn't a lot of Lua code, it definately sparked my interest in using it more.

The second session I spoke on Ruby for C# developers (Powerpoint here (with the Fib sequence corrected thanks to feedback from one of the attendees)). The presentation went very well, and I even went slightly over time. But a lot of people came up to me afterwards to thank me because they hadn't heard of Ruby and think they'll be trying it out.

I was disappointed that I didn't get to more of the Ruby and .NET integration - I just ran out of time. I did cover very quickly how to hook into a Windows Forms event using Ruby. That code looks like:

require 'dotnet'
loadLibraryFromFile 'RubyWindows.exe'
form = Form1.new
button = form.cmdMixItUp
form.outputText.Text = ""
button.click.add do
  form.outputText.Text += "Hello from Ruby, " + form.userName.Text + "!\r\n"
end
Application.Run(form)

Which loads an exe called RubyWindows.exe, hooks to the Form that is in it, then attaches a listener to a Click event on a button on that form.

We then had lunch, and Brian and the guys brought in a wonderful spread from a local deli. I should have the pictures of the event up today or tomorrow, so you'll get a chance to see that.

Third session was Brian's Agile talk. It was a very loose format where we, as participants, basically did a planning game to figure out what he was going to talk about, and then he spoke in 10 minute "iterations", and calculated his velocity to see how many of the topics he was going to get to. It was good to see that his velocity was pretty stable (even if there were a lot of topics left on the board at then end).

The fourth session I gave my Fitnesse and .NET talk (Powerpoint here). It was a good presentation, though looking back I wish perhaps I would have gone more in-depth to the code behind the Fixtures. I also ran out of time in that one - I think I need to (actually) keep by chatting down a bit. (The code and Fitnesse test files are the same I used at the Mid-Mo presentation, and are available at that entry)

Overall it was a great time, and by the end people were coming up to Brian asking him to do this twice a year, so it looks like that might happen. Brian will also be posting a summary of all the sessions and links to all of the code on his blog, so watch for that!

I took several pictures while I was there, so I'll get those up today or tomorrow.

See you all at the next one!

Ruby ActiveRecord and Windows Forms

In getting ready for my talk this weekend on Ruby, I scanned the RubyCLR project founder's web site, and came across this post:

This is a really exciting development in RubyCLR. I’ve taken the first steps in integrating Rails’ most excellent ActiveRecord object-relational mapping layer with Windows Forms data binding.

He then goes on to show some screen shots and sample code. How much easier can you get then:

class MainForm
  def initialize
    form = Form.new
    form.Text = 'ActiveRecord and Windows Forms'

    grid = DataGridView.new
    grid.dock = DockStyle::Fill
    grid.data_source = Person.find_all
    form.controls.add(grid)

    @form = form
  end
end


I can't wait to download the latest HEAD and play with it. Great job John!

Pair-Programming in the Wall Street Journal

In the article Tech Companies Check Earlier for Flaws, the WSJ almost does a good job bringing the need to slow down an inject slack into an organization in order to produce higher quality code. I say almost because while it hits some good points, like:

While the BlackBerry had escaped serious scrutiny for security holes, Herb Little, a RIM security director, worried the company hadn't paid enough attention to the software that runs on the BlackBerry and other devices. "The idea was that we could be doing more," says Mr. Little, who is based at RIM's Waterloo, Ontario, headquarters. "We had to raise the bar."

and

Revamping the software-development process creates a Catch 22: being more careful can mean missing deadlines.

It also has suggestions like:

Now Mr. Little uses Coverity every night to scan the code turned in by engineers. The tool sends Mr. Little an email listing potential red flags. He figures out which problems are real and tracks down each offending programmer, who has to fix the flaw before moving on.

which is only about halfway there. If you want to increase the quality of code, one way is to have more frequent code reviews. I suggest after about every word a developer types in. Surprisingly, so does the WSJ:

Since then, Mr. Ludwig has adopted Fortify software and improved communication between his team of security experts and programmers who write software. A few years ago, each group worked more or less separately: The programmers coded, then the quality-assurance team checked for mistakes. Now, programmers and security types often sit side by side at a computer, sometimes lobbing pieces of code back and forth several times a day until they believe it is airtight. The result: "Issues are being found earlier," Mr. Ludwig says. But, he adds, "I'm still trying to shift that curve."

Pair programming. Whole Team. Slack. Guess these things do make sense.

Jet-Powered VW Bug

Very interesting article in the San Fransisco Gate online about Ron Patrick, a 50 year old Engineer who has put, very cleanly, a 1450-horsepower T58 Jet Engine on the back of his VW Bug. My favorite quote:

He said that a jet-boosted run will "pin the speedometer and that's at 140." He thinks that when it hits 160 mph -- he hasn't seen that ... yet -- the car will start lifting off the ground, but "the fun is not necessarily how fast you want to go. The fun is the sound of the thing. Just starting it up, it's like a (Boeing) 747 landing in your front yard."

The article is worth it for the pictures and video of the thing being lit up. Enjoy!

Upcoming Events

This Saturday, May 6th, is the long awaited St. Louis Code Camp. Some good sessions throughout the day, and I'll be giving two talks: one on an introduction to Ruby for C# Developers, and the other Achieving Customer Zen with Fitnesse.

Then, Wednesday May 10th I'll be at the Mid Missouri XP User's Group as part of an XP demo where we'll step through user stories, acceptance tests, TDD and customer interaction.

Hope to see you there!

Responsible Development

A couple of weeks ago, I was talking to a reporter about Offshoring and Extreme Programming. One of the challenges was the term, "Extreme Programming". How do you get the ideas across as a whole unit without turning people off by the term?

Turns out, even Kent Beck is having to do that. He is calling it "Responsible Development":

I began using the phrase "Responsible Development" on a recent tour of workshops and I really like it. I find it helpful to ask myself, "What would be the responsible thing to do now?" As an adjective, "responsible" doesn't have any of the zip and zing of "extreme", but I've grown leery of the kind of attention attracted by zippy and zingy words.

I love the idea of thinking what the most responsible thing to do now is. Personally it seems to dovetail nicely with the concept of doing the simplest thing possible. If we might know more a month from now about how to do something, is is the most responsible thing for us to guess at it now?