The First Rule of Beta

The first rule of Fight Club is – you do not talk about Fight Club. The second rule of Fight Club is – you DO NOT talk about Fight Club.

Beta isn’t quite the big secret that is used to be, but it’s still under NDA. Lots more people are getting involved with every release. The first beta I was involved in with Adesk, a friend pulled me aside and clued me in to how to be a better Beta tester. I’m not an expert at this by any means (See Lee Ambrosius!) but in the interest of public service, I give you,

The Rules of Beta

  1. Beta is not a wishlist. I don’t think this can be emphasized enough. The code is done, and it’s pretty likely nothing new is being added. Really. Nothing. OK, maybe some minor stuff, but no new features. This is feature creep and to be avoided at all costs. Feature creep gets you R13.
  2. It’s never as simple as it looks, and the law of unintended consequences is a real bitch. What we as outsiders view as simple modifications often lead to deeper levels of complexity in a real hurry. It took me a long time to believe this one.
  3. What you want is not what everyone else wants. It’s possibly not even what the guy down the street wants. Remember that C3D is a global product, and every bit of coding probably means more coding in 12 other languages. We in the US are a small part of the C3D market!
  4. Beta testing without a backup plan is fun, but foolhardy. Trust me on this. I managed to hose my machine to the point of FDISK last year. Backup, use VMWare, do something…unless you really enjoy the reformat on a Tuesday night.
  5. It’s about functionality, not appearance. Much of the time, the icons, dialogs, etc. are still in development during the beta testing cycle. Try not to get spun up about a dialog, test the function!
  6. Reproduce. No, not like that. When you encounter an issue, try it again. If you can’t reproduce on your own machine, how can you possibly expect an analyst to reproduce the issue on their machine thousands of miles away with none of the same junk on his or her machine.
  7. Document. Document. Document. A random error message without the details steps isn’t very helpful. Many of the same error messages can be generated from different places in the program, so knowing how you got there is absolutely crucial.
  8. Be nice. There are real humans on the other end of the process, people who have poured their careers into what you’re now labeling a disaster. This is another one I still fight with, but knowing how much effort and personal attention the dev teams put into their product makes it easier.

What else? Beta team folks? Long time testers? What else should every new beta player now? Comment below! Beta is an important chance to feedback to the dev team, so do it right!

Fight Club and Road House in one post, that has to be some kind of weird first.

9 comments

  1. In regards to your rule 4.

    The only computer at GBA with Beta installed is JWedding-XI.

    JW: Seems right, I’m a sucker for beta. Side-by-side as we speak.

  2. Murph says:

    Another “rule” don’t expect everything that is in a beta to be in the final release. Over the years there has been some “cool” features that were tested in Beta yet it took a few releases to be finalized bug wise to go in a final release. In ref to JW’s rule #4, Yep don’t run a beta on your main production workstation and don’t even try using a few feature in beta to save some time on that project that’s due Monday AM. Now if I “was” a beta tester, can’t tell you if I am or not 🙂 I would run the betas on one of the old workstations to see how it runs with some of the old boxes that a lot of users are still running.
    PS: Anyone know the DOS command to run SubDivDesign.

  3. sdotson says:

    I have to disagree with #5. While we can certainly overlook small visual changes the overall interface is a very important part of the product. When icons and dialogs change drastically it is VERY important to provide feedback on these new interfaces.

     JW: I knew I recogized the name. Sean’s site is the definitive answer for things Inventor. Check it out by clicking his name.

  4. I agree that Sean’s site is a pretty good site for getting your arms around Vault.

    I learned a lot there without contributing squat.

    Thanks Sean from one of the troll’s that learned and didn’t return the favor.

    Unfortunately the Mech’s make my head hurt.

  5. Steve Boon says:

    I would admit to breaking #1 but I’m not allowed to talk about it. Of course the new features in the Beta inspire more wishlist items, but you can’t post them except to the Beta discussion groups.

  6. Jason Hickey says:

    James hasn’t added one that I mentioned, so I’ll say it here. If you get offended by this one, it may be you that I’m talking to.

    If you don’t know the basics of how to operate the program, beta testing may not be for you. This is NOT your chance to evaluate the software and decide if it’s a right fit for your company if you don’t even know the most simple tasks. Here’s why:

    Beta by it’s very definition is unstable. It crashes. It may not perform correctly. As Murph said, it may have functionality that doesn’t make it into the shipping product. If you’ve never seen Civil 3D before, you WILL BE DISAPPOINTED by the beta product.

    If you accept these limitations, then realize that the beta forum is not the place to ask questions that would be answered in a training class. That forum is for providing feedback on new/enhanced functionality, not for learning which icon to click on to start the program.

    As for Sean disagreeing with #5 – I agree to a point. Majorly different looking dialog boxes, sure, give feedback. But don’t argue with the programmers if the little bulldozer’s blade should be up or down on that icon – most of us run resolutions so high that we never knew it really was a bulldozer until someone pointed it out (or was it a dump truck???) As long as the tooltip tells what the button does (long enough for me to memorize that button location), I think it’s great.

  7. blairs says:

    All feedback counts, be constructive, treat it no different than you would have someone look at your work. Someone worked hard on that Icon or section of code or XML text. We get defensive of our work. Beta testing is tough, I use a secondary machine, with seperate migrated data. If we can break the software, find bugs, suggest enhancements that make it to production we have done our job, if not in the beta we testing, the next release. I feel bad when hot fixes and sp’s follow releases follow shortly after software that I beta tested on. It’s like the user forums with all the people with hardware and install problems. I’ve been using IV since version 4 and never had a hardware and install problem, or a problem installing the IV11-DWF extension.

  8. oktiv says:

    I love it “Feature creep gets you R13” can i get that on a shirt!

  9. I like Rule # 8 – Many often forget that about the “people” on the other end…not everything is in their control.