Stories that sell technology

Some years ago that was revolutionary. These days you put up a website and invite people to email you. But don’t you hear mostly from the deeply dissatisfied and the insanely satisfied? What if you’d rather take your public’s pulse more systematically to watch trends, follow emerging perceptions, or spot problems before they get out of hand?I sell cannoli at street fairs, and lately I’ve seen a slight downward drift in sales. But why? When the hit man in The Godfather uttered, “Leave the gun. Take the cannoli,” he breathed new life into the market for this ancient Sicilian pastry. Was the cannoli’s popularity now beginning to decline? How could that happen? Everyone still seems to love my cannoli. To my face, no one complains. But is it my cannoli? All cannoli? There were so many questions, but how to get the answers?

I called Tom Bershad, founder and CEO of MarketLab Research, Inc., a nationwide firm based in Philadelphia. Could he judge the cannoli market? Sure! Let’s see, a simple phone study, including analysis, would start at about $15,000. Focus group? The normal price is about $5,000, but for you … (No, he wouldn’t take cannoli as barter.)

Since I already have a few thousand customer email addresses I’ve collected for my cannoli alerts, I figured there had to be a cheaper way to conduct my own market survey. And there was.

Survey the Scene

The isurveys Website (www.isurveys.com) lets you compose your survey online and put a link to the completed survey on your own website. If you use this system, you’re spared the details of server and database administration — that is, a dedicated Internet connection such as DSL (as opposed to a dialup connection), and ideally a separate, dedicated Mac set up for file serving. Quicker than you can say, “Holy cannoli,” I registered and set up a two-question survey (just to try it out).

The procedure could hardly have been simpler. After you register, you hit the button that says, “Proceed to set up a survey,” which takes you to a screen in which you write your questions. Completed surveys look slick and professional. After you compose your questions, they tally up the charges. I opted for a seven-day run, which would cost $59. That’s $25 for setup, $10 for each question, and $2 a day for hosting. For an extra $ 10, you can let respondents see a running tally of results.

At these fees, if I plan to be running more than one survey, I’m better off buying software and braving the setup on my own. Waves in Motion, the company that produces iSurveys, is prepared for people like me. It produces The Dragon, a desktop survey-making tool. Your completed surveys will look just like those in iSurveys. If I do more than one or two surveys, I’ll save money with Dragon.

Enter the Dragon

Before you can make The Dragon breathe fire, you have to find its pilot light. Where is the starting point in this tangle of files and folders? Unlike most software these days, Dragon 4.1 makes you hunt. But here’s a tip: From the documentation folder, open “quickstart.htm” in your browser. That seems to be where the developer wants you to begin.

Once inside, the survey-making procedure is easy and straightforward (though you do have to overlook numerous typos). As with the iSurvey site, I was able to set up a good-looking survey in less than an hour.

Once into Dragon, I discovered the magic of “cross tabs.” This feature helps analyze results. Say that 57% of respondents indicated that they prefer hot dogs to cannoli. In the next moment I wonder: Is this why cannoli sales are down? Competition from hot dogs?

A moment later, cross tabs save the day. I compare this prefer-hot-dogs-to-cannoli blasphemy with another question in which I’ve asked how many times they’ve had cannoli. Quickly, my hypothetical panic turns to hypothetical insight. In my mental war game, I imagine that of the 57% who prefer hot dogs, only 10% have had cannoli more than once. But those who prefer cannoli over hot dogs, watermelon, and even life itself have had cannoli many times.

What does it all mean? In my stupor of relief and gratitude, I think it means that I don’t have to worry so much about those who prefer hot dogs. They tried cannoli once and may never again. The people I really care about are the cannoli repeaters.

Setting up a cross tab in Dragon is easy - just select two questions, issue a command, and the program produces a grid showing who answered what.

Dragon runs on FileMaker 4, which costs about $200. But keep in mind that for Web surveys it also requires that you set up a server for FileMaker.

Though some features in Dragon seemed to work well, the typos and scattered feel of the product made me decide to keep looking.

Power to the People

Though PowerKnowledge’s PowerTab lacks some of The Dragon’s charm, it makes up for it in sheer professional punch. It allows you to create ‘Web surveys and supports manual entry if you’re moving the data from print, fax, or phone surveys. It also runs on just about any Web server.

PowerTab opens with the Expert Helper, which after asking you a series of interview questions tries to write your questionnaire for you. Since I prefer to write my surveys myself, I instead plunged right into the Survey Editor. Setting up questions is almost as easy as in Dragon. Both give choices of single selection (”Have you eaten cannoli?” Yes or no.); multiple choice (”How many cannoli have you eaten in your life?” A) more than 100, B) 50 to 100, C) I to 49, etc.); or text answer (”Describe your first cannoli experience”).

PowerTab also creates “rating scale tables,” which display a list of attributes to rate on a scale of one to five.

Once you’ve finished writing your questions, you can save your survey for the Web, printing, or both. And when you do so, the program walks you through a few essentials: Where will the survey be on your Website? Where will the Perl script, which controls the textentry fields and other interface features of your site, be housed? This kind of hand-holding I don’t mind.

After results have poured in, the report section helps you figure it all out. The “simple” report gives the number of responses for each possible choice and automatically figures the percentage. It also gives the option to include the numher and percentage of respondents who simply skipped the question altogether, which is useful–perhaps the question or the given choices were unclear.

You can break results down further in PowerTab’s detailed reports, or “banners” (which are the same as Dragon’s “cross tabs”), so you can compare responses to one question with responses to another. The program breaks down responses into smaller subgroups for easier analysis. One of the handiest features I found was the ability to import FileMaker data to compare with my survey data.

But be careful how you wield this program’s power. For one thing, it lets you edit questions while your surveys “in the field,” as the research pros say. This can skew the results, so if you’re tempted to make a change after many people have already responded, you might want to hold back and chalk it up to experience.

To the Polls!

An even more sophisticated tool is ePoll, a Java-based Web survey. And as with Dragon and PowerTab, you’ll need a dedicated connection with a Web server.

ePoll doesn’t make you settle for taking respondents through just one list of questions, regardless of what responses each person gives. ePoll can change the questions based on each person’s responses. If a respondent says she’s never had cannoli, why annoy her by asking when she last ate one? Avoiding this improves her experience, making it more likely she’ll actually complete the questionnaire. The program allows you to filter responses, so you can call up only the ones that meet your specific criteria. If you want to try it out, you can avail yourself of a free ePoll demo.

Wish List

I had hoped that one of these packages would provide me with a tool for easily following up after a survey. This is usually important for pulling in good response rates. Your customers are busy, and sometimes they ignore your first e-mail. Many people won’t respond until you remind them once or twice That worries me. Many of my cannoli regulars enjoy a traditional lifestyle. They never hurry. Luring them into any kind of survey will take work.

Let’s say I asked 2,200 people to come and fill out the survey, but only 150–a measly 7%–have actually taken me up on it. It’s a pain to remind the other 2,050 in Dragon or PowerTab without annoying the 150 originals. I would have to manually eliminate the 150 from the followup list, one by one. This would feel frustrating very quickly, and after a few minutes of that, I might feel like leaving the cannoli and and picking up the gun.

Flight Check

As easy to use as all the software was, something nagged at me. Would I get the results I needed? What if my questions and the choices I offer skew the results one way or the other? Would enough people respond to the survey?

Here are a few quick tips in composing your questionnaire.

• Even if you have nearly as many of your customers’ phone numbers as you have e-mail addresses, don’t call them - it’s just too much work.
• Shoot for at least 300 responses. That’s about where market researchers I talked to find that results get steady. Cross tabs require even more total responses, since the data is sliced and diced into smaller chunks.
• Make sure each question and each choice are crystal clear.
• Make sure your questions and choices are as neutral as possible. ]Don’t suggest choices.
• Keep it brief. Make every question count, and try to keep it to 10 questions or fewer. Everyone’s busy.
• Test your survey before you cut it loose on the public.
• If you need help with any aspect of your research–such as setting objectives, writing the questionnaire, analysis–inquire at a local business school or small business organization.

Fielding the Survey

What do you do after the whole survey’s prepared? You blast e-mail to the whole list, of course. But what you say in that e-mail is important. First, always offer an incentive. It can be inexpensive, but not one that will skew your results. I’d love to offer cannoli coupons. Bad! All the cannoli fans would respond and no former fans would. Better to offer some other coupon, a lottery ticket, or simply a dollar bill. Contests are legally dubious. Remember to follow through. You should also make sure to give a deadline. It should give enough time to respond but not so much that people think, “Oh, I can do that later,” and then forget. A good rule of thumb is five days. Also, try to keep it brief. Say please and thank you. Tell them that in such a small group, each response is important.

Besides, you want to stay on good terms Finally, tell how quick and painless it with them and not risk being thought of will be.

The Results Are In
No wonder the pros get big bucks. But I’m not worried. My quick-and-dirty little survey will tell me much of what I need to know. All I want to do is offer people cannoli they can’t refuse.

Ted Cuzzillo is a Point Richmond, California, writer, editor, and cannoliphile. He gets his hands dirty writing questionnaires and collecting market research data in Silicon Valley.