Editor-in-chief Jonathan Dorn and his team at Backpacker began using Web-based editorial surveys last January in an effort to capitalize on a recent relaunch. What they discovered was a cost-effective (essentially free) method of keeping the magazine’s content in line with reader expectations in an easy-to-execute format that enabled the team to make changes quickly and confidently with response rates averaging 50 percent.
In 2003, Rodale launched comprehensive research surveys at the behest of CEO Steve Murphy who wanted to begin a process of updating the company’s magazines, including Backpacker. According to Dorn, the 320,000 circ magazine went through a survey process that included focus groups and interviews with 2,000 readers. A year later, he and his staff began planning how they were going to execute against the survey’s findings, most significant of which was developing regional editions of the magazine.
When the dust settled, Dorn had a freshly redesigned magazine that rolled out in seven editions;six regional and one national for every issue except one, the gear guide.
Frustratingly Slow Feedback
Early indications were positive. The first two editions were up 15 percent on the newsstand and were pulling similarly on the sub-card responses. But Dorn had no handle on exactly why the readers were satisfied. A formal research study was out of the question. “As an editor, it’s frustratingly slow to get feedback,” says Dorn. “I’m getting results on a January issue and it’s August and I can’t impact anything until December. So we’ve developed a second type of research;something that’s a little bit more down and dirty, an online survey with a reader panel.”
Dorn’s panel has 2,000 readers split 65/35 between subscribers and newsstand buyers. “It’s not all long-term subscribers who are going to be in love with whatever we do,” he says.
The online survey, backed up with software developed in-house that provides a Web-based interface and illustrates results with bar charts and graphs, has allowed Dorn to canvass the panel and get feedback in time to put changes in effect for the next issue. “From the time that I finish writing the survey to the time that we get the results is no more than three weeks. So we can use it to turn on a dime.”
For reaction to the regional editions, key questions revealed that readers liked the coverage but wanted even more hike routes;just not in the format Dorn had anticipated. “We used the research to create the regional editions but then changed them when we were rolling them out.”
Since then, Dorn says Backpacker has run about a half-dozen online surveys, each staying live for about two weeks allowing for optimal response rates, which have been averaging 50 percent. This is far above
Dorn’s self-imposed minimum of 250 responses. “I would feel uncomfortable making decisions about changing anything in the magazine if I was getting less than 250 responses,” he says. “And I certainly wouldn’t have made any changes of the import as the regional editions if I didn’t have at least 500 respondents to that.”
Dorn has not seen any significant drop-off in response since Backpacker launched the online surveys and estimates that anything more frequent than monthly is pushing it;he plans at least one for each of the nine issues of the magazine.
The surveys run a maximum of 25 questions, some as little as 12, and with the online format, they have room to give respondents a chance to expound in detail. “Some of the more free-form answers give you a lot more texture than simple multiple choice,” says Dorn.
The online format, with its on-the-fly approach, may be looser than third-party research but Dorn places high value on timeliness. “The turnaround is so much faster. I recognized the potential for statistical anomalies in the way the questions are asked but magazines change so fast that I just can’t bear to wait to have a couple words changed.”
Dorn isn’t advocating the eclipse of traditional printed surveys by online. A better system, he says, is a “mixed media” approach that utilizes a combination of the traditional mailers and blow-in cards for example, and supplement their longer turnarounds and lower response rates with the quick-hit, high-response online surveys.