Posted by
I’ve done reader surveys at the end of each year, starting with Offbeat Bride in December 2008. After five years of reader surveys, I’ve decided to take this year off… and here’s why.
It got too complex
As the Empire grew, I first added additional surveys for each site, then eventually merged them all into one massive survey for all four sites, with piped logic that showed you certain questions depending on your answers. Things got really fucking complex.
It got to the point where parsing the data was a major challenge. Yes, if I wanted to know what percentage of married Offbeat Bride readers also read Offbeat Home & Life at least once a week, I technically had that data. Figuring out how to pull that data, however, was a grunt… and I say this as someone who’s taken demography classes! Last year, the data was so complex that I never even got through analyzing it all.
Advertisers don’t ask for the data
The original impetus for the reader survey was half thinking it sounded fun (FUN! I like stats!), and half wanting to have reader data for advertisers. After 5 years, however, I’ve learned that 99% of advertisers want straight-forward data that’s easily available from Google Analytics: How many readers do we have a month? How much traffic do we have for a given city, state, or country? No-one’s asking questions like “How long are Offbeat Bride’s engagements? What percentage of married Offbeat Home & Life readers found their wedding vendors on Offbeat Bride?”
The feedback wasn’t always actionable
Back when we still had an open text box to share feedback at the end of the survey, we got a deluge of thousands of comments… very little of which was actionable.
I’d estimate that at least a fourth of the feedback that came in the reader surveys was requests for functions or content that we already had:
2012 Reader Survey: the first-glance overview
Our 2012 reader survey closed late last night, and it looks like about 3200 of you responded. Before I get into anything else, let me... Read more
- “You need Offbeat Bride archives by state!” (funny, we already do — have you tried searching the site for your state?)
- “Why can’t I get posts via email?” (um, you can…)
- “You should do more posts about X” (Did you search the site for X, and see our tag archive dedicated to it?)
- “Why don’t you ever do posts about Y?” (…have you ever submitted a guestpost about Y?)
We also got lots of feedback requesting functions we’d already explained we’d never be doing, like an Offbeat Families forum. We also got a lot of ranting, which while valuable in its own way, can be difficult to know how to take action on… if one person thinks our transgender content is tokenizing and offensive, should we stop running trans* posts?
We have no shortage of reader feedback
I’ve always tried to be relatively transparent about how I run the Offbeat Empire, and part of that is encouraging a lot of dialogue here on offbeatempire.com. Basically, if you’re a reader who’s interested in getting meta with me, I am pretty much always listening! I do my best to make it clear that the digital door is always open to feedback, and based on the number of you who take the time to share your thoughts via blog comments and emails, I think the message gets through. Also, given the big shift with Offbeat Families a few months ago, I feel like we’ve heard a LOT from readers this fall.
Ok, so what am I doing instead?
Instead of doing a huge complex reader survey once a year, I’m going to be doing small polls more often. See, I still have all sorts of questions for y’all, I’m just not going to ask them all at once. That way, it’s easier for me to parse the data, and easier for y’all to participate. In fact, I’ll have a poll for y’all next week.
So, there you have it. No reader survey this year, although it very well may be back next year. In the meantime, as always: FEEDBACK AHOY!
Well, at least there will still be data (from the little surveys). When I saw the link on Facebook, I was like “NOooo, where am I gonna all my tasty data?!?”, and “How am I gonna tell them all my opinions?!”
So I guess I’ll wait until early next week before I start incessantly refreshing the page, waiting for the first, tasty, survey! 😀
Honestly, I could probably keep you entertained for months with the data I didn’t get through from LAST year’s survey. 🙂
And that is why my next degree will be in Statistics… so I can with other people’s data all year long 🙂
On the flip side, this sounds awesome for readers who may not be invested enough to spend 20 minutes answering a long, all-inclusive survey, but still care enough to answer a couple of poll questions. I’d put myself potentially in that category – I think I’ve seen a couple of feedback posts that I never got around to actually answering the survey due to my habit of opening a lot of “to-do” tabs that never get done before I end up shutting down my computer for the day.
One problem with survey data is the fact that it’s self-selecting, so giving a broader audience the opportunity to select itself in a low-time-investment kind of way seems like a positive in terms of getting a broader opinion, even if the demographics can’t necessarily line up as smoothly.
My first thought on this was ‘no, I have so been looking forward to the offbeat survey’. I love completing surveys and seeing the results from it all. Then ‘yay, more surveys, I love surveys’. I’m looking forward to surveys and results all year round rather than a once a year binge.
As a relatively new regular reader to the Offbeat sites, I was kinda sad that I missed out on taking those past surveys when I’ve read posts about them. But, I am eagerly awaiting the mini-polls! So it’s all good. 🙂
I, too, am one of those “Love data! Must has more data!” people. But as a person who also assists with / manages a lot of surveys for my workplace, the reasons you have for for NOT doing a survey are TOTALLY legit. And in our over-surveyed society, always a good idea to be looking at if surveys are really appropriate.
In summary, I support this decision! 🙂
Yeah, part of what was coming up for me was that I was way out of my league when it came to survey design and operations — to the point where every year, a chunk of the survey feedback wasn’t about the website, but rather feedback about the survey itself. It got too meta for me! I’m not trying to learn how to be a better survey manager… I’m trying to improve my publications!
As a business owner, I wear a LOT of hats… but I’m realizing that maybe advanced demographer doesn’t need to be one of them. While my sociology background makes me love demographics data, I have a lot of other work to do besides survey branching and logic forking etc etc. Easier to stick with simple polls! 🙂
Aw, nuts! But I’m a narcissist who loooooooves answering questions about myself! Sigh, I guess that’s not quite a good enough reason to host a large multisite survey 😉
HAAAAAAAAAA. Yes, it’s true: the special snowflake factor is very high with our surveying.
Just to let you know, on the home&life survey today one of the questions says that you can chose up to 10 options, but it only accepts less than 5.
This has been fixed.
I just wanted to let you know that I have tried taking the survey on both mobile (iphone) and my computer (chrome), and in neither one can I complete it! I saw something earlier where you were blaming poll daddy, but this just sucks. I would like to give feedback, but I’m giving up trying.
I’m sorry. Polldaddy has been super buggy and I won’t be using it again. Last night, in trying to compensate for bugs, I may have made it worse. Sorry for the hassle. It’s fixed now.