Aug 19, 2013 2 min read

Lies, Boundary Cases and Checking Our Work

Lies, Boundary Cases and Checking Our Work

I’ve been doing some research into marketing automation solutions over the last month, both for myself and for a client. That included signing up for several solutions and running them against my own website. Along with that, I’m getting daily “lead” reports.

My weekly email report from HubSpot included this surprising subject line: “Visits up 93,200%, Leads up 300%.”  Maybe I can cut down to consulting only one day a month.

Clearly, this is just some dumb math. Comparing this month to last month – when I was not running HubSpot and therefore had zero visits tracked their system – is a high school algebra mistake. Nice that they divided by 1, instead of 0, which would have suggested infinitely more visitors than in July.

click to see original email

It flags two product problems, though:
– Sending side: marketing automation companies live and die by their ability to deliver meaningful, actionable information. That includes email nurturing campaigns like this one, intended to spur product usage. You’d want to apply the same level of product QA to marketing content as to production reports,
– Receiving side: I presume that every single HubSpot trial user gets four outrageously inflated weekly progress reports before meaningful month-to-month comparisons kick in. That’s tens of thousands of clunker emails. Apparently no one has bothered to report the problem. Maybe we’re all just exercising our “laugh, grimace, and hit delete” reflex? Disappointing if marketing folks are really using this in production, since they should be looking to their automation providers for email campaign best practices.


It’s easy to lose track of your product’s initial experience. After a while, your eyes skate past the obvious stuff and see only what’s changed. If you’re in product management, you should start fresh every couple of months, and go through the entire sign-up process – behaving as naively as possible.

Even better, have some untutored outsiders go through the process and write down every confusing step and keyword. Use your web traffic logs to see where folks are getting lost or clicking HELP. And read everything that you sending your customers before they read it.

Great! You’ve successfully signed up.
Welcome back! You've successfully signed in.
You've successfully subscribed to Rich Mironov's Product Bytes.
Your link has expired.
Success! Check your email for magic link to sign-in.
Success! Your billing info has been updated.
Your billing was not updated.