The line between art and science frequently blurs. I can explain the science of the curve ball to you, but I can’t throw it with the artistry of a Jon Lester or Tim Lincecum. There is art in the instinctive application of the science.
Modern marketing is no different. You artfully design several different offerings for a product or service and then you scientifically decide which one works best. Maybe as a group your collective instincts can help you eliminate 2 of the 5 offerings and focus your ads on the remaining three, picking only the best for the major advertising campaign based upon actual results and not upon which offering was your favorite.
In helps the validity of your testing if the underlying process is without bias or deviation. If not, the bias or deviation needs to be considered/accounted for somehow. For instance, if your best call center staff works Monday-Friday from 8A-4P, then you better make sure that all of the different ads run during that time frame. You don’t want to pick the ad that converts the best because it was worked only by the best salespeople. Or, if your salespeople HATE internet leads but love phone leads, it is tough to compare the success of internet campaigns with a TV campaign or a newspaper campaign that results in phone calls and not web leads. (We’ll cover the difference between web leads and phone leads in a future blog.)
The bottom line is this: trust your instincts, but trust your data more.