Small data vs. big data: Why you need to “sweat the small stuff”

| |

As compelling as big data is for its ability to find hidden customer behavior patterns and make predictions about how to spend money in the right way, there is a time and place for its little brother: Small Data.  While some may see Big Data as the only methodology of choice to drive the strategic direction of their organization, others see Small Data as the best way to get actionable results, quickly. 

While researching this very topic, I asked Mike Sherman, Marketing, Insights and CRM/Big Data Expertwho will be chairing a “Big Data & Legacy Systems” session at the upcoming Chief Data & Analytics Officer Forum Hong Kong, 4-5 October 2016, what his thoughts were on small data.

These are some of the personal experiences he had to share.

All too often, companies collect masses of data without thinking why the data is needed. It is ill-fated to take on a big data strategy before understanding how it will be put to use as more data can obfuscate the right data. Big data is not necessarily important, but having the right data is always important, which could be Big Data, but might just be Small Data. 

“A common challenge for websites is acquiring and keeping new users”

Many companies mine server logs, thinking aimlessly of ways to attain and retain customers or worse yet, bombard users with multiple, irrelevant emails. The better approach is to identify key moments that matter and provide timely (often small) pieces of information that help turn these into memorable, positive experiences for the customer.

During one engagement, my son Alex stumbled upon, a public data aggregator. After signing up for a trial account, he failed to login to his account (after several tries), as he had forgotten his password. Avoiding the hassle of filling out a form to change his password, he instead left in frustration with no plans to return. However, moments later, he received an email from prompting him to change his password with the majority of information already completed. The timely message caused him to change his mind and proceed to sign in to the website.

This was not a moment that required Big Data - it does not take a lot of data to identify that after several unsuccessful login attempts, a user likely forgot their password. Yet, it was a critical moment, where a single timely email made him an advocate, instead of a lost user.

“But sometimes big data does make a difference, as it allows you to extract the right small piece of data”

At a Southeast Asian cable television operation, ethnic packages were created and targeted solely on demographics: Chinese package for Chinese families, Bahasa package for Malay families, Tamil content for Indian families, etc.  This was safe and avoided selling inappropriate content.  However, we hypothesized that this was needlessly limited. 

By extracting the languages watched (Small Data) from the Big Data of viewing logs, we identified that some Malay or Indian families were viewing Chinese content and vice versa.  The initial reaction of management was that this was wrong, we could not sell Chinese content to Malays or Indians (or the reverse), so they wanted to disregard the findings (which they assumed must have been flawed). 

By the next day, their response was, well maybe our demographic data is wrong for these families.  Maybe.  Or maybe it is a mixed marriage, or a child is studying Chinese by watching content on TV or the maid is watching or …. you get the point, it could be for many reasons.  We didn’t know why they watched but we conclusively knew they did watch content in multiple languages.  So we agreed to test cross-ethnic packages – the result: a substantial increase in the sales rate, which did not surprise me because if you already watch some Chinese content, that is a pretty good indicator that you might be interested in more, regardless of your demographics.

So in the debate over Big Data, better to focus on the right data, small or big.

Mike shared the above excerpt from his upcoming book, “52 Things I Wish Someone Had Told Me About Customer Analytics”, by Alex & Mike Sherman.

By Charlene Cassie

Charlene Cassie is the Conference Director for the Chief Data & Analytics Officer Forum Hong Kong, 4-5 October.  Tune in for more insights into the state of play of data and analytics in the APAC region. For enquiries, email:


There are many specialized conference and meeting room out there which are in high demand. This is because of the aura and atmosphere in the seattle convention center. You must visit their site to get a sneak peek of the rooms.

Post a Comment

Powered by Blogger.