Maths and Data

Mathematics……. do I hear groans and shudders all around? This was the subject that was the bane of our existence, probably from age 0 – 20 (or 30 if you are a Ph.D). This was also the subject which skewed down our grades!

Well, Mathematics is the darling of the world now – and I don’t just mean in designing exotic derivatives but for everyone from Facebook to UPS to Exxon Mobil. I am talking about Data Sciences and Algorithms.

Let me start by recounting a recent incident at a Target store in the US. Every time we go shopping, we inadvertently communicate a lot about our likes, dislikes, age, preferences, even our social status and education. Target assigns every customer a guest ID number which is linked to the customer’s name, email ID, phone number and so on. Now this guest ID is ‘tagged’ with the customer’s purchases, buying patterns, etc.

Charles Duhigg in his NYT article recounts his interview with Andrew Pole, a Target statistician:

“[Pole] ran test after test, analyzing the data, and before long some useful patterns emerged. Lotions, for example. Lots of people buy lotion, but one of Pole’s colleagues noticed that women on the baby registry were buying larger quantities of unscented lotion around the beginning of their second trimester. Another analyst noted that sometime in the first 20 weeks, pregnant women loaded up on supplements like calcium, magnesium and zinc. Many shoppers purchase soap and cotton balls, but when someone suddenly starts buying lots of scent-free soap and extra-big bags of cotton balls, in addition to hand sanitizers and washcloths, it signals they could be getting close to their delivery date”

“As Pole’s computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a “pregnancy prediction” score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy.

One Target employee I spoke to provided a hypothetical example. Take a fictional Target shopper named Jenny Ward, who is 23, lives in Atlanta and in March bought cocoa-butter lotion, a purse large enough to double as a diaper bag, zinc and magnesium supplements and a bright blue rug. There’s, say, an 87 percent chance that she’s pregnant and that her delivery date is sometime in late August.”

Over a period of time, Target became good, really good, at predicting what the customer could need. So what happened next is both funny and scary at the same time. A shopper called a Target store manager complaining that his daughter, who was still in high school was getting advertisements for baby products. He wanted, very angrily, to know if Target was encouraging high school girls to get pregnant. The store manager immediately apologized and assured him that this will not happen again. The store manager was so worried that he called the shopper again after a few days to apologize again…guess what.. his daughter was pregnant!!!! Just that he did not know it then.

To my mind, nobody is as good at this stuff than Try browsing some books on Amazon for a few days – browse only the books you like of course. The next time you visit, you will notice a section called ‘Recommendations for you’. These recommendations are excellent. I myself have gotten to a stage that if I feel like doing some book buying, I go to Amazon, then search books that I have already read and liked, and then wait for the recommendations to come up. At least 8 times out of 10, the recommendations are bang on!!

The driving factor is the explosion of data in the world – our Facebook posts, shared Pinterest posts, instragram photos, blogs, and so on are all part of that data. For instance, Facebook intends to start targeted advertising based on your posts. Say you post a update “Can’t wait for my holiday” – Facebook will then serve ads for cheap airline tickets or car rentals or swimsuits and so on. With 400 million users, Facebook can’t do this manually – enter algorithms and data analysis! This sort of data analysis or ‘Data Sciences’ analyzes large data sets and spots patterns and even ‘learns’ consumer behavior by analyzing this data. This analysis is also useful is predicting behavior or needs, like the pregnant teen at Target.

I am not getting into the mathematics of data sciences, it is beyond the scope of this post (and honestly, beyond the scope of knowledge), but Maths has probably never been so important.

Now the problem is that for the past few decades, unlike engineering or business, not many people have actually been taking up Applied Mathematics as a career. According to McKinsey in a Gigacom article, “by 2018, the United States will have a shortage of “1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions,” and a shortage of almost 200,000 people with the deep analytical skills necessary for data science.” And of course, our wonderful primary and secondary education system will ensure that India will always be short of mathematicians.

So if you are one of the rare ones, get ready….your life is about to get a lot better…

2 Responses

  1. More than math, it’s good level of understanding in statistics that’s missing, if you’re talking about ‘big data’ that is.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to Top