With degrees in both computer science and business, along with a PhD in marketing, you may wonder why Shasha Lu, University Lecturer in Marketing at Cambridge Judge Business School (CJBS), is spending so much of her time watching the way women evaluate clothes. But imagine trying on a garment, only to find that when you’ve put it on, a computer is able to read your facial expressions and body language so that it can make recommendations for other items of clothing you might like. If Lu has anything to do with it, this will become a reality.
Visual data has become a main source of corporate and public (unstructured) big data and the focus of Lu’s work is how to harness the power of these images and videos to mine valuable information by combining image/video analytics with current marketing models. “Ultimately,” she explains, “it’s about gaining better customer insights to solve business problems and improve business practice.”
As a new domain in marketing research, Lu explains that it’s all about combining computer science techniques with existing marketing models to develop managerial tools for business. “It was during my PhD that my interest in artificial empathy – automatically inferring people’s internal states based on the information they emit – really took off,” she says. “My supervisor at the time, Professor Min Ding, was really interested in this area and I didn’t take much convincing to share his enthusiasm because I’d always been so fascinated by artificial intelligence.
“My father worked in business when I was little, so I was influenced by that. Plus, I was a great fan of science fiction, especially the innovative stuff like time machines. The idea of getting a machine to come to useful conclusions from someone’s expressions and behaviour was therefore massively exciting to me and we now have the technology and computing power to actually make this possible.”
Moving to CJBS to further her interest in leveraging image and video data was a huge move, she admits. “I came from Shanghai, which is a very big city, similar to London in that it never sleeps. Cambridge, on the other hand, provides a much quieter, small-town life, but I love it.”
Among Lu’s main interests outside work is fashion, which has made her clothing project – entitled A Video-Based Automated Recommender System for Garments – even more motivating.
“When people, especially females, go out shopping, we tend to try a garment on before making the final purchase. When we do so in front of the mirror, the sales person often observes two key pieces of information. The first is whether the customer likes it or not, which is inferred by her emotional response from her facial expression. The second is which particular part of the garment she likes or dislikes, which is observed through behavioural response. If she’s touching her collar or scratching the lower hem with an annoying expression on her face, for example, we may infer she doesn’t like that bit of the garment.
“Our project uses a camera mounted on top of the mirror – placed outside the changing space – to capture the evaluation process and analyse facial expressions and behavioural responses. Then we combine it with data showing what other customers’ reactions are to the same garment in order to make recommendations to the current customer in real time. In addition, the store can use other pieces of information, such as height and body shape, to make better recommendations or provide better service.”
Customising movie trailers is another project that Lu is involved in. “This is still ongoing, so we don’t have concrete findings yet, but as it stands all move trailers target the same group of people; if there’s, say, a trailer with an action element that’s watched by someone that doesn’t go in for action films, they may well be put off seeing it, whereas in fact the movie might have plenty of non-action that makes the film attractive to this viewer,” she explains.
“We know that it’s possible to analyse the content of movies to extract all kinds of information – lighting, colour, motion and shot, for instance – all of which relates to people’s preferences, and we are working on using this to customise movie trailers according to people’s preferences, thereby improving the effectiveness of the movie trailer and getting people to watch movies they might otherwise dismiss.”
Even the nature of online dating may be influenced by Lu’s work in the future. “We know that certain information about someone’s face in an online profile picture has an impact on the person looking at them. That could be anything from the distance between the eyebrows and nose or to the distance between the nose and mouth. We propose to analyse people’s preferences in order to provide suggestions of candidates that match the types of faces the person is most likely to go for.”
Currently, says Lu, there’s a relatively small group of academics involved in using image and video data in the business world in this way, albeit on a global basis, with growing interest at Harvard Business School, where work is being done, for instance, on inferring people’s preferences for television commercials. “But it’s a growing area,” she says.
Marketing, points out Lu, can feel theoretical and distant, whereas the work she’s involved in is very accessible. “Not only is this area of work innovative and full of potential, but it’s about the everyday aspects of living, and it’s this combination that makes it a really exciting area to be involved in.”