IBM Watson for a day

IBM Watson for a day

On the last couple of Improvement Days we had at Sitewards, I have been working on an internal side project called Hypervisor. Working on the project for couple of months already made me a bit tired of it, so on our last Improvement Day I thought of giving a try to something else, something that I don't work with on an everyday basis. I chose Watson.

What is Watson?

Watson is a question answering computing system that IBM built to apply advanced natural language processing, information retrieval, knowledge representation, automated reasoning, and machine learning technologies to the field of open domain question answering. Wikipedia

IBM says it's an AI platform for business… whatever that is supposed to mean, right? They released quite some APIs which are exposing Watson's capabilities, such as Conversation (for building virtual agents and chat bots), Language Translator, Natural Language Analyzer and Classifier, Personality Insights, Speech to Text/Text to Speech, Visual Recognition, etc. The question is: what do we do with it?

The idea

My plan for the day was to do an investigation and possibly a small prototype of an app that uses the Personality Insights service from the Watson API to analyze emails from a prospective customer. The end goal (if ever reached) is to discover insights about their personality characteristics, values and needs to tailor our marketing and sales approaches.

I spent some time going through the docs before jumping on the API itself, I would highly encourage you to do as well. Otherwise, using the Personality Insights service is very simple. There is only one API method to call and that is /profile. There are different ways to work with the API, the most basic one being via cURL, but you can as well do it through some of Watson’s SDKs. What you get back as a response is a JSON or CSV profile.

Personality Insights profile

The profile contains metadata and characteristics about the author of the supplied input. The characteristics are split in three categories called Personality, Needs and Values. The personality one is the most complex of the three and describes the so called Big Five dimensions, which includes agreeableness, conscientiousness, extraversion, emotional range and openness.

So, going back to what IBM says about Watson being an AI platform for business:

Businesses use the service's personality models to better understand their customers and to design and develop more personalized and targeted campaigns, products, and services. Businesses can easily obtain a list of preferences that are associated with an individual's dominant characteristics and respond accordingly.

Every characteristic comes with a percentile value attached to it. Let's say curiosity is 80th percentile. This of course does not mean that the author is 80 percent curious, but rather that the author is more curious than 79 percent and less than 19 percent of the sample population data.

Now the trick is… how to get the meaning out of those numbers.

While playing around with the API and understanding the meaning of all the characteristics contained in the profile, I was testing the service by using speeches by politicians like Trump, Putin and Merkel. It was a very interesting discovery for me that they share almost the same profile characteristics which sort of proves that the political PR teams share almost the same format of creating speeches and serving crap to the people, but that's a different topic.

Khm… the idea

While this sounds like a great idea, to be able to tailor marketing and sales approaches and to be able to decide whether you want to spend time on a given potential customer, at the end of the day I reached a conclusion that to get some meaning out of the personality profile, we need a lot of data which we can analyze.

This is why IBM suggests using the Personality Insights API for deriving insights from social media and/or enterprise data, because you can have a big enough pool of data that you can analyze. It's almost impossible to get a meaningful result/conclusion/direction for a potential customer with just a couple of emails. My thinking was that we need to analyze vast amounts of emails to gather statistical data on what sort of characteristics are possessed by customers who ended up partnering with us. With the statistical data we will be able to build our own profile format for a potential customer similar to what the political PR teams have for a potential voter.

Afterwards, having this data we can analyze our new customers and compare them to our profile format, but even in this case once again we need enough data on the customer itself (like more emails/phone calls instead of only one), so that this time Watson can give us good enough results. And then the question is: if you exchange 10 emails or have 10 phone calls with the potential customer yourself, do you even need an AI platform to let you know if you should go on? You’ve probably made the decision yourself already. Or what do you think?

This does not mean I've quit on the idea though. I will try to have couple of talks in the following weeks with the guys from our Business Department. I need to find out how much data I can use to get some meaningful outcome from my little experiment. The more, the better!

If everything goes well I will work on it again on our next Improvement Day. I surely had a lot of fun working with Watson that day and I learned some cool things along the way. If I manage to work out something more on this topic, I'll make sure to post again.

Previous article

Abstract controller for Silex

Next article

The power of vulnerability