somatic blog

machine learning

On AlphaGo, intuition, and the master objective function

By jason

It is another great milestone in Artificial Intelligence that computers are now able to beat humans in the game of Go. The DeepMind/Google team came up with a combination of new algorithms that effectively reduced the search space of possible positions-1,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000, 000,000,000,000,000,000 to be exact- to a much smaller number that a computer could work with.

What I'm more excited about is that Deepmind/Google are publicly talking about intuition, and trying to build systems that sort of emulate it. In their blog post, they mention "The game is played primarily through intuition and feel".

They then go into detail on how they constructed a machine learning system that reduces the amount of positions the system needed to evaluate at each turn. Their system basically makes estimations of expected outcomes instead of directly calculating directed outcomes, thereby kinda sorta simulating intuition.

What is intuition?

So what is the definition of intuition? According to wikipedia it is "a phenomenon of the mind, describes the ability to acquire knowledge without inference or the use of reason."

Talking about this subject in relation to artificial intelligence has been a sore spot. Can intuition exist inside of a computer? The truth of the matter is we have no idea, people have been debating this since the beginning of computers and even much earlier with Descartes and others.

Another core problem is the name and definition of intuition. Should we call it intuition, emotion, gut reactions, lizard brain, or feelings? And, what exactly does that mean. We have been trying to objectively define this for a long time.

There is a specific definition that I have been using for a while is based off one of favorite books- Thinking, Fast and Slow.

The author, Daniel Kahneman, has done studies of the mind for decades, and came to the conclusion that the mind is running 2 different algorithms:

  • System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious
  • System 2: Slow, effortful, infrequent, logical, calculating, conscious

System 1 is a machine for jumping to conclusions. It is a heuristic based system that reacts to incomplete information and stimuli fast enough to navigate in our world in real time given all the uncertainty. It gives answers that are typically good enough, but can make mistakes given the small amount of information it processes. Escaping an animal trying to attack us, swerving away from a car that suddenly stops, looking towards a bomb explosion, adding 1+1. Those are all examples of System 1 processing.

System 1 vs System 2

System 2 on the other hand is a completely different system. It is a slow and deliberate system that does things like complicated math, answering tests, writing a novel. System 2 can override System 1 when it senses that the answer from System 1 is wrong. It is presumed to reside in our neocortex and as far as we know, only humans have System 2.

It seems that humans and ALL living organisms have system 1. If we assume the System 1/System 2 combination is how our brains are working, then I would make the statement that we have already built an awesome System 2 in computers. All the current machine learning algorithms we build are based off logic and calculations which is what System 2 is all about. But System 2 cannot really exist by itself, it needs guidance.

Objective Functions

So all of these machine learning systems are always optimizing for a specific objective function. Depending on the algorithm and problem you are trying to solve, you use different algorithms and objective functions. With classical machine learning applications like regression and classification, the algorithm is trained on training data. The objective is to have the machine get as many answers correct on a previously unseen validation set.

training

On DeepMind's video game system, they hook up the algorithm directly to the score of the video game and so the system is optimizing for winning in the video game.

AlphaGo's objective function is to win at Go, and that is it, it can't do anything else.

Notice how each of these machine learnings models have a single fixed objective function.

And what is the objective function of general living organisms? Is it to eat? To reproduce? To live as long as possible? To be "happy"? To achieve homeostasis? How about for a human? Is is the same as other living organisms? If living organisms have a single objective function, it would seem like they all have different objective functions.

And what is my objective function as I write this blog post? I have the overarching goal of wanting to publish my thoughts on AlphaGo. As I'm writing this, my intuition on the direction of how I should write this keeps changing. How do we put this kind of intuition into a computer- Think about what you do in your daily life? How do you go about deciding what to do next?

if we want to get to general artificial intelligence,I believe that a heuristics-based reaction system, or System 1,should replace the objective function of a machine learning algorithm. In fact, they are the same thing.

The Master Objective function

We are still in the early stages of extracting value out of deep learning. We have already surpassed many milestones with deep learning and will continue to do so for a while. Once we have extracted everything we can out of it and reached a plateau, I think the lesson will be that we can build almost any deep learning model where we can turn the problem into: enough data -> deep learning algorithm -> single objective function.

We will succeed in making self driving cars that move us from point A to B efficiently, computers that play video games, models that can distinguish dogs from cats,machines that can draw faces, and many other interesting models. For us to get to the next level, I think we must seriously look into building models that have dynamic objective functions. I believe that objective function will look something like the system 1 where it is evaluating in realtime how to navigate its environment. It will not be a function that optimizes for one thing, but be dynamic and be able to change directions, unlike current machine learning algorithms that have one single objective. I wish I knew exactly what it looks like, but I don't know. There are already some amazing people exploring this space and it is something I will explore in a future blog post.

If we don't do this, we are going to continue the trend of making "breakthroughs" with machine learning systems that can win a single task, but will FAIL when it does anything that deviates from the task it was trained on.

Closing remarks

AlphaGo is a nice milestone and PR piece. Bravo for DeepMind for starting to think about the problem of intuition. If they really want to "Solve Intelligence" and find the master algorithm that powers general intelligence, start building the master objective function. Expect to see a lot more experimentation in building machine learning models using intuition. I encourage machine learning researchers to think more about how to build these intuition based machine learning system and try new experiments. Please contact me if you have ideas or want to collaborate.

Add AI powered image effects into your apps effortlessly

Start now