Home > Professor lights > Vol.04 Masatoshi ISHIKAWA

Vol.04 Masatoshi ISHIKAWA

Exploring the future with
ultra-high-speed,
massively parallel image
processing technology

Masatoshi Ishikawa

Professor, Department of Creative Informatics
and Department of Information Physics and
Computing, Graduate School of Information
Science and Technology, University of Tokyo

Profile Graduated from the Department of Mathematical Engineering and Information Physics, Faculty of Engineering, University of Tokyo in 1977 and completed a master course of Department of Mathematical Engineering and Information Physics, Graduate School of Engineering, University of Tokyo in 1979; Gained a doctor’s degree in engineering at the University of Tokyo in 1988; Then, before assuming his current position, he held successive important posts such as Senior Researcher at the former Agency of Industrial Science and Technology in MITI (now National Institute of Advanced Industrial Science and Technology) and Executive Vice-President of University of Tokyo.
He is the recipient of numerous awards, including the Shiju-Hosho (Medal of Honor with Purple Ribbon awarded by the Government of Japan) in 2011.

Did you know there is a Janken (Rock-Paper-Scissors) robot that always wins and never loses a game? This robot system basically recognizes the 3 hand shapes in the rock-scissors-paper game, and in a mere 1,000th of a second later extends a robot hand in a winning rock, scissors or paper shape that always beats its human opponent. Humans can only recognize about 30 image frames a second and so can’t identify their opponent’s hand shape once they start to extend it during the game. Upon being uploaded to YouTube, this robot became a super-popular robot that got an astounding number of access hits. Here, Vol. 4 of “Professor Lights” in Photon terrace focuses on Professor Masatoshi Ishikawa of the University of Tokyo who creates a seemingly unending stream of startling technology including the Rock-Paper-Scissors robot described here.

Watching birds and building a jet plane

Professor Ishikawa says, “During my student days, in the field of bioengineering, I studied to design and engineer structures that work the same as the human brain. The field of bioengineering involves many areas and attracts researchers interested mainly in the living body and those interested mainly in creating something that work the same as biologic functions. I was drawn to biologic functions.” He was interested not only in making something identical but rather probing deeply into how to reconfigure those functions to the highest possible level and this brought about the creation of things that exceed what a human can do. “I would look at a bird and instead of thinking about how to make a bird, I would wonder how to make a jet airplane.” This is the approach of Professor Ishikawa.

Technologies created from Professor Ishikawa’s research lab cover eight fields, including high-speed robots such as the Rock-Paper-Scissors robot, factory automation (FA) inspection, and the automotive and transportation fields. The high-speed image processing technology applied in all these fields holds the key that exceeds human functions and intellect. Most of these creations have been uploaded to YouTube, so by all means take a look for yourself. Incidentally, the Rock-Paper-Scissors robot has captured the highest number of YouTube hits at 4,150,000 views followed by the “High-speed book scanner” in second place with 960,000 views. Though taking second place, this high-speed book scanner has the astounding ability to scan 250 pages per minute.

Rock-paper-scissors robot with 100% win rate
Rock-paper-scissors robot with 100% winning rate
https://www.youtube.com/watch?v=3nxjjztQKtY

 

Rock-paper-scissors robot with 100% win rate
High-speed book scanner
https://www.youtube.com/watch?v=03ccxwNssmo

 

 

Taking on strange challenges to seek originality instead of playing catch-up with others

The professor’s nice, carefree smile could be seen all
throughout the interview.

Professor Ishikawa has given our world so much exciting technology. We asked him how he spent his childhood years.

“I liked to make stuff but I made stuff that was different from what other people made. For example, when we had a handicraft class in school, I always made the strangest things (laughter). Everyone was “making something pretty” but I had no desire to do that and anyhow “made something different or strange” …”

After graduating from college, Professor Ishikawa believing that “doing something original or strange was roughly the same as doing something new” chose research as his personal road to take. However, at that time Japan was devoting all of its efforts to catching up with technology in Europe and the US and so was trying to win by improving the performance of items that were already being made.

“At that time I thought that playing catch-up with others would probably be boring and wondered if it would be fun to try something else. So the moment I said “No more catch-up!” I decided not to work on something someone had already made. I would build something different. The idea of making something different or something new to me meant the same as the word “research.”

Though a little off the subject here, looking up the word “research” in a dictionary shows the following: “Research comprises creative work undertaken on a systematic basis in order to increase the stock of knowledge, including knowledge of humans, culture and society and the use of this stock of knowledge to devise new applications” (Wikipedia). For a long time in Japan, research totally meant “the investigating of objects.” However, Professor Ishikawa strongly emphasizes that it is also important to include the meaning “creating new value” in the word “research.” He says that, “creating new value in Japanese culture is itself what research is all about. And that is why I pour all my efforts into doing something original or strange” he laughs.

After the war and about the time of the 1970s Japan had finished playing catch-up with Europe and the US. Japan was then working to “create new value” to keep pace with the changing times from the 1990s through 2000 and beyond, because there was almost nothing more to imitate for those running on the cutting edge of technology. However, research in Japan was still stuck in the “the investigating objects” stage, so there was no creative power left to make new things. “At that point I came along and speeded up the process of pursuing new areas with my method of working all out to research strange stuff,” says Professor Ishikawa.

Researchers don’t decide what succeeds what fails — our society does

Professor Ishikawa’s lab utilizes one standard when deciding if someone’s research is really applicable in the world or not. In this process, once something is seen to have possible applications, a prototype is fabricated and shown to the world to decide if it has specific applications. This is the ground rule of the research lab.

Professor Ishikawa tells it like this: “Okay look, this is what POC or namely Proof of Concept is all about. The most important thing our lab is working on right now is how to improve and perfect high-speed image processing technology. We are constantly working to make ever-better parts and components for the world. At the same time, we also show for example “This is what we can do by applying this technology to cars.” This process of demonstrating the feasibility of a technology is what we call POC or Proof of Concept. More specifically, when we develop a technology or product that is not really usable at that point, POC uses a small sample of work to show how much performance we can expect by applying it to a certain field or how convenient it may prove.

Achieving “Proof of Concept” means that a technical concept has been proven as usable in the actual real world.

He says, “In our lab, we make a car POC if we think something is usable in a car or make a robot POC if we think something is usable with a robot. We make a POC for anything such as document scanning machines or bio-application devices. Research needs the POC process because otherwise we wouldn’t know if the research has value or not. A researcher may by himself think something is “good or bad” but will often take the wrong path unless he gets a pure objective view of what he made. Whoever makes a POC will believe his own work is “really interesting” but no one knows which will ring the bell if we look objectively. Honestly speaking, most are not as good as expected (laughter).”

The Rock-Paper-Scissors robot seen at the start of this article was initially not even rated by anyone. Even in his lab, some staff thought that it was too silly and no one would be interested in it. But once people started taking a closer look, it suddenly became a dynamite hit. They really had absolutely no idea what the world would think about it.

“Our basic policy here at the research lab is to make a POC since we don’t know what to expect and then see how the world reacts to it. We are waiting for reaction from the world that pushes us to advance the work.”

The research results listed in this year's report for Professor Ishikawa's lab amount to more than 100 inventions. Besides the Rock-Paper-Scissors robot, electronic book scanning, and projection mapping, many other inventions such as the bipedal running robot and 1ms Auto Pan-Tilt for perfect recentering have been uploaded to YouTube where they have capture a large number of views.

Bipedal running robot (6th place in number of views)
Bipedal running robot (6th place in number of views)
https://www.youtube.com/watch?v=n0xRcFqwnCM
1ms Auto Pan-Tilt for perfect recentering (7th place in number of views)
1ms Auto Pan-Tilt for perfect recentering
(7th place in number of views)
https://www.youtube.com/watch?v=9Q_lcFZOgVo

Suddenly bathing in spotlight due to a change in the society

If the reaction to a POC isn’t good, then they do not make further attempts to develop it. However, some of POC may later on suddenly find themselves in the center of the public spotlight.

“About 8 or 9 years ago, we made the opposite of a batting robot. Batting robots usually can hit 100 percent of all balls thrown at them. We thought if a robot could hit 100 percent of the balls thrown at it, then it could also avoid 100 percent of the balls coming its way. In other words, none of the balls could hit the batting robot, but this wasn’t very interesting (laughter)”.

As might be expected, almost nobody wanted to comment on a batting robot that could perfectly avoid the balls coming its way. But, several years later this robot started to gather a lot of attention.

“This sudden focus of attention on the batting robot was due to a change in robot safety standards. At that time, the safety guidelines specified that enclosures had to be built around robots having motors of 80 watts or more to keep people away from them. However, the safety guidelines have changed so that people can enter the robot enclosure if a robot having an 80 watt motor or higher definitely stops operating when a human came close to it. So robots that could accurately track people’s movement and stop operating would be a good thing.”

So, unwanted technology at the time the POC was first proposed may suddenly transform into highly valued technology after you wait long until the society needs change. Such an outstanding change occurs occasionally. “Then in that case, we introduce a research result obtained 10 years ago as if it is a recent development. Of course we don’t want to tell a lie, so we simply don’t mention when this technology is made,” says Professor Ishikawa. “But people will soon find it is old, because these days everyone uses high-vision to make videos yet our old video used the 4 by 3 aspect ratio….” he adds with a laugh.

Making skillful use of social media and getting the world’s opinion

The method Professor Ishikawa applies at his research lab of creating a POC and awaiting an evaluation from the world requires first of all exposing that technology to society in order to see what they think of it. To do that, he makes use of social media such as YouTube and Facebook.

“We got a really strong reaction from the US. It takes about a month to get some kind of response from Japan, but Americans aren’t shy about praising surprising new things. When the Rock-Paper-Scissors robot first came out, we got 3 million access hits in the first one week to 10 day period, but almost all of these were from Europe and the US and with Japanese viewers not even making up 10 percent of the total.”

Analyzing this response also shows that…

“A response of 3 million views means that you will receive responses from about 0.1 percent of those viewers or in this case about 3,000 mail messages. Among all that mail about 10 percent or 300 mails will actually be worth reading. And a further 10 percent of those 300 mails or namely 30 mails will actually say something important. In other words, they’ll say something like “How about if you try improving it here?” So some 3 million hits will give about 30 meaningful responses,” says Professor Ishikawa. Interpreting these kind of rough figures is something he learned by experience.

By the way, can the response to a video be good or bad depending on how well the video is made?

He says, “Even if the video has good research content you won’t get much of a response if it was poorly made. On the other hand, if the video looks good you may get a tremendous response even if the content is not so good. In the case of pure science, posting a very elaborate video to YouTube may hurt your credibility, so it won’t be well received by others.”

In the video of the tremendously popular Rock-Paper-Scissors robot, one only sees a person and a robot endlessly playing rock-scissors-paper and the robot scoring continuous wins. The editor of this video did almost nothing. But this easy-to-understand scenario is something people instinctually find fascinating and links to the good response it received.

Rules for doing something strange

Professor Ishikawa’s research lab has a special method for creating something new or something strange. His rule is not to “find a problem and think of a solution” which is the conventional method but rather to find something that wasn’t there until now.

“The moment you think you find a problem or issue, there is no big business there. For example, trying to create a big business like Facebook or Google is nearly impossible unless you find “something not there until now” as a first step. Our research lab has a number of hidden secret methods to find “something not there until now.” For example, if told, “the challenge is to reach a difficulty level of 10,” then “reaching a difficulty level of 10” becomes the issue, so even if you reach a difficulty level of 10 you still can’t create a big business. You might be able to start up a small business but that still won’t help you run with the world’s top competitors. So what do you do? You change the difficulty level 10 to 100 without any meaning.”

Setting the difficulty level from 10 to 100 without any meaning, of course, something no one else is doing and also requires making a qualitative change. So Professor Ishikawa’s advice is that, “boldly setting the difficulty level from 10 to 100 means you can reach a whole new area.” For example, if the performance of an automotive component is 8 and the next performance goal is 10, then you may easily arrive at a solution. But if you set the target performance to 100, then you have to start coming up with concepts completely unlike anything currently available. The result of your effort may possibly extend to an entirely new breakthrough that is not simply an extension that builds on what someone else has done.

Using light with ultra-high-speed and parallel processing properties in a system

Professor Ishikawa has achieved astounding results not seen until now ranging in all kinds of directions on unexplored areas of science. What does he think of “light” that is the theme in this “Photon terrace” website?

He says, “Because our research lab is currently working on high-speed image processing, our goal is to process two-dimensional information as is in parallel and output the result. Light plays the main role in this case. The processed results are sent to the computer and ultimately converted to electrons. I used to assert that it was better to use light in the processing up to linking to a computer. That was found to be wrong and now we are attempting to do that by electronic processing.”

Example of dynamic projection mapping
Example of dynamic projection mapping
https://www.youtube.com/watch?v=p7IL0Gvux7U
The reason they are trying to separately process two-dimensional information with light and do subsequent arithmetic processing by electronics is that there seems to be a relation between light and general versatility. “No matter how hard one tries to achieve general-purpose processing functions with light, those general-purpose properties do not seem to appear. We attempted this in the past and produced some general versatility by applying learning functions at the time. But the lower versatility of optical devices cannot compete with electronic device performance. So I think the best way to go in applying the properties of light is the two-dimensional parallelism capable of processing two-dimensional information in parallel."

 

The dynamic projection mapping is one example of using “multi-purpose” light by applying two-dimensional parallelism. Projection mapping used up until now was a display of images on stationary objects at Disneyland or Tokyo Station. However, dynamic projection mapping is capable of displaying images on moving objects. For example, dynamic projection mapping can project the state of a self-rotating sphere on a white ball that is moving up and down and left and right.

“We did this by creating an optical system capable of shifting the optical axis. So light that up to now could only be projected on fixed objects could now be projected onto moving objects, making it more available for “multi-purpose” uses. Rather than flexibility in numerical processing this is more like flexibility or versatility on physical structures. I think it better that we should shift our use of light to these kinds of mediums” he comments.

(Left-hand photo) Dynamic projection mapping device. The camera located at the highest point tracks the ball while the computer does ultra-high-speed, massively parallel processing. The image tracking of the ball is projected by feeding back the ball position information to a mirror in the lower section (Right-hand photo).

Professor Ishikawa goes on to say that along with parallel processing, high speed is what makes it possible to express the true capabilities of light.

“Though it may take second place to electronics in terms of versatility, light is unbeatable in terms of high speed. Our research lab succeeded in creating a projector capable of showing 1000 frames per second and we are trying to succeed by using that high speed everywhere. We are in effect using the capability of a system to extract the superb high-speed property of light.”

Once you learn the principle — start thinking about what you can do with it

Lastly, we asked Professor Ishikawa to give a few words of advice to junior and senior high school students who will play a vital future role in our society.

“What I want to say is how about giving some thought to changing our lives or changing our world through light? How can we change the world by using light? Well there are hundreds of ways we can change it. If you start thinking how to solve “something strange” among those hundreds of ways, then that process is the start of new research.”

The enthusiastic look on Professor Ishikawa’s face as he pushes forward on his ideal road to science is truly impressive.

The enthusiastic look on Professor Ishikawa’s face

as he pushes forward on his ideal road to

science is truly impressive.

Professor Ishikawa continues by commenting, “If for example you teach Newton’s motion equation in school, then you should always add words describing how to use this motion equation. In other words, when you teach a new scientific knowledge, you should teach the principal along with how to apply it or teach it with the aim of developing its potential applications. I think that is what is missing to today’s education process in Japan.”

He says, “I would like to see schools teach the fact that new knowledge gives birth to new value. I mean that schools should not only teach the nature of things from a scientific viewpoint, but also include an engineering viewpoint that tells us what to use things for and how to use them.That process generates Proof of Concept or POC. A number of these POC efforts will end in failure but some will succeed and help carve out a new age in science.”

The videos on this page are available on Ishikawa Watanabe Laboratory channel on YouTube.
https://www.youtube.com/user/IshikawaLab