Facebook founder and CEO Mark Zuckerberg has given a sneak peek into Jarvis, the artificial intelligence personal assistant he has been building this year. With the Jarvis project, Zuckerberg wanted to use his voice to control everything in his house, from the music to the light, to the temperature. He also wanted to operate the front gate for friends and family by recognizing their faces. His first build over the project was homemade and has been tested at his house.
To show that it is just not on paper, Zuckerberg has released a video in which his family is interacting with Jarvis to complete their daily task to make living better and more tech oriented.
How Jarvis Works
Natural language processing and facial recognition capabilities were built into Jarvis, enabling it to understand spoken or text commands and recognize who is issuing them. The software build can determine guest at the door, operate all the electronic devices and is capable of doing thing on request.
Zuckerberg said, “One aspect that was much more complicated than I expected was simply connecting and communicating with all of the different systems in my home. Most appliances aren’t even connected to the internet yet.”
According to him assistants such as Jarvis would not only need devices in homes to be linked to the internet, they would have to run on common standards.
Zuckerberg faced issues while testing “Jarvis” – he found out that command given by him were executed, but when tested by his wife, it gave a different feedback.
Similar problems could arise when asking for music to be played without the AI being told which room, especially if a napping baby is a factor.
The more context AI has, the better it can handle open-ended requests, he noted.
“At this point, I mostly just ask Jarvis to ‘play me some music’ and by looking at my past listening patterns, it mostly nails something I’d want to hear,” he said.
Zuckerberg said he tends to text his Jarvis using a ‘bot’ he built for the Facebook Messenger service instead of speaking to it, mostly not to disturb people around him.
Biggest challenge faced by Zuckerberg was to write code for different system that speak different languages. He said “ We use a Crestron system with our lights, thermostat and doors, a Sonos system with Spotify for music, a Samsung TV, a Nest cam for Max, and of course my work is connected to Facebook’s systems. I had to reverse engineer APIs for some of these to even get to the point where I could issue a command from my computer to turn the lights on or get a song to play,”
The voice for his AI Assistant “Jarvis” is given by Morgan Freeman which is a pretty big thing for Zuckerberg himself.
Here is another teaser :
Zuckerberg went on to explain the needed to make this system more interactive and connected. According to him the industry needs to develop common APIs and standards for the devices to talk to each other, the most difficult part building Jarvis was to have him understand his commands as well as his wife’s.
Zuckerberg concluded that he has previously predicted that within 5-10 years we’ll have AI systems that are more accurate than people for each of our senses — vision, hearing, touch, etc as well as things like language.
“It’s impressive how powerful the state of the art for these tools is becoming, and this year makes me more confident in my prediction.” Zuckerberg said.
“In a way, AI is both closer and farther off than we imagine. AI is closer to being able to do more powerful things than most people expect — driving cars, curing diseases, discovering planets, understanding media. Those will each have a great impact on the world, but we’re still figuring out what real intelligence is,” he signed off, adding that he would be taking up a new challenge in the coming year.
