When you drop a coin and it falls into a patch of grass near a parking meter, if you are a sighted person you simply scan the grass until you find the quarter, pick it up, and use it to feed the meter. You don’t need to consciously control the process. Your eyes and your brain do their own thing and before you know it, you have found the coin.
Creating a computer program capable of this same feat is remarkably difficult. A quarter hidden on its edge among blades of grass at random angles with differing brightnesses presents a currently-insurmountable problem for computer vision programs—especially if time and computing resources are limited.
Bharath Hariharan, who joined the Cornell Computer Science faculty as an assistant professor in July of 2017, wants to use deep learning methods to help computers learn to see more like people do. “I got interested in computer vision as an undergrad,” says Hariharan. “I started to see that what was a simple, unconscious process for most humans was incredibly difficult for computers.”
Hariharan earned his undergraduate degree from the Indian Institute of Technology in Delhi and while there landed an internship with Microsoft Research India. He was involved with a project on multi-label learning for computer vision. “When it came time to make a decision about grad school, it was easy,” says Haraharan. “I didn’t even put a resume together because I knew I wanted to get a Ph.D., not a job in industry. I am an idealist and it seemed the best way to make a positive change in the world was to take the academic path.”
Hariharan’s undergraduate advisor at IIT, Manik Varma, recommended he apply to work with Jitendra Malik at UC-Berkeley, and that is exactly what Hariharan did. At Berkeley, Hariharan worked on the problem of instance segmentation. In his thesis he defined the problem and its evaluation metrics and then he proposed a solution to the problem. The problem he examined was how to train a computer to sharpen the boundaries around objects in photos in order to map the objects’ precise locations.
Midway through his doctoral work, deep learning came along and changed everything. Hariharan continued with his work, but knew that his next project would explore ways to use deep learning to improve computer vision. Once he received his Ph.D. from Berkeley, Hariharan joined the Facebook Artificial Intelligence group for two years as a postdoctoral researcher. At Facebook, he had time to define his own problem.
“I thought about things for a couple of months and I realized that deep learning methods get better and better with more training data,” says Hariharan. “So I decided to look at what happens when you don’t have enough training data.” Hariharan understood that the machine would then need the ability to learn from less data—it would need to be able to do some unsupervised learning.
And this is the work Hariharan is continuing now that he is at Cornell. “We take vision for granted because we learn it naturally,” says Hariharan. “We want to replicate the ability humans have of learning without labeled examples.” Hariharan is experimenting with using still images, video data sets, and simulations to train computer vision. He is also looking at the idea of metalearning---or, learning about how we learn---to see if ideas can be incorporated into computer vision to help it learn better.
Hariharan’s goal is that someday computer vision will be something we take for granted. “I want to make it so that anyone can use CV anywhere,” says Hariharan. “It will be on everyone’s phone. Traditional machine learning has already reached this point; we want to make CV reach this same point.”
Hariharan, who is teaching a graduate course in computer vision in the fall of 2017, says he came to Cornell for two big reasons: “First, the Computer Science Department here is so strong and helpful and collaborative and welcoming. Mentorship and advice are forthcoming and I will learn so much from my colleagues. Second, Cornell attracts amazing students. I am looking forward to teaching and having a positive impact on students and on the field.”
When Daniele Visioni was six months away from finishing his physics Ph.D. studies at Italy’s University of L’Aquila he took a long bus ride to Zurich, Switzerland to give a talk at an academic... Read more about Assessing the feasibility (and advisability) of using Sunlight Reflection Methods
Back in 2008 when EAS Assistant Professor Riley Culberg was an undergraduate at the United States Military Academy at West Point, it is unlikely he would have predicted he would end up on the faculty... Read more about Using radar to figure out where glacial meltwater goes
When Grace Barcheck was a sophomore at Washington University in St. Louis, she took a geoscience class that changed her life. You can trace a direct line from that class to her current position as one... Read more about Studying plate tectonics with new tools--and a new perspective
