Jeff Schneider PhD '95
Interview from 2016 Multicast Newsletter
Jeff Schneider received his PhD from URCS in 1995, completed a Post-Doc at Carnegie Mellon University and launched a successful career in machine learning that has spanned both industry and academia. He currently is the engineer lead scientist Uber ATC in Pittsburgh.
Ed Hajim, for whom the Hajim School of Engineering & Applied Sciences is named, always tells graduates to ‘catch the wave’ of new technology. Back in the early ‘90s when you were doing your research and thesis, machine learning was in its infancy. There were only a few known neural net theories in development at the time. Chris Brown, your adviser, said you tried them all to see what would work in your research to train the robotic arm to hit a target. What was it like to be in the “Wild, Wild West” of machine learning and catch the wave? Many of these were just theories, but you were applying them to a real world application. How did you make them work? How frustrating was that trial and error method?
JS: Yes, it really was the Wild West back then. Looking back with what we know now, it was like no one knew how to do anything back then. But that's also what made it incredibly exciting. There were tons of very simple, very successful ideas just waiting to be discovered and we had a big playground to go find them amongst all the other ideas that were not so exciting.
I have many good memories from that process. I think it was the very fact that all you had was trial, error, and really, philosophy, that made it so exciting. I arrived at the University of Rochester because I wanted to study computer vision. After about a year, I remember thinking, “this is just way too hard with the approaches we're using”, and then deciding to do machine learning instead. Of course now machine learning is the key method for doing computer vision and I'm doing that as well.
I also remember the frequent debates I had with the natural language students. I would say it doesn't make sense to invest a lot of effort parsing a sentence for which you don't know the meaning of any of the words in any useful way. I argued that I could give them an unordered set of words and they would understand it much better than if I gave them a parsed sentence without the actual words. They would reply immediately with sentences that were completely ambiguous without the parse and the debate would go on. Of course Bag of Words went on to be one of the most successful approaches to text once the internet boom hit.
Can you point to a turning point in your research that has been especially rewarding for you?
JS: There have been several. The first was the first time my Puma's arm learned to whip its flexible link (a metal meter stick) to throw the ball a good distance accurately. I stayed up all night tending to it while it was doing its own automated trial and error. By morning when Chris arrived it was working and I showed it to him right away. Everyone who saw the video was amazed because it truly learned something it hadn't already been shown. It was really that moment that launched me onto the path as a successful robotics and machine learning researcher. It also cemented my interest in active learning and optimization as the most exciting aspect of machine learning and I've maintained that to this day.
In 1999 my first PhD student, Drew Bagnell, successfully hovered an autonomous helicopter whose controller had been learned from a test pilot's data. That was amazing because prior to that many people believed helicopter dynamics were simply too difficult to learn. The work launched his career and led to even more amazing acrobatics from Andrew Ng and his students.
In 2003 I began working with a biotech, Psychogenics, to use learning to recognize the effects of psychoactive drugs in mice. We built a mouse cage with cameras on the walls and ceiling and force sensors in the floor. We would treat mice with unknown drug candidates, put them in the case, and then ask the learning algorithm: “Does that look like a mouse on an anti-depressant?” To my (and everyone else's) shock, it worked. The system could recognize a dozen different classes of drugs accurately and it is still used to do drug discovery. That experience ignited my interest in doing machine learning for science. I describe it this way: For a long time we wanted to develop learning algorithms that would learn as well as a human. Now I want to develop learning algorithms that learn as well as humanity – that's what science is. That work propelled me into cosmology, materials science, nuclear fusion, and many other scientific pursuits.
During your graduate work at Rochester Andrew Moore was at MIT. Knowing that he was the best person at the time for machine learning, what did it say about your advisor Chris Brown and the flexibility of the URCS program that was you were allowed to spend a semester with his research group at MIT? How did that influence your research and your life going forward?
JS: Ha-ha, an easier question would have been to list the things it did not influence. I went on to do a postdoc with Andrew at CMU during which I co-founded a company with him. He was a tremendous mentor and my primary collaborator until he left CMU to found Google Pittsburgh. We've stayed close ever since and he remains a great professional influence for me.
I cannot say enough about Chris and the URCS program for making this happen. The possibility literally never occurred to me until Chris and I were discussing other people doing research relevant to my work and he suggested I go visit their lab at MIT. He reached out to them on my behalf and gave me the freedom to go visit. Pretty much everything about my career would have been completely different if he had not done so.
After receiving your PhD, you began a Post-Doc at CMU in the Robotics Institute. Simultaneously, you started a company to commercialize machine learning applications called Schenley Park Research, Inc. You held the position of CEO at that company till ’04 but maintained your research relationship with CMU during that period. Did the company continue after you left? Why did you feel it was time to move on?
JS: No, in 2004 we closed the company. We had a dream of commercializing our machine learning work into a machine learning platform that would be used across lots of companies and industries. After 8 years, although we were profitable, we realized we had become a custom software and consulting shop and decided that's not what we set out to do. The funny thing is that we were just a bit ahead of our time. The dream of a universal machine learning platform is still alive and well and many companies are trying to create that today.
It seems like you are always reinventing yourself. For a two-year sabbatical in 2004-06, you worked as the Chief Informatics Officer at Psychogenics, Inc. in Tarrytown, NY. Your responsibilities included assembling the informatics group and leading the development of new machine learning based drug discovery methods. You’ve always had a foothold in both academia and industry. How does that work for you? How do these new involvements inform your research work and move it forward?
JS: I have never been a “pure” academic. I want to see my ideas have a real impact in the world. I see my shifting between academia and industry as a virtuous cycle. As good research ideas mature, I want to carry them through to commercialization. As I spend time in industry, it gives me an updated perspective on what the really big challenging problems and I return to academia with a new research agenda.
The list of companies that you have consulted for span so many different sectors of business and industry. The categories include consumer electronics, food and beverage companies, pharma, finance and the government. What is the common denominator for their need of your services? How are you making their product or organization function at a higher level
Choosing machine learning turned out to be the right choice and the right time. Everyone who has data needs machine learning. And now everyone has data. Data and machine learning are the common denominator for almost everything now. This allows me to work in many different areas and keeps things exciting for me because I always learn something new from each domain. While data and machine learning are the common denominator, the successes area all different. In pharmaceuticals it was the ability to discover central nervous system (CNS) drugs better, faster, and cheaper than ever before. For manufacturing companies it is running their processes more efficiently with better products and less waste. Some companies have developed new products based on machine learning. Others have used it to gain strategic insights into their customers and where their real profit opportunities are.
You are one of dozens CMU Robotics Institute faculty and researchers that recently moved to Uber’s Advanced Technology Center in Pittsburgh. There has been much press about how Uber “poached” the Robotics Institute to help them develop a fleet of self-driving vehicles. What is the relationship between CMU and Uber now? How can academia maintain their highly skilled faculty with the financial competition that industry offers?
JS: The press coverage got a bit out of hand on this one. The Robotics Institute has hundreds of people so the fraction that left was not that large. It was a real feather in CMU's cap. A university's mission is to create and disseminate knowledge and with Uber's new lab, CMU did so very successfully. There is no other university that could have spawned such an amazing new endeavor at this scale. The relationship between CMU and Uber continues to be very positive.
Academia should embrace the future. The old model of a professor, a university, and tenure being locked together for life doesn't make sense in a fast paced high tech world. It tends toward stagnation of ideas more than innovation. Universities should focus on creating a continuous incoming stream of new and returning professors along with the new ideas they bring. Then they can encourage and even facilitate their faculty's engagement with the world outside of academia.
Andrew Moore, Chairman of Computer Science at CMU, has indicated that he encourages his faculty to avail themselves of opportunities in industry for sabbaticals and limited length commitments to help them benefit financially and give them experience applying theories in the world. He hopes that they return to academia with the knowledge they have gained while in industry. Do you foresee that you will return to CMU full time at some point in the future?
JS: Absolutely! The virtuous cycle will continue just as it has before.
(Editor’s note: as of 2018, Jeff is back full time at CMU doing research)
Your work schedule seems grueling, but there must be time for relaxation and recreation. How do you spend your non-work time? Do you have hobbies or passions?
JS: I like to play sports. Back in my UR days, I remember playing ice hockey, basketball, softball, ultimate, and tennis with the other people in the department. Now I'm more into beach volleyball, golf, and cycling. I also like Latin dancing and I'm a big fan of travel – 52 countries and counting!