I work on understanding natural language semantics by using commonsense background knowledge to bridge between different representations. I developed knowledge collection and analysis tools for the Open Mind / ConceptNet project. For my Masters thesis, I made a code reuse system that understands the relationship between blocks of code and natural language descriptions of what the code is for. You can describe what you want the code to do, and the system finds existing code that other people wrote when faced with similar challenges.

An interface for targeted collection of common sense knowledge using a mixture model. Robert Speer, Jayant Krishnamurthy, Catherine Havasi, Dustin Smith, Henry Lieberman, Kenneth Arnold. Proceedings of the 13th international conference on intelligent user interfaces (IUI), February 2009.

Learning hierarchical plans by reading simple English narratives. Dustin Smith and Ken Arnold. Presented at the Commonsense Workshop at the ACM International Conference on Intelligent User Interfaces (IUI), February 2009.

Our team of 4 took two webcams and a Cell processor and made a full software stack for interacting with a computer by hand gestures. We built a SIMD-optimized stereo object tracking system on the Cell processor, a networked video processing pipeline, and applications for drawing, presentation, and map navigation. We also submitted 11 patent disclosures for gesture recognition applications.