PLOW: Procedure Learning On the Web

Overview

The goal of the PLOW (Procedure Learning on the Web) is to build a system with which a user can teach the computer to perform tasks on the web. PLOW learns from both explicit demonstration of the task together with natural language instruction. The natural language play by play provides key information that allows rapid and robust learning of complex procedures including conditionals and iteration in one short session. PLOW demonstrates the power of an integrated approach to learning, combining deep natural language understanding, reasoning and machine learning.

Papers

A good overview paper describing the system is PLOW: A Collaborative Task Learning Agent, which won an Outstanding Paper Award at the Twenty-Second AAAI National Conference on Artificial Intelligence (AAAI-2007) in Vancouver, BC, Canada. Additional papers are available from our main Publications page.

Videos

The first demonstration of PLOW demonstrated the basic techniques by learning fill in a requisition form for purchasing a book, where the system must learn to look up the process and ISBN number. [QuickTime | MP4]

The second version of PLOW introduced the ability to learn iterations using mixed-initiative interaction where the system attempts to infer and complete iterations for the user as it is being taught. As a result, PLOW can learn iterative procedures without seeing even one complete trace of the task in execution. Traditional machine learning techniques would typically require many complete traces in order to reliably infer an iterative construct. [QuickTime | MP4]

The next version of PLOW (November 2007) includes significant improvements to the graphical interface as well as new capabilities such as the ability to understand information extracted from pages. The graphical display helps the user understand what PLOW is doing and what it has learned, reducing the need for corrections and improving the learned procedures. The ability to understand extracted elements allows PLOW to learn more complex tasks involving online analysis, such as only keeping results whose distance is below some threshold (as illustrated in the clip). [QuickTime]

PLOW has also been ported to work on text-based terminal interfaces, instead of Web Browsers. This video shows the PLOW for Terminals (PLOT) system and its use on a terminal system (CHCS) which the US Military Health System uses for booking appointments. [QuickTime]

Documentation

Documentation related to PLOW development has been relocated to the the docs directory.


Last change: Jan 29, 2009 by blaylock