Active: a unified platform for building intelligent applications

Computers have become affordable, small, omnipresent and are often connected to the Internet. However, despite the availability of such rich environments, user interfaces have not been adapted to fully leverage their potential. In our view, user interfaces will evolve to become more than simple tools that act using a click-and-act paradigm – people want software that can act as assistants to whom tasks can be delegated. This new type of software will provide us with more user-centric systems, able to interact naturally with human users and with the information environment. Although progress has been made in this direction over the last decade, current research in the field has shown that building intelligent assistants is a complex task that requires expertise in many fields, ranging from artificial intelligence to core software and hardware engineering. The difficulty of deeply integrating all the technologies and AI methodologies required to produce robust intelligent assistants has greatly limited the impact and widespread adoption of this form of software. In this thesis, we propose to design, implement and evaluate a new methodology and associated tool suite aimed to ease and accelerate the development of intelligent assistant software. Active, our solution, introduces the original concept of Active Ontologies, and combines it with a service oriented architecture to serve as the foundation for user-centric applications. The Active software suite features a programming editor, a runtime server and associated management tools. Using this unified platform, we developed techniques for rapidly creating intelligent assistant applications that weave together language processing, process modeling and dynamic service orchestration. To validate our approach, three prototypes have been implemented and evaluated. First, we created an assistant that helps mobile users retrieve data and access online services. The system allows users to fetch information through natural language dialog about restaurants, hotels, points of interest, flights status and weather forecasts. As a second prototype, we have implemented an assistant to help surgeons in the context of the operating room. The system uses a combination of voice and gesture recognition to help surgeons navigate through pre-operative information, visualize and control a live video stream coming from an endoscope mounted on a robotic arm. Lastly, we have created a system that helps organize meetings through emails and instant messages with the organizer and attendees. These three prototypes are deployed in very different application domains, and yet are built with the same tools and methods; this demonstrates the flexibility and versatility of our approach. We conducted a user study to validate our claim that, in specific domains, intelligent assistant software can perform better than conventional software approaches. First, we compared our mobile assistant against Google Mobile™, the leading commercial