Smart Handouts: Personalized e-Presentation documents
Berna Erol, Dar-Shyang Lee, and Jonathan J. Hull
Ricoh California Research Center
2882 Sand Hill Rd. Suite 115, Menlo Park, California, USA
{berna_erol, dsl, hull}@rii.ricoh.com
ABSTRACT
A novel system is described that significantly enhances the usefulness of handwritten notes taken during a presentation by creating a multimedia document that includes scanned images of handouts, personal notes, and links to a multimedia recording of the presentation. Notes are linked to the e-presentation media with automatic content analysis without any special notes capture device. Layout segmentation and template matching automatically detects the presence of presentation handouts during scanning. Presentation-level and slide-level linking of handouts to e-media use text and image features from slides. Experimental results show 95% accuracy in linking of the scanned handouts to the e-presentation media.
1. INTRODUCTION
Many e-presentation capture systems have been developed [1, 2]. These systems capture audio and video, presentation slides, notes, and whiteboard data, resulting in media-rich documents. Today, capturing multimedia data is easier than it has ever been and the real challenge is in making it accessible to users. Most of today's e-presentation systems support web-based access and provide cross-indexing between slides and an audiovisual recording of a presenter. While a web-based e-presentation playback interface can be useful for people who are motivated to search, browse, and access recorded presentations and lectures, e.g., a student right before an exam, it is not as useful in a corporate environment.
When people look for specific information or review a presentation, their own personal notes are often excellent starting points for retrieval. In our more than four years of research and regular use of an e-presentation capture system in a corporate environment, we observed that even though an electronic note taking system was available, users generally preferred to take notes on presentation handouts.
In this paper, we present a method that allows a user to take notes on presentation handouts with a regular pen, and after the presentation is finished, scan the handouts and invoke a process that automatically creates a pdf file that contains links to a multimedia recording of the presentation. Referred to as a Smart Handout, this is a personalized e-presentation document which shows the presentation slides, notes, metadata, and has media links to the electronic presentation recording as shown in Figure 1. When a user needs to review what occurred when a particular slide was presented, he can simply click on a slide or a video key frame in the Smart Handout. This invokes the webbased presentation playback interface and starts the playback from the time associated with the slide or key frame. Additional metadata present on the handouts, such as the Q&A activity and key frames, helps the user recall the presentation and efficiently navigate to the point that interests her. The fact that the user does not need to access an additional interface for searching for a particular presentation recording significantly reduces the effort needed to access e-media.
This paper describes a novel algorithm that automatically generates Smart Handouts. A new layout segmentation and template matching algorithm automatically detects whether a scanned document is a regular document or a presentation handout. A presentation matching algorithm, which is based on OCR and n-gram matching, retrieves the presentation recording where the slides in the handouts were presented. Linking of the scanned slides with slides captured in a presentation recording uses edge histograms. Users' handwritings on scanned slides is detected and segmented for better matching accuracy.
The rest of the paper is as follows. In the next section we give an overview of the related work in the literature. Section 3 presents our method for automatic generation of Smart Handouts. Experimental results and conclusions are given in Section 4 and Section 5, respectively.
2. RELATED WORK
There are e-presentation/e-learning capture systems described in the literature that allow cross-referencing of user annotations and notes [2]-[8]. In some of these systems, notes are entered on a computer or a PDA that timestamps each typed note [3][5]. For handwritten notes capture, Tablet PCs, PDAs, and special pads are used in systems such as [6][7]. These note-taking systems synchronize notes with the e-learning media, but require the use of specialized devices. These devices may not be accessible by all users at all times thus creating a usage barrier. In fact, a survey we performed of 21 participants revealed that For review purposes only. Further dissemination of the content is prohibited. the majority (>80%) of people prefer to take notes on paper presentation handouts rather than on a laptop or a PDA.
