US20120189268A1

US20120189268A1 - Picture-based video indexing system

Info

Publication number: US20120189268A1
Application number: US13/437,308
Authority: US
Inventors: John G. Posa; Barry H. Schwab
Original assignee: INMOTION IMAGERY Tech LLC
Current assignee: INMOTION IMAGERY Tech LLC
Priority date: 1995-11-02
Filing date: 2012-04-02
Publication date: 2012-07-26
Also published as: US6574416B1; US20030206716A1; US8150239B2

Abstract

A video indexing system uses pictures representative of a recorded video program to assist a user in determining the contents of a recorded medium without having to view the program itself. The pictures preferably represent segments of the program which are spaced apart in time, enabling a relatively small number of such pictures to characterize a lengthy program, and are presented in separate windows on the screen of the same device used to display the video program. The pictures may include still or moving imagery. A viewer optionally may select a particular picture with a pointing device, to commence replay of the recorded program from that period in the program, or to recall stored audio information so as to assist in identifying the selection. The picture information may be stored on the same medium as that used to record the video program, or a different medium may be used.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. patent application Ser. No. 10/417,300, filed Apr. 16, 2003, which is a continuation of U.S. patent application Ser. No. 09/191,573, filed Nov. 13, 1998, now U.S. Pat. No. 6,574,416, which is a continuation-in-part of U.S. patent application Ser. No. 08/556,746, filed Nov. 2, 1995, now U.S. Pat. No. 6,526,219. The entire content of each application and patent is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates generally to video recording, and, more particularly, to a system for storing and displaying “thumbnail” images representative of the contents of a video recording medium, thereby enabling a user to locate a particular section rapidly and conveniently.

BACKGROUND OF THE INVENTION

As the popularity of video cassette recorders (VCRs) has spread, many users have amassed extensive libraries of recordings, representing a wide range of program materials collected over an extended period of time. In most cases, users are faced with the alternative of either maintaining detailed records as to the contents of individual tapes, or sorting through an entire collection until a desired tape is found. Since the rigorous cataloging of tapes has little appeal to users, most users simply resort to notes affixed to individual cassettes, or they write descriptive information on a tape label. However, once the number of tapes in a library exceeds ten or fifteen units, this method for marking becomes unmanageable. Moreover, in the future, new, more compact, video recording media, such as smaller cassettes, magnetic disks, and recordable optical discs or “videodiscs,” are likely to make such marking methods increasingly impractical.

SUMMARY OF THE INVENTION

In contrast to existing video indexing arrangements which are text-based, this invention uses pictures representative of a recorded video program to assist a user in determining the contents of a recorded medium. The use of pictures enables the index to be generated automatically from the recorded program material itself, thereby minimizing or eliminating the need to enter textual or other identifying information. Not only is suitable picture information already available in the program material being recorded, but it also is clear that even a small number of pictures may convey significant detail about the contents of a particular program.
Broadly, given a video program to be recorded, a method of indexing that program in accordance with the invention, includes the steps of separately storing picture-based information representative of the program and later displaying the picture information without having to view the program itself. An index according to the invention may be generated after as well as during the recording process. Considering a video program as a sequential set of images, the invention prescribes separately storing information representative of a small subset of the images and displaying images from this subset as a method of identifying the contents of the video program. Preferably, the pictures represent periods of the program which are separated in time, enabling a relatively few such pictures to characterize an otherwise lengthy program. By way of example, 12 pictures, each representing segments in the program which are spaced apart in time by 10 minutes, may be used to identify and/or index a 2-hour movie. The same display device used for viewing the program material may be used to display the pictures, which preferably are presented in separate windows on the screen. In alternative embodiments, the pictures may include still or moving imagery and, as a further option, the viewer may select a particular picture to commence replay of the recorded program material associated with the picture selected, or initiate playback of stored audio information associated with the picture selected as part of identifying the chosen program segment.
The picture information may be stored on the same medium as that used to record the video program, or a different medium may alternatively be used. To reduce storage requirements, the information is preferably data compressed then de-compressed for display purposes. To derive the information used to generate the pictures, the video program may be sampled on a periodic basis or distributed and received with integrated encoded picture information, which then may be stored separately.
To ensure that the pictures themselves adequately represent the recorded program, aspects of the program may be analyzed prior to the extraction of the picture information to ensure that blank segments, commercials, or other unwanted information is not stored inadvertently, instead of pictures representative of program subject matter. As a further aid to the characterization of a recorded program, the system may automatically store pictures associated with the beginning and/or ending of a particular program or recording session.
In terms of apparatus, the invention may either incorporate a video recorder and a display device, or alternatively, use existing equipment, such as a VCR or disc drive, for recording purposes. A television monitor, computer screen or any other suitable display device may be used for viewing. In conjunction with such hardware, the inventive system includes an input for receiving a video program to be recorded by the video recorder, means for deriving and storing pictures representative of time-separated segments of the program, and means for formatting the stored pictures for viewing on the display device. In a comprehensive embodiment, the hardware includes a picture memory (such as a battery-backed-up random-access memory) for storing the images associated with each of the time-separated segments of the program, a frame buffer in communication with the picture memory and display device, and video display generation means for outputting the contents of the frame buffer to the display device in the form of a video signal. To convey motion in different windows on the display device, the picture memory stores not one but a series of images associated with each of the time-separated segments of the program, and further includes means for periodically updating the frame buffer with individual images from a particular group, so as to convey motion. In this embodiment, which supports moving pictures in multiple windows, a multi-channel tuner may be added and sequenced so that pictures representative of the video programs available on each of the tuned channels will appear in the separate windows, rather than indexing information.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a drawing of a picture used to discuss its active area and storage requirements, in accordance with this invention;

FIG. 2 is a drawing which shows the picture of FIG. 1 in one of a plurality of windows generated by the hardware of the invention; and

FIG. 3 is a block diagram of the system according to an apparatus aspect of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Broadly, this invention automatically generates a display of pictures representative of program material stored on a video recording medium. Although the term “video” is used throughout this specification, it should be recognized as including all forms of electronically recorded imagery, whether or not compliant with a standardized format, including existing video formats such as NTSC, PAL, SMPTE Standard 240M HDTV, and so on. Operationally, then, given a recording medium capable of storing video information, whether in the form of a tape or disc, whether magnetic, optical, etc., the user accesses the medium in an appropriate playback system, and, on an associated display device, there appears one or more “thumbnail” or “vignette” images representative of the material recorded on that medium. In the preferred embodiment, a plurality of windows appears on the associated display device, each window including imagery associated with time-separated portions of the recorded program material. For example, the invention may be configured so as to generate an array of windows on the screen (for example, 9, 16, or some other number), each window representing a time-specific portion of the program material, but the contents of each window preferably being separated in time by several minutes, thereby enabling a relatively small number of such windows to represent a more comprehensive amount of recorded program material, even hours worth of such programming. To be more specific, a system according to the invention could generate, for example, 20 windows, each representative of portions of the recorded program material time-separated by, say, 15 minutes, thus enabling the 20 windows to be indicative of 5 hours of recorded program material. As will be described in much greater detail below, the gap in time between each window need not be fixed, but instead may be chosen in accordance with some other parameter, such as the beginning or ending of a recording sequence. As will also be described below, the imagery in each window need not be a still picture, but may include motion, preferably a few seconds of images related to a localized time slice. Although the invention should find immediate applicability in the determination of the contents of an unmarked or inadequately labeled video cassette, (in which case the invention would preferably be incorporated into a video cassette recorder (VCR)), the invention is not limited to this end application, and may be used in conjunction with the identification of electronic imagery in disc form, for example, for use with an appropriate reader interfaced to a personal computer or work station, including multimedia CD-ROMs, and so forth.
Generally speaking, then, when video program material is being recorded in a system equipped with the present invention, information used to generate image windows of the type discussed above is also recorded, preferably on a periodic basis and in digitally compressed form, as further described below. Although the information used to generate the image windows is derived from the program source material, different alternatives are available, according to the invention, for deriving such material and for the storage thereof. For example, an incoming source signal containing program material to be recorded may be sampled on a periodic basis, and these sample images may be digitally compressed and stored for use during playback of the material. However, the information used to generate the window imagery either may be derived directly from the video signal or data stream associated with the program material itself or, alternatively, broadcasters may cooperate by transmitting program material including supplemental information which may be used to generate image windows associated therewith, either in an unused portion of the signal for the program material, such as during a blanking interval, on a separate, unused channel, or by other appropriate means. In terms of storage, the information used to generate the window imagery either may be stored separately from the recording medium used to store the program material itself, or, alternatively, may be stored along with the program material on the same medium, either in predetermined areas or interspersed with the program material. The tradeoffs associated with these various options will become more evident in light of the descriptions which follow. As an alternative to picture-based indexing in progress with the recording of video program material, a fully or partially recorded medium may also be indexed according to the invention on an off-line basis.
Before considering the various alternative implementations, storage requirements first will be discussed in conjunction with applicable compression techniques. In FIG. 1 there is shown a picture having an active area 102 comprising an image area having dimensions of 640×480 pixels, which represents a typical NTSC 2-field frame based on square pixels. One of skill should readily appreciate the applicability of the invention to video standards having different configurations or image dimensions in pixels. In order to reduce storage requirements in conjunction with the invention, since the subject matter contained in the central area of the image most likely will carry the greatest significance, a band around the outer perimeter of the image need not be sampled, resulting in a sampled area 104 of 576×432 pixels. This assumes that 24 pixels on the top and on the bottom of the image and 32 pixels on either side are disregarded, although bands of different widths may be used according to the circumstances. In one alternative, the system may include hardware for “recognizing” appropriate subject matter, in which case the sampled area may form part of a moving window which moves according to actions of the subject matter.
Given a sampled area of 576×432 pixels, if every sixth line is then considered for storage, the result is 72 lines per image, and, if 96 equally spaced apart columns are sampled per line, this results in 6912 total pixels. If three bytes are used per sample, two bytes associated with chrominance information and one byte for encoding luminance information, this results in 20,736 bytes of information, without resorting to any compression techniques. However, utilizing available compression engines such as JPEG, a compression ratio of 16:1 is readily achievable, resulting in 1,296, or approximately 1,300 bytes for a sampled frame. Preferably, to this will be added a number of bytes which may be used for the purposes of identification, including an alpha-numeric label, location tag, time/date stamp, and so forth. As such, an estimate of 1,500 bytes per picture may be more appropriate in practice.
FIG. 2 illustrates one possibility for a display generated by the hardware to be described, according to this invention. In this example, the picture or frame of video information just discussed with reference to FIG. 1, having previously been stored, is now recalled and displayed in one of many windows so as to facilitate the indexing of the video program. Given a display with a “safe action area” of 432 lines each having 576 pixels (which is, again, a sampling area associated with a typical NTSC frame having square pixels), if each thumbnail image to be shown is allocated a display area of 144×108 pixels, sixteen such images readily may be displayed, each having a framing border and optionally providing space for one or more lines of textual information. Evidently, depending upon the video standard, amount of available resolution, and the number of such vignettes to be displayed, the number of windows may be varied from at least one to more than the number exemplified by FIG. 2, again, depending upon the circumstances.
As mentioned, this picture information either may be stored apart from, or in conjunction with, the same medium used to store the program material as it is being recorded. Assuming that such information is to be recorded on a video cassette along with the program material, various alternatives are available. Particularly, in light of suitable data compression, the picture information may be stored on an on-going, progressive basis, along with the program material, for example, in the vertical retrace interval, although other alternatives are possible, such as the use of other synchronization periods within the video signal, or the control track, particularly for the recording of indexing pointers as discussed below. However, by using the vertical interval, a number of non-visible scan lines may be used to record serial data representative of the pictures, in much the same way that closed-captioning information is recorded. Disregarding horizontal blanking, a scan line represents an available time period of approximately 50 microseconds and, if this serial information is written at a rate of 2 megabits per second, just over 100 bytes will be available if, for example, eight lines of the vertical interval are used for these purposes. Given these 100 bytes per field, if this is carried out over 15 consecutive fields, the requisite 1,500 bytes needed for the storage of a single picture will be available, thereby enabling four pictures to be recorded per second automatically, assuming compliance with an interlaced NTSC standard of sixty fields per second. Thus, assuming sixteen sample pictures are to be displayed for a given length of program material, each sample picture automatically will be recorded in the blanking interval as just described, preferably with a period of several minutes elapsing between the recording of each sample picture.
The vertical interval will be recorded more or less continuously with information associated with all of the pictures in the sequence. As it comes time to record a sample picture, this new picture is added to the existing sequence of information representative of all previously sampled pictures. Thus, when information representative of sixteen pictures has been accumulated, four seconds of vertical retrace intervals will be required for the recording thereof, and, in this particular embodiment, this four seconds of vertical retrace information simply will be rewritten in a continuous cycle every four seconds, since sixteen pictures were previously determined to be the total number associated with that program material. Thus, once these four seconds have been recorded on the tape, during playback and indexing, the system need only recover a total of four seconds worth of tape playback in order to gather all of the information necessary to regenerate all sixteen of the pictures for display. Correspondingly, if indexing is commenced at a point when, for example, only twelve of the pictures of the full capacity of pictures available for display on the screen of the display device, then this system will only need to “back up” for three seconds of tape playback in order to gain the information required to generate and display the twelve sample pictures.
One of the principle advantages of the embodiment just described is that, by using the same recording medium for both the program material and the pictures representative of that program material, separate storage media need not be required for the picture information. However, considering the fact that electronic hardware is required to derive and store the picture information onto the recording medium, it may be advantageous to add separate picture storage capabilities, particularly in light of the operational features to be gained. One potential disadvantage of such separate storage, however, is that, should the medium carrying the program material be carried to a different machine for playback purposes, unless the separate picture storage capability is carried along therewith, the indexing capability according to the invention will not be available during playback on this different machine. However, considering the fact that most consumers use the same system for recording and playback, the use of separate picture storage should not represent a great disadvantage.
One potential problem, however, does exist in the case that a tape that previously has been recorded is re-recorded, with new program material inserted so as to replace one of the existing program. Here, the picture index information in parts of the tape will reflect the contents of the earlier recording, rather than that of the subsequent recording. Although this is likely to be an infrequent occurrence, it does suggest a preference for an embodiment which records the indexing information at a specific location, such as at the beginning of the tape, as this style of embodiment will maintain the accuracy of the indexing information even when re-recording has occurred.
The following embodiments, then, include the use of a separate picture storage capability, preferably in the form of a random-access memory such as dynamic RAM storage or “flash” memory, preferably used in conjunction with a battery back up facility of some kind to ensure that the indexing information is not lost during an interruption of power. As an alternative, a non-volatile type of memory may be used, which may be transferred into a faster, volatile form of memory if retrieval is too slow. The use of a separate memory for storing the picture information may be used with or without storage onto the medium storing the program material itself. For example, information associated with each new picture may be saved separately from the program recording, then, upon activation of a particular operator command (such as END-RECORDING), the information may be transferred from the separate recording memory onto the tape or disc in bulk fashion, in contrast to the accumulating procedure described earlier. For example, at the end of a recording session, one or more subsequent complete video frames (i.e., utilizing all of the available field lines, and not just the vertical blanking interval) may be used to receive all of the picture information stored during that recording session, then accessed when recording resumes or in conjunction with playback activities. Alternatively, upon the termination of a recording session, the program storage medium automatically may be directed to a particular physical location wherein all of picture information is stored for later retrieval. In the case of a video tape, for example, the tape may be rewound automatically, with the picture indexing information always being stored at the beginning of the tape, so that it conveniently may be accessed when the tape is later inserted into the VCR, thereby enabling the tape to be indexed prior to the next viewing. While this presents certain disadvantages involving the length of time required to rewind the tape after its recording session, such a disadvantage will be minimized dramatically as the industry moves from tape-based recording media to new video discs available from Sony and others, which promise to store many gigabytes of information on an individual disc, and allow availability of this information on a substantially random-access basis. With such media, it should be a much more straightforward procedure to automatically translate the appropriate recording head to a predetermined area of the disc at the conclusion of a recording session, and then record the picture indexing information.
As a further alternative to the use of separate picture storage in conjunction with the storage of program material on a high-capacity medium, such separate memory may be made large enough to accommodate multiple picture images associated with a number of different program storage media, without requiring the transfer of the picture information back onto the program storage media itself. That is, in these alternative embodiments, each program storage medium will be identified in some manner and referenced to the separate picture memory, such that, after the recording has concluded, when accessed for playback the appropriate sections of the separate picture memory will be accessed, and the images associated therewith displayed. For example, in a VCR environment, the system may include several megabytes or gigabytes of hard-disk RAM storage, which at 667 images per MB of storage, could accommodate hundreds or, perhaps thousands of index pictures corresponding to numerous program storage media. In such a case, during the recording process certain supplemental information may be recorded onto the tape along with the program material, such as a tape/disc or program storage medium identification number (recorded on a periodic or continuous basis), and/or frame indexing data, time codes or other pertinent information facilitating more convenient access to particular portions of the program material. For example, in the case of a video cassette, while recording program source material, a unique cassette identification code and/or frame number simultaneously may be recorded onto the tape in a non-visible portion of the signal, such as the vertical blanking interval, or encoded into the control track, as in several indexing techniques well known in the art. At the same time, picture information associated with the program material, either derived from the incoming source program and locally compressed or received in conjunction with the program material through a cooperative broadcaster providing such information, simultaneously would be stored in the separate picture storage medium along with the tape ID code and frame indexing information, if available. Upon playback or through the activation of an INDEX command, under operator direction the stored picture information would be accessed from the separate memory, and formatted for display purposes, preferably in the form of an array of windows, each containing a picture. However, by using a recorded numerical index, time code, or other means for advancing to a particular section of the tape (or disc), an operator may then use a cursor, keyboard or any kind of pointing device to select one of the displayed pictures of the array, which would then cause the program storage medium to advance to the section associated with that picture, to commence replay therefrom, for example, or to begin recording over previously stored program information no longer to be retained.
It should be mentioned, however, that other alternatives according to this embodiment exist for the storage of the picture information or index data to access such information. For example, information may be imprinted on, or encoded within the cassette shell itself, then read by the apparatus of the invention as part of the display of the picture information. In one arrangement according to this embodiment, cassette shells may be provided incorporating semiconductor or other solid-state type of memory, for example, bonded to the inside of the shell, with electrical contacts provided on the outside of the shell for making connection to the IC chip. Upon insertion of the cassette into playback equipment incorporating the invention, the picture information may be stored in the solid-state device and retained there until the next time it is accessed or modified, during subsequent usage. Preferably, any power required for such a memory chip would also be supplied by the playback equipment receiving the cassette, and delivered to the device through the contacts provided externally on the cassette shell. In lieu of a chip or other imprinted means storing the actual picture information, a bar code, for example, may be used to identify that particular cassette and, optionally, to further encode areas associated with a separate memory within the equipment receiving the cassette or other medium, within which separate memory the actual picture information is maintained.
FIG. 3 illustrates in block diagram form, a hardware embodiment of the invention incorporating separate picture storage means. This figure illustrates a comprehensive version of the invention including features which already may be present in other pieces of equipment, and therefore may not need to be replicated. For example, although, in the preferred embodiment, the same device used to present the video program itself is used for the display of the pictures associated with the indexing thereof, this need not be the case in all embodiments. For example, given a different device, such as a computer, capable of receiving and reading the medium upon which the program material is stored, indexing may be carried out in an entirely separate system, for example, through the use of the computer display. Also, a tuner 302 is shown, in the event that source material is modulated in some form, requiring a conversion to baseband video. However, such a tuner already may be present in an existing VCR or other piece of viewing apparatus. Additionally, the program record block 310 may form part of an existing conventional VCR, in which case other components associated with the invention may be housed in a separate enclosure and attached to the VCR as an add-on unit.
With the understanding that those modules already present in associated equipment need not be duplicated, the system of FIG. 3 operates as follows: an incoming video signal, either directly input along path 301, or converted by tuner 302, is placed onto a signal line 304 using an input video switch 306 which, like most functional blocks in the system, is under the control of a central microprocessor 308. This microprocessor 308 may be of conventional design, and preferably is implemented in the form of a single-chip microcomputer having “on-board” temporary storage and read-only memory (ROM) used to store software program code. The video signal present on path 304 may be switched directly to an output line 312 through output video switch 314, in the event that a user simply wishes to view program source material without recording or indexing of any kind. If the invention is implemented within a VCR, these video switches 306 and 314 already may be present. For that matter, the single chip microcomputer 308 may also be available, and if the unit has sufficient additional inputs, outputs, program memory and performance, also may be utilized for some or all of the functions of the invention which will now be described.
The incoming video signal on path 304 is presented to a mixer 320, a picture detector/generator 322, and, optionally, to a synchronization pulse detector 324 which, in turn, feeds a signal to the microprocessor 308. The sync detector 324 may be unnecessary, in the event that the picture detector/generator 322 extracts sufficient information from the incoming video signal to perform all desired functions. The picture detector/generator block 322 in the preferred embodiment is capable either of detecting compressed picture information provided along with program material by a cooperating broadcaster, or alternatively samples the incoming video signal under microprocessor control and generates the picture information directly from the video signal, in which case block 322 is assumed to include appropriate digitizing and video data compression means. Regardless of the source, the compressed picture information is stored in the separate picture memory means 326 for later retrieval during playback, this picture memory means 326 preferably being supported by back-up power source battery 328.
A video selector switch 330 is used to route information either from the pictured detector/generator block 322 or alternatively from the picture memory means 326, under microprocessor control in accordance with the selected operational mode. For example, if the picture index information is generated locally and intended to be stored together with the recorded program information but without separate picture storage, switch 330 will be used to route the generated picture information directly into the mixer 320, to be recorded simultaneously with the program material, in block 310. Alternatively, if the picture information is recorded onto the program medium as part of a batch transfer process, switch 330 may be used to form a path between picture memory means 326 and the mixer 320, so as to effect the transfer of a group of pictures to the recording medium, again under microprocessor control. To ensure that the picture information is properly recorded onto the program medium, the synchronization detector 324 may be used for such a purpose. In the event that the picture information is stored only in picture memory means 326 and not as part of the program material on the recording medium, the picture memory means 326 preferably also will record an identification code for the medium and, optionally, frame identification information or time-code data used as indexing pointers to the picture stored in conjunction with a particular recording session. Since it will be advantageous to record at least such auxiliary information into the record medium during storage of the program material, switch 330 will route such information on a selective basis from a picture memory means 326 through mixer 320 for inclusion in the recorded video signal. Particularly, in the event that some form of time-code is used for this purpose, a separate block 342 labeled “tape travel control” may be used to keep track of the sequencing of the main recording medium (whether tape or disk), feeding a signal to the picture memory means 326 along signal path 343.
Regardless of where the picture indexing information has been stored, upon playback and/or entry of an “INDEX” command via remote control 350 linked to microprocessor 308 through infrared detector 352, the system causes one or more of the stored pictures to be output through signal path 312 in communication with a suitable display device (not shown). As a first step toward displaying this information, the system will determine whether the picture information has been stored on the program medium tape or disk, or instead has been stored separately in the picture memory means 326. Typically this easily will be accommodated by interrogating the program recording medium when it is inserted in the player unit or otherwise activated for access. If it is determined that a different section of the medium needs to be accessed in order to recover this information, the tape travel control block 342 may be utilized for such a purpose. If the system determines that, indeed, picture indexing information has been stored on the recording medium, it will be extracted along path 360 and delivered to a video generator/frame buffer block 362, under control of the microprocessor 308 to be properly formatted for display purposes. In the event that a single array of windows is insufficient to display all of the images associated with the recordings of particular program materials, or is incapable of representing the entire contents of the recording medium, a number of multi-window displays may be used, for example, as “pages”, with the microprocessor facilitating such displays in conjunction with picture memory means 326, video generating/frame buffer 362, and inputs from a user via remote control 350. As a further convenience, a printer 390 may be attached through an I/O port 392, enabling an operator to print out the pictures onto paper or a label for affixation to the program record medium or packaging.
If it is determined that the pictures associated with certain program material have been recorded separately in picture memory means 326, the system only may need to interrogate the program recording medium to determine an identification number and thereby locate the associated pictures in memory means 326. Having located a correct set of indexing pictures, these pictures then may be displayed. As discussed above, in the event that frame indexing, time codes, or the like have been used to cross reference recorded program material and the pictures associated therewith, once an array of pictures is provided to an associated display device, touch-screen techniques may be used with appropriate display facilities, or “point-and-shoot” pointing techniques may be used in conjunction with the remote control 350, thereby enabling one of the displayed pictures to be pointed to and selected, following which the tape travel control block 342 will cause the recorded program to advance or rewind to that portion of the tape associated with the picture selected. Again, as utilized herein, “tape” travel control should be construed more broadly to include disks and any other form of recording media.
As an alternative to the storage and display of still pictures associated with the recorded program material, each window instead may include motion imagery representing a few seconds of the recorded material at that particular time. Since it is an object of this invention to enable an operator to more easily and conveniently locate previously recorded material, particularly when stored in an unlabeled or even mislabeled manner, use of motion pictures in each window might further assist a viewer in determining the representative program material.
In order to generate such motion imagery, many of the steps just described would continue to be used, however, at the time associated with sampling the source material, not one but a few or numerous frames would be recorded and later displayed in such a manner that each window could communicate a few seconds or more of “action” which would then be repeated, much as the movement of hurricanes and cloud formations is depicted in televised satellite weather forecasts. So as to conserve memory, consecutive frames would not be stored but, instead, a number of frames would be skipped between the updates. Although this might result in uneven motion within a window, it would allow a much greater range of discernability within a particular window for a given storage requirements. In order to generate such windows with motion, pictures are retrieved from the memory means 326 and delivered to the video generator/frame buffer block 362, in accordance with a predetermined update rate. In order to reduce bandwidth requirements along path 364, each displayed window may be updated on a sequential basis, resulting in the movements in the various windows being “out-of-sync” with one another; however, it will be straightforward for the viewer to interpret the images. That is, for example, the contents of picture memory means 326 may be sampled with respect to the contents of 16 window No. 1, causing motion to appear in that window, while displaying all other windows as they were in the previous frames. Next, the memory means 326 may be sampled and window No. 2 updated, causing relative motion therein to be perceived, while continuing to present the information in all other windows without change, including window No. 1, and so on.
The ability to sample the memory means 326 in order to update windows having motion imagery enables the hardware of the system to perform other advantageous functions not associated with video indexing. For example, the input video source may be sampled in a manner similar to that just described in order to display multiple windows, each with motion imagery, and each representative of a different incoming channel, yet requiring only a single tuner. In this a case, the system operates as follows. Under microprocessor control, the tuner 302 is set to a first channel, and information representative of the image on that channel may be loaded into the frame buffer aspect of functional block 362, appearing in a first window on an associated display device due to the action of the video display generator in block 362. At a short time later, the microprocessor will tune tuner 302 to a second channel, and an image will be extracted therefrom to appear in a second window on the display device, and so on, until multiple windows on the screen simultaneously display images from the various channels so tuned. By continuing the process of channel sampling in a cyclical fashion, each window may then represent not only a still picture of the program being broadcast on each channel, but may be updated in a manner similar to the use of motion images for indexing purposes, as previously described, thereby resulting in multiple windows, each with moving pictures representative of different channels, and all derived through the use of a single tuner. The operator, through controller 350, interface 352, and central microprocessor 308, may select which of the various available channels to display, and may be selected in “touch-screen” or “point-and-click” fashion, as described herein above.
Another mode of operation made possible by the apparatus according to the invention, is the ability to use images from the picture memory 326 as opposed to the recorded program 310 to monitor the progress of a programming sequencing operation in excess of ordinary playback speed. For example, in the case of a video program being recorded onto a video tape, utilizing current technology, in order to monitor the progress of a rewind or fast-forward operation, the playback head must remain in contact with the tape, with the result being a “chopped-up” screen display during such operations. Not only is this sheared display annoying to a viewer, but it may also limit the speed with which the tape may be advanced or rewound, owing to the fact that the scan head must be in contact with the tape to transfer the images to the display.
Utilizing the principles of the invention, however, rather than extract frames from the recorded program for display purposes during fast-forward and rewind, so long as a correspondence between frames is maintained between the program record block 310 and picture memory 326, frames representative of the stored video program may instead be used directly from the picture memory 326 during such operations, thereby obviating the need to use a playback head or other form of pick-up remaining in contact with the storage medium. In practice, using fast-forward as an example, the playback head may be retracted from the recording medium, allowing it to travel at a high rate of transport speed, while images are instead extracted from picture memory 326 and output through path 312 to appropriate display electronics, in the form of flashing images representative of the underlying program. Coordination between the recorded frames of the video program and those associated with the program segments of the picture memory would be managed by tape travel control block 342 and control time-code detect block 340 in conjunction with microprocessor 308.
As with other embodiments of the invention, the advantageous use of the apparatus of FIG. 3 just described may make use of a picture memory on the same medium as that used to store the video program or may utilize a separate medium, depending upon the desired implementation. Even in the case of the video tape, the same medium may be used by storing the frames associated with the segments of the program outside of the main recording area, as in the control or audio tracks, enabling the tape to be retracted from the helical-scan head during rewind/play-back operations, while leaving a less restrictive sensor associated with the audio or control tracks in place. Alternatively, the picture memory may be implemented as an entirely different medium, such as a solid-state or other form of rotating magnetic or optical medium, in which case nothing other than transport elements would need to touch the media during rewind/fast-forward operations.

Claims

1. An apparatus, comprising:

a memory;

a processor;

instructions stored in the memory that, if executed by the processor, causes the apparatus to perform operations comprising:

reading picture information including a subset of images of a plurality of images of program material, the image subset representing segments of program material separated in time, the subset of images being stored separately from the program material on a same random-access physical media;

generating an output based on the subset of images, the output configured to display images of the separately stored subset of images in a plurality of respective windows on a display device;

receiving a selection associated with an image of the subset of images; and

outputting the program material to the display device beginning with the section of the program material associated with the selected image and continuing with subsequent sections.

2. The apparatus of claim 1, further comprising generating the output as a plurality of pages to accommodate the plurality of respective windows.

3. The apparatus of claim 1, further comprising directly accessing the program material in accordance with an association of the selection of one the subset of images and an index.

4. The apparatus of claim 1, wherein the index is stored on the random-access physical media.

5. The apparatus of claim 1, wherein audio information is stored in conjunction with an image of the subset; user-selecting a displayed image from the subset, and wherein the stored audio information is replayed corresponding to the selected image.

6. The apparatus of claim 1, further comprising a remote control, wherein the selection is received from the remote control associated with the apparatus.

7. The apparatus of claim 1, wherein at least one of window includes motion imagery.

8. A video indexing method, comprising the steps of:

storing a video program having time-sequential frames representative of motion imagery on a random-access computer readable medium; and

separately storing a subset of the frames associated with segments of the video program that are spaced apart in time,

wherein separately stored frames of the subset are suitable for display in a plurality of windows on a display device,

wherein the subset of the frames associated with segments of the video program is stored separately from the video program and on the random-access computer readable media, and

wherein at least one window is selectable by a user, and upon selection of a window, the video program is output for display on the display device.

9. The method of claim 8, wherein the video program is output for display on the display device beginning with a segment of the video program associated with a frame displayed in the selected window and continuing with subsequent segments of the video program.

10. The method of claim 8, wherein the subset of frames is derived by periodically sampling images of video program.

11. The method of claim 8, further comprising:

accessing the random-access computer readable medium to initiate a replay of the video program from a point in the video program corresponding to a selected window.

12. The method of claim 8, further comprising:

storing audio information in conjunction with an image of the subset;

enabling a user-selectable image from the subset; and

replaying the stored audio information corresponding to the user-selectable image.

13. The method of claim 8, further comprising:

analyzing the video program to determine the subset of the frames associated with the segments of the video program; and

creating an index of the frames to the segments of the video program.

14. The method of claim 13, wherein at least one window includes motion imagery.

15. A tangible non-transitory computer readable medium having stored thereon a video program having time-sequential frames representative of motion imagery, and a separately stored thereon, a subset of frames associated with segments of the video program which are spaced apart in time, the computer readable medium having software code stored thereon for causing a processor to carry out a method, comprising:

outputting separately stored frames of the subset for display in a plurality of separate windows on a display device,

wherein the tangible non-transitory computer readable medium is a random-access media enabling direct access to the subset of frames associated with segments of the video program are randomly accessible by a computing device.

16. The tangible non-transitory computer readable medium of claim 15, wherein a user-selection of a displayed image from the subset initiates a replay of the video program from a point in the video program corresponding to the selected image.

17. The tangible non-transitory computer readable medium of claim 16, wherein the replay is initiated by directly accessing the video program corresponding to the selected image.

18. The tangible non-transitory computer readable medium of claim 15, wherein audio information is stored in conjunction with an image of the subset, wherein a user-selection of a displayed image from the subset causes a replay of the stored audio information corresponding to the selected image.

19. The tangible non-transitory computer readable medium of claim 15, wherein an index of the subset of frames to the segments of the video program is stored on the tangible non-transitory computer readable medium.

20. The method of claim 15, wherein at least one window includes motion imagery.