With the advance of high-speed network technologies, the availability and popularity of streaming media content over the Internet has grown rapidly in recent years. The delivery and caching of streaming media must be handled in a different fashion than that of traditional non-streaming objects such as HTML or image files, because of its distinct characteristics and user viewing patterns. We propose a novel scheme that provides users with the video summary (a number of key-frame images) before they download the file, and options for them to select the starting playback position. We introduce the content analysis service to achieve these functionalities. The video content analysis performs shot boundary detection, key-frame selection, and face detection and tracking. The results of the processing are a segmented video sequence and an XML-based meta-data describing the video content. We also design a caching system that utilizes our video abstraction and summarization technique. Our integrated video delivery and caching system combines content-aware segmentation, prefix caching, prefetching, and cooperative caching. We describe how our scheme can be applied in three proposed caching architectures.