专利汇可以提供Motion compensation method for video sequence encoding in low bit rate systems专利检索,专利查询,专利分析的服务。并且A method for motion accounting and compensation while encoding allows improvement of the restored image quality. Image is a predicted frame in video sequence. Motion compensation is understood as an error residual signal compressing scheme, executed during video sequence encoding. According to the teachings of the current invention, errors are accounted for during encoding. Error accounting is subtracting the original image from the predicted frame. The error values are written to the output stream compactly with the minimum code length. The stored error values are added to the predicted frame during the decoding process. The method taught herein is applied for all the predicted frames from the video sequence, thereby allowing one to improve the quality of decompressed video film that was compressed with a significant compression ratio.,下面是Motion compensation method for video sequence encoding in low bit rate systems专利的具体信息内容。
This application claims benefit of Provisional Patent Application No. 60/396,331, filed Jul. 16, 2002, entitled “Motion Compensation Method for Video Sequence Encoding in Low Bit Rate Systems.”
The current invention relates to video compression. In particular, the invention relates to motion compensation field for low bit rate compression systems.
Video films are an effective means for describing various subjects. Information about the subject is accumulated in video databases. At the present time a great multitude of types of databases have been developed that are oriented to the applications of special methods of analysis and information processing.
Current digital transmitting systems have a number of advantages for video processing in comparison with analog systems. Recently developed techniques have led to improved methods to reduce video or image size. Such methods are extremely useful for digital data storing and processing or manipulating. So it may be said that data size reduction is a compression process. As to the architecture, it is possible now to put a complete compression process into a single chip. The main objective of a compression process is to achieve the highest compression ratio and in the same time to provide the minimum data loss that may lead to decompressed image quality degradation.
The present invention is better understood upon consideration of following definitions. It is reasonable to perform a video film as a frame sequence. There are a number of frame types that are used to create a frame sequence while encoding or compressing a video film. As used herein, the terms encode and compress are taken to have the same meaning. Similarly, as used herein, the terms decode, restore, and decompress are taken to have the same meaning.
MPEG-1 Defines Four Different Types of Frames.
I-frames (intra-frames) are compressed using intraframe coding; they do not use information from any other frames of the video sequence. This situation is similar to applying Lossy JPEG on individual pictures.
P-frames (predicted frames) are encoded using motion compensated prediction from past I-frames or P-frames. P-frames can be used as reference points for additional motion compensation.
B-frames (bidirectionally predicted frames) provide the highest degree of compression. They are encoded using motion compensated prediction from either past and/or future I-frames or P-frames. B-frames require significantly fewer bits than I- and P-frames.
D-frames are similar to I-frames, but they are encoded using Discrete Cosine Transform (DCT). D-frames are not used in MPEG-2. Therefore, there are three main frame types in MPEG: I-frames, B-frames and P-frames.
The frame stream of the current invention may be described by the following frame types: I-frames, P-frames, reference frames and the current frame.
The current frame is a frame encoded in the present moment of time. The invention disclosed herein uses information from the reference frame to encode the current frame.
Backward-reference frame is a reference frame appearing before the current frame during restoration or decoding.
Forward-reference frame is a reference frame appearing after the current frame during restoration or decoding.
Predicted frame is a frame restored on the basis of a decompression algorithm during encoding but not restoration.
It is sometimes necessary to use frames encoded without using any information from other frames while encoding. Such frames will be referred to herein as Intra frames. The need for intra frames may be conditioned by, for instance, the first scene in a video film or a scenario change that takes place in the film.
Intra-frames require no motion compensation. Therefore, the current invention does not apply motion compensation methods for I-frames.
It is thus an object of the present invention to provide a motion compensation method suitable for various video sequences.
A further object of the present invention is to provide good compression quality, i.e. a high compression ratio with the help of the given motion compensation method.
Another object of the present invention is to improve the P-frames post restoration quality.
The process of computing changes among frames by establishing correspondence between them is called a temporal prediction with motion compensation. The motion compensation process is preceded by motion estimation. In turn the motion estimation process is defined herein as a process of searching corresponding pixels (or blocks of pixels) among frames. The important feature of motion estimation algorithms is that they allow elimination of the temporal redundancy of the video sequence.
Motion estimation methods are based on the fact that the scene of video film does not change radically within 1/25 to 1/30 sec. The MPEG motion compensation scheme uses block-based motion compensation scheme to reduce temporal redundancy (i.e., by means of motion compensation, codes of similar blocks can be reduced by referencing them to the image of adjacent frames). The more blocks a frame references, the more similar the adjacent frames are. Therefore, by analyzing the references among encoded frames, their similarities (or dissimilarities) can be determined. For P-frames, after motion estimation, there may be compared two macroblocks—the macroblock from the current frame and the macroblock from the search area of the reference frame. In short two macroblocks are subtracted and their difference is then DCT coded. For B-frames two motion vectors are yielded. The encoder can form a prediction error macroblock from either of two candidate macroblocks or from their average. The prediction error is then encoded using DCT. The quantized DCT coefficients are then encoded using a variable-length coder.
The motion estimation scheme is not a subject of this patent application and the current invention supposes prior implementation of motion estimation procedures. However, it should be noted that motion estimation is needed only for encoding process, but motion compensation is needed for both the encoder and the decoder.
The motion compensation scheme of the current invention comprises the following steps:
Step 1 and step 2 are executed during the encoding process, and step 3 is executed during the decoding process.
The current invention presents a motion compensation scheme, comprising of the steps described herein. The invention is better understood upon consideration of the detailed description below and the accompanying drawings.
Turning now to the drawings, the invention will be described in preferred embodiments by reference to the drawing figures.
It is reasonable to perform video as a frame sequence, in turn every frame may be performed as digital matrices. There are some well-known formats for picture representation, such as the RGB model and the YUV model. The RGB model exhibits color as a combination of red, green, and blue. In the YUV model, Y is the color intensity, U is the difference between red and green intensities, and V is the difference between the blue and green intensities.
There exist some models performing YUV standard, one such model is the 4:2:2 model. In this model, the Y-component value is different for every pixel, but the U and V component values are the same for two adjacent pixels, i.e. every intensity value conforms to a pixel; color differences' values are the same for the neighboring horizontal pixels. The table below shows such model.
The most informative component is the Y-component, and the U and V components are much less informative. Moreover it is possible to operate three digital matrices such as Y, U and V components. The current invention will be described using the YUV model.
Referring to
This method is described more fully below. According to the teachings of the present invention, steps 1 and 2 are executed during the encoding process, whereas step 3 is executed during the decoding process.
In the prediction error evaluating process, shown in
The initial data needed for the current motion compensation method are P-frame, O-video sequence original frame, frame width (W) and frame height (H). The motion compensation method of the current invention is then applied for each predicted frame.
Step 1 (Prediction Error Estimation): The first step of the motion compensation method is prediction error estimation (prediction error is the difference between images in the original frame and the motion compensated prediction images in the predicted frame). In this step, the e error is calculated by subtracting P-frame, from the original frame. An array of the error values is then constructed using the equation below:
where Eij is an array containing prediction error values, H is the frame height, W is the frame width, Oij is the original frame in the video sequence, P is a predicted frame, and w is a certain constant.
The constant w, is the pre-processing quantization value. Using this value we can control the quality (accuracy) of compensation work results. It can be chosen in range [1,255]. For best performance, this value is 1 and the value 255 means that the compensation is made only for error values=|255|.
Once obtained, the data in the error array must be stored compactly. The following steps are used to compress the data for compact storage. One purpose of using this method is to get less data loss, because significant losses severely degrade the visual quality of the decompressed images.
Step 2 (Error Array Compression): After the prediction error has been estimated and the error values have been placed in an array, the data in the array is compressed according to the sub-steps (a-f) described below.
a (Error array quantizing): The prediction error estimation yields a sparse matrix performing error array. This data must is now quantized (converted to digital information). The following sub-steps enable one to effectively compact the representation of error values and considerably improve the compression quality of the error array.
b (MERGE COMPACT): After quantizing the error array is still large due to the fact that its dimensions do not change. Additionally, there are a lot of zero elements and uniform values in the error array. The MERGE COMPACT method allows the error data to be compacted to ¼ of its original size.
Referring to
The input and output data for the MERGE COMPACT method is shown in the tables below:
Input Data:
Output data:
The MERGE COMPACT method is executed according to the following steps:
c. (NEW MERGE COMPACT): The MERGE COMPACT method reduces the size of the error array, but there is a loss of data inherent in that method. The NEW MERGE COMPACT method allows the size of the error array to be further decreased while controlling the data loss due to the choice of the k parameter (described below). Referring to
The input, output, and local data for the NEW MERGE COMPACT method is shown in the tables below:
Input Data:
Output Data:
Local Data:
An important aspect of this method is the creation of another array (M) for bit mask number storage. The bit mask number corresponds to the number of elements in the Qcompact block from the initial Ecompact array (i.e., if there are 14 combinations of elements in Qcompact block there would be 14 bit masks). During decoding, the bit mask number is read from the M array and elements are restored according to that number. Before choosing bit masks, every block is processed. As illustrated in
The NEW MERGE COMPACT method is executed according to the following steps:
Bit masks are shown in the table below:
d (Significant area isolating method): Significant area isolating is used to cut off the outermost rows and columns of the data array after executing the previous methods. Under the assumption that there are a lot of zero elements in external rows and columns of the data array, the external array areas are checked.
It is extremely important to save nonzero elements during this stage of the error residual signal-encoding scheme.
Referring to FIG.7, the outermost rows and columns are cut off according to the predefined threshold Tr, which is sufficiently small. This threshold value range is [0,min(Height,Width)]. But in practice it must be small. The value of Tr will vary depending on image size, and can be chosen like min(Height,Width)/200 (200 here is an empirically chosen value). When this threshold value is 1 there is no quality loss. The cut rows and columns are supplemented by zeros and the number of cut rows and columns are recorded to the output stream. The isolated area is stored in the central error data array. The input, output, and local data for the significant area isolating method is shown in the tables below:
Input Data:
Output Data:
Local Data:
The significant area isolating method is executed according to the following steps:
e (Figure covering method): The major task of the current motion compensation method is to write error values compactly to the output stream. The array obtained after executing the preceding methods is sufficiently sparse in that it will contain a lot of zeros. Therefore, it is reasonable to isolate areas containing zero elements, and doing so will allow one to compactly write error values to the output stream.
The compact data representation is achieved by error data array compressing. One of the steps for error array processing is a method of covering the error array with geometric figures that isolate uniform areas. The figure covering method is illustrated in
For every array element, the largest possible geometric figure that contains only zero components is constructed. By using the minimum number of figures to cover the largest possible area, large portions of the entire array may be covered. The types of figures used are shown in
Additionally, there is a threshold for the number of elements in a geometric figure. According to the teachings of the current invention this threshold is defined relative to picture (frame) dimensions and correspondingly to error array dimensions and zero elements number. The P threshold is dependant on figure size and its proportion to the size of the image. It is important to save only big figures, because small images cannot provide gains in compression or compensation signal. In practice, if the size of the figure is more than 6-10 percent of the image size, and the number of zero elements in this figure is above 98 percent, the figure is saved. The input and output parameters, and local data used for figure covering are shown in the tables below:
Input Parameters:
Output Parameters:
Local Data:
The Figure covering method is executed according to the following steps:
(A) For every error array element and for every type of figure, the following steps (1-3) of the Figure covering method are executed:
A list of figures covering the areas of error array that contain only zeros is generated by this method. Because different figure combinations are possible, it is important to choose the best combination. The best figure combination is determined according to the largest area covered and the least quantity of figures in a given array. The smaller the number of figures the higher the degree of compression. If is the best combination of figures, it can be determined using the method below.
(B) For all combinations of figures the following steps (4-8) of the Figure covering method are executed:
Step 4. Calculate the number of elements (p), covered by the given combination of figures containing of f figures.
f (Topology mask constructing): The final stage of the encoding process is a method for topology bit mask constructing. This method is illustrated in
The Ac array is separated into two new arrays, a bit mask array (B) and an array containing the values of the nonzero elements (El). The El array is then separated into two arrays, an array containing the absolute values of elements (Abs) and an array of the element's signs (S). The second array (S) is a bit array, so if an element is positive, write 0 to that array, otherwise write 1:
e>0, s=0;
e<0, s=1.
After separation, the resulting arrays are encoded using run-length-encoding technique. The input and output parameters for constructing topology masks are shown in the tables below.
Input Parameters:
Output parameters:
Prediction error compensating: Prediction error compensation is the process of adding the stored error values to the predicted frame. This is a reverse process to the prediction error evaluating process, and it is completed during the decoding process.
The current invention allows one to compress data related to motion compensation in compressed video. If data array elements are compacted in accordance to the teachings of this invention, compressed video may be replayed without any loss of the data and without severe degradation of the video quality.
The current invention discloses methods and procedures for motion compensation in video compression. The methods and procedures disclosed in the current application can be executed or preformed in a computer, other microprocessors, programmable electronic devices or other electronic circuitry that are used for encoding video film. They can be loaded into the above devices as software, hardware, or firmware. They can be implemented and programmed as discrete operations or as a part of a larger video compression strategy.
In compliance with the statute, the invention has been described in language more or less specific as to structural features. It is to be understood, however, that the invention is not limited to the specific features shown or described, since the means and construction shown or described comprise preferred forms of putting the invention into effect. Additionally, while this invention is described in terms of being used to provide a method of motion compensation for compressed video data, it will be readily apparent to those skilled in the art that the invention can be adapted to other uses as well. The invention should not be construed as being limited to motion compensation and is therefore, claimed in any of its forms or modifications within the legitimate and valid scope of the appended claims, appropriately interpreted in accordance with the doctrine of equivalents.
标题 | 发布/更新时间 | 阅读量 |
---|---|---|
基于子块的预测 | 2020-05-08 | 293 |
一种确定运动信息的方法、帧间预测方法及装置 | 2020-05-11 | 777 |
视频编解码方法、装置、计算机设备和存储介质 | 2020-05-11 | 345 |
从仿射运动预测的非仿射块 | 2020-05-12 | 176 |
一种图像预测方法和设备 | 2020-05-12 | 528 |
模式依赖的仿射继承 | 2020-05-11 | 679 |
仿射的运动矢量访问范围 | 2020-05-11 | 218 |
不同视频块尺寸的仿射模式计算 | 2020-05-13 | 331 |
两步帧间预测 | 2020-05-13 | 206 |
一种帧间预测方法和装置 | 2020-05-12 | 432 |
高效检索全球专利专利汇是专利免费检索,专利查询,专利分析-国家发明专利查询检索分析平台,是提供专利分析,专利查询,专利检索等数据服务功能的知识产权数据服务商。
我们的产品包含105个国家的1.26亿组数据,免费查、免费专利分析。
专利汇分析报告产品可以对行业情报数据进行梳理分析,涉及维度包括行业专利基本状况分析、地域分析、技术分析、发明人分析、申请人分析、专利权人分析、失效分析、核心专利分析、法律分析、研发重点分析、企业专利处境分析、技术处境分析、专利寿命分析、企业定位分析、引证分析等超过60个分析角度,系统通过AI智能系统对图表进行解读,只需1分钟,一键生成行业专利分析报告。