Interframe prediction processor with mechanism for providing locations of reference motion vectors used in macroblock adaptive field/frame mode
专利汇可以提供Interframe prediction processor with mechanism for providing locations of reference motion vectors used in macroblock adaptive field/frame mode专利检索，专利查询，专利分析的服务。并且An interframe prediction processor designed for high-speed video coding and decoding. The interframe prediction processor has an address selector and a predictor calculator. The address selector receives information about MBAFF type and coding type. When performing motion compensation for a given MBAFF picture with reference to blocks surrounding a current block, the address selector provides memory addresses of reference motion vectors according to the coding type of the current block pair, as well as to the coding type of the reference blocks. The predictor calculator that determines a motion vector predictor for each constituent block of the current block pair based on reference motion vectors read out of the provided memory addresses.，下面是Interframe prediction processor with mechanism for providing locations of reference motion vectors used in macroblock adaptive field/frame mode专利的具体信息内容。
What is claimed is:
This application is based upon and claims the benefits of priority from the prior Japanese Patent Application No. 2006-223564, filed on Aug. 18, 2006, the entire contents of which are incorporated herein by reference.
1. Field of the Invention
The present invention relates to a device for predicting a frame from other frames, as well as to video coding and decoding devices using the same. More particularly, the present invention relates to an interframe prediction processor, a video coding device, and a video decoding device that perform interframe prediction of video frames on a block basis.
2. Description of the Related Art
Digital video compression techniques are widely used in many applications. MPEG and H.264 are among the standard specifications in this technical field, where MPEG stands for “Moving Picture Expert Group.” The coding and decoding algorithms used in those standards divide each given picture into small areas and process them with motion compensation techniques. Such picture areas are called “macroblocks.” A video coding process involves intraframe prediction and interframe prediction. The intraframe prediction reduces redundancy within a single frame by using orthogonal transform, quantization, and other data compression algorithms. The interframe prediction, on the other hand, reduces redundancy between successive frames by extracting and encoding motion compensation residual (i.e., the difference between a current frame and a motion-compensated reference frame). The resulting video data is then entropy-coded for transmission or storage. A video decoding process reverses the above steps to reconstruct original video from compressed video.
Some standard coding algorithms including MPEG and H.264 adaptively select a frame-based coding mode or a field-based coding mode to process interlaced video signals. These algorithms further permit switching between interframe prediction and interfield prediction for each two vertically adjacent macroblocks (or “macroblock pair”) for motion vector calculation. This coding mode is called “macroblock adaptive field/frame” (MBAFF) mode. Which of those three modes to use can be specified by a “coding type” parameter on an individual picture basis. In this description, the three coding modes will be referred to as “frame,” “field,” and “MBAFF” modes. The first two modes may also be referred to collectively as non-MBAFF mode, where appropriate.
The top-middle part of
The top-right corner of
To improve the accuracy of motion vector prediction for macroblocks containing edges and outlines, some existing coding algorithms split up such macroblocks into smaller blocks and calculate a motion vector for each block. For example, H.264 supports macroblock partitioning that divides a basic macroblock of with a size of 16×16 pixels into various block sizes as necessary, including minimum sub-partitions of 4×4 pixels.
Most part of the video coding algorithm described above is devoted to calculation of motion vectors for interframe prediction. It is therefore desired to develop a faster and more efficient method of motion vector calculation. One such method is proposed in, for example, Japanese Unexamined Patent Application Publication No. 2004-266731. The proposed video coding method alleviates memory capacity requirements of reference pictures by selectively storing a limited number of reference macroblocks that have been determined to be used in the next motion-compensated inter-coded frame.
A video coding device calculates a motion vector (abbreviated as “MV” where appropriate) corresponding to each individual macroblock and then determines a motion vector predictor (abbreviated as “MVP” where appropriate) for a current macroblock from previously calculated motion vectors of its surrounding macroblocks. The video coding device then encodes motion vector differences (abbreviated as “MVD” where appropriate) between MVs and MVPs and outputs them, together with macroblock information, as a coded video data stream. This video stream is received by a video decoding device. The video decoding device decodes the coded MVDs and macroblock information and calculates motion vectors from MVDs and MVPs, where MVP of a macroblock can be determined from motion vectors of surrounding blocks that have already been decoded. Using those motion vectors, the video decoding device reconstructs the original video stream.
A video decoding device has a memory for vector storage locations reserved to store motion vectors of already decoded macroblocks for later reference in reconstructing motion vectors of subsequent macroblocks. More specifically, those vector storage locations accommodate motion vectors of the currently processed macroblock, as well as motion vectors that have been calculated for macroblocks adjacent to the current macroblock.
In MBAFF mode, the MVP of a macroblock is determined from the current macroblock pair and its adjacent macroblock pairs. Since those macroblock pairs may not always be of the same type, the conventional MBAFF algorithm compensates for this inconsistency when reading or writing motion vectors from/to vector storage locations. Suppose, for example, that the current macroblock pair is in frame form while macroblock pairs surrounding it are in field form. In this case, the conventional algorithm converts those adjacent macroblock pairs from field form to frame form before writing motion vectors in the memory. This conversion task imposes an additional processing burden, thus degrading the efficiency of video coders and decoders.
In view of the foregoing, it is an object of the present invention to provide a device that performs interframe prediction at a high speed for video signal coding and decoding. It is also an object of the present invention to provide video coding and video decoding devices employing the proposed interframe prediction device.
To accomplish the first object stated above, the present invention provides an interframe prediction processor that performs interframe prediction of video frames on a block basis. This interframe prediction processor has an address selector that provides memory addresses of reference motion vectors according to given coding type information about a current block pair and adjacent block pairs thereof. The address selector is activated when performing motion vector prediction with reference to blocks outside the current block pair, and when given macroblock adaptive frame/field (MBAFF) identification type information indicates that a given picture is supposed to be processed in MBAFF mode allowing selection between frame prediction and field prediction for each individual block pair. The coding type information indicates whether each block pair is organized in frame form or in field form. The interframe prediction processor further has a predictor calculator to determine a motion vector predictor for each constituent block of the current block pair based on reference motion vectors read out of the provided memory addresses.
Also, to accomplish the second object stated above, the present invention provides a video coding device that encodes video signals into compressed video data by performing motion-compensated prediction between a source picture and a reference picture on a block basis. This video coding device has an address selector that provides memory addresses of reference motion vectors according to given coding type information about a current block pair and adjacent block pairs thereof. The address selector is activated when performing motion vector prediction with reference to blocks outside the current block pair, and when given macroblock adaptive frame/field (MBAFF) identification type information indicates that a given picture is supposed to be processed in MBAFF mode allowing selection between frame prediction and field prediction for each individual block pair. The coding type information indicates whether each block pair is organized in frame form or in field form. The video coding device further has an interframe prediction processor that determines a motion vector predictor for each constituent block of the current block pair based on reference motion vectors read out of the provided memory addresses, and calculates a difference between the calculated motion vector of the current block and the determined motion vector predictor, so that the calculated difference will be encoded.
Furthermore, to accomplish the second object stated above, the present invention provides a video decoding device that reconstructs original video pictures from a compressed video signal that has been produced by performing motion-compensated interframe prediction on a block basis. This video decoding device has an address selector that provides memory addresses of reference motion vectors according to given coding type information about a current block pair and adjacent block pairs thereof. The address selector is activated when performing motion vector prediction with reference to blocks outside the current block pair, and when given macroblock adaptive frame/field (MBAFF) identification type information indicates that a given picture is supposed to be processed in MBAFF mode allowing selection between frame prediction and field prediction for each individual block pair. The coding type information indicates whether each block pair is organized in frame form or in field form. The video coding device further has an interframe prediction processor that determines a motion vector predictor for each constituent block of the current block pair based on reference motion vectors read out of the provided memory addresses, and reproduces a motion vector of the current block from the determined motion vector predictor and a decoded motion vector difference.
The above and other objects, features and advantages of the present invention will become apparent from the following description when taken in conjunction with the accompanying drawings which illustrate preferred embodiments of the present invention by way of example.
Preferred embodiments of the present invention will be described below with reference to the accompanying drawings, wherein like reference numerals refer to like elements throughout. The description begins with an overview of the present invention and then proceeds to more specific embodiments of the invention. In the rest of this document, the size and shape of a macroblock (and its partitions) are indicated by a prefix representing its width and height. For example, the term “16×8 macroblock” denotes that the macroblock is 16 pixels wide by 8 pixels high. This size prefix can also be applied to those sub-macroblocks. The macroblocks with a size of 16×16 pixels are referred to as “basic macroblock.”
According to the present invention, the interframe prediction processor 1 has, among others, an address selector 2 and a predictor calculator 3 to provide the following functions. The address selector 2 is responsive to an input from the predictor calculator 3 that designates a specific block within a given picture. According to the present invention, the address selector 2 provides memory addresses of reference motion vectors based on given MBAFF type information and coding type information. MBAFF type information indicates whether the given picture is supposed to be coded in MBAFF mode or in non-MBAFF mode. MBAFF mode allows selection between frame prediction and field prediction for each block pair. Non-MBAFF mode includes frame mode and field mode. Coding type information indicates whether each pair of blocks, including the current block pair and its adjacent block pairs, is organized in frame form (i.e., in the form of interlaced top and bottom field lines) or in field form (i.e., in the form of separate top and bottom fields).
When performing motion compensation with reference to blocks outside the current block pair, and when the MBAFF type information indicates that the given picture is supposed to be coded in MBAFF mode, the address selector 2 provides memory addresses of reference motion vectors according to the coding type information about the current block pair and its adjacent block pairs. The address selector 2 may be designed to have a table to provide memory addresses of reference motion vectors, as will be described in detail later.
The predictor calculator 3 determines a motion vector predictor for each constituent block of the current block pair, based on reference motion vectors read out of the provided memory addresses.
The above-outlined interframe prediction processor 1 operates as follows. The address selector 2 receives information about MBAFF type and coding type. When performing motion compensation for a given MBAFF picture with reference to blocks surrounding a current block, the address selector 2 provides memory addresses of reference motion vectors according to the coding type of the current block pair, as well as to the coding type of the reference blocks. The predictor calculator 3 reads motion vectors out of the provided memory addresses and determines a motion vector predictor from those reference motion vectors for each block constituting the current block pair.
The next and some subsequent sections will discuss H.264 video coding and decoding devices as a more specific embodiment of the present invention. The H.264 standard offers the following options for macroblock size: 16×16, 16×8, 8×16, and 8×8. If the 8×8 mode is chosen, each of the four 8×8 macroblock partitions may be split into smaller blocks with the following sizes: 8×8, 8×4, 4×8, and 4×4 (known as macroblock sub-partitions). The baseline coding algorithm encodes each given source frame on a macroblock basis, switching between intraframe coding utilizing spatial redundancy within a frame and interframe coding utilizing temporal redundancy between successive frames. A video decoding process reverses the above steps to reconstruct original video from compressed video. While H.264 allows bidirectional motion prediction, the following description assumes forward prediction for simplicity purposes. The present invention, however, should not be limited to the forward prediction. One skilled in the art would appreciate that the present invention can also be applied to bidirectional prediction by adding some memory areas to store motion vectors for both past and future reference frames. For explanatory purposes, the next section will describe a first embodiment of the present invention, assuming non-MBAFF coding. The first embodiment will then be followed by a second embodiment of the invention, the description of which assumes MBAFF coding.
The macroblock divider 101 divides a given source picture into macroblocks of a predetermined size. The motion vector estimator 102 compares video signal components of a source macroblock with those of reference macroblocks stored in the past frame buffers 103, thereby calculating a motion vector for that source macroblock. Since motion vectors for neighboring blocks are often highly correlated, the motion vector of a specific block can be predicted from vectors of its surrounding blocks. The interframe predictor 104 thus calculates a motion vector predictor MVP from such surrounding motion vectors and then calculates a motion vector difference MVD by subtracting MVP from the true motion vector. The intraframe predictor 105, on the other hand, performs intraframe coding of a frame stored in the current frame buffer 106. The prediction mode selector 107 selects a coding mode that maximizes the efficiency of coding of the current macroblock by, for example, comparing prediction error of the interframe predictor 104 with that of the intraframe predictor 105. The prediction mode selector 107 creates coding mode parameters to indicate the selection result. The coding mode parameters are passed to the entropy coder 110 as part of data to be entropy-coded. The coding mode parameters include the size of macroblock partitions and the size of sub-macroblock partitions.
When the intraframe mode is selected, the divided source macroblocks are supplied as is from the macroblock divider 101 to the orthogonal transform & quantization unit 109. Those macroblocks are orthogonal-transformed and quantized, and the resulting transform coefficients are passed to the entropy coder 110 for entropy-coding and transmission. The transform coefficients are also decoded by the dequantization & inverse orthogonal transform unit 111 and sent into the current frame buffer 106.
When the interframe mode is selected, the interframe predictor 104 supplies the entropy coder 110 with MVD for entropy coding. In addition, a picture predicted with MVP is given to the subtractor 108 through the prediction mode selector 107. The subtractor 108 subtracts the predicted picture from the source picture on a macroblock basis and sends the result to the orthogonal transform & quantization unit 109. The quantized output data is decoded locally by the dequantization & inverse orthogonal transform unit 111 and directed to the adder 112, where the prediction picture supplied through the prediction mode selector 107 is added. The resulting picture is written into the current frame buffer 106. The entropy coder 110 encodes given data using the entropy coding algorithm, thus outputting a coded video signal carrying video data in compressed form. The reference picture stored in the current frame buffer 106 is subjected to the deblocking filter 113, where the boundary region between macroblocks are smoothed out. The resulting picture is entered to a past frame buffer 103 through the frame buffer manager 114.
A given coded video signal is entropy-decoded by the entropy decoder 201 and then dequantized and back-transformed by the dequantization & inverse orthogonal transform unit 202. The outcomes of this decoding process include, among others, coding mode parameters and, in the case of interframe coding, motion vector data in the form of MVD. Based on the decoded coding mode parameters, the prediction mode selector 206 selects either the intraframe predictor 205 or the interframe predictor 210.
When the intraframe predictor 205 is selected, the picture frame reconstructed by the dequantization & inverse orthogonal transform unit 202 is routed to the current frame buffer 204. When the interframe predictor 210 is selected, the motion vector differences MVDs are decoded back to the original motion vectors that the video coding device 100 (
The interframe predictor 210 in the above-described video decoding device 200 decodes motion vectors as follows.
Shown in the topmost part of
H.264 further allows an 8×8 macroblock 14 to be divided into smaller partitions called “sub-macroblocks.” The above-noted size prefix can also be applied to those sub-macroblocks. In the example shown in
The memory 211 stores motion vectors that have been calculated earlier for preceding macroblocks. Some of those motion vectors are used to calculate MVP of the current macroblock. The memory 211 also offers a storage space for a motion vector for the current block, as will be described in detail later. Actually, calculated motion vectors are stored in some selected vector storage locations. Such locations will be referred to hereinafter as the “representative vector storage locations,” and the motion vectors stored in those locations will be referred to as “representative vectors.”
The MVP calculator 212 specifies each relevant adjacent macroblock surrounding the current macroblock Cu to the address translator 213, thus obtaining the address of each corresponding representative vector storage location in the memory 211. With the representative vectors read out of the obtained addresses, the MVP calculator 212 calculates MVP. While there are various ways to determine MVP, the present embodiment calculates the median of motion vectors of adjacent macroblocks as a motion vector predictor MVP:
MVP=Median(MV-A,MV-B,MV-C) (1)
where MV-A, MV-B, and MV-C refer to motion vectors of adjacent macroblocks MB-A, MB-B, and MB-C, respectively. Motion vector MV-C in formula (1) may be replaced with another motion vector MV-D of macroblock MB-D in the case the current macroblock is located on the right edge of the frame, where MV-C is not available. The median of three variables x, y, and z is expressed as follows:
Median(x,y,z)=x+y+z−Min(x,Min(y,z))−Max(x,Max(y,z)) (2)
where Min(x, y) denotes the smaller of x and y, and Max(x, y) the greater of x and y.
The address translator 213 translates a given adjacent macroblock size (or sub-macroblock size) into a representative vector storage location address. The motion vector calculator 214 reconstructs a motion vector from the MVP calculated by the MVP calculator 212 and the decoded MVD as follows:
MV=MVP+MVD (3)
The resulting motion vector MV is saved in the memory 211, more particularly, in a representative vector storage location corresponding to the current macroblock Cu.
The memory 211 maintains motion vectors calculated previously for each macroblock. The MB-A vector storage manager 215 reads out one of those stored motion vectors that corresponds to a macroblock to be referenced as an adjacent macroblock MB-A in the subsequent vector prediction process and enters it to an MB-A vector storage location in the memory 211. Likewise, the MB-BCD vector storage manager 216 reads out stored motion vectors corresponding to macroblocks to be referenced as adjacent macroblocks MB-B, MB-C, and MB-D in the subsequent vector prediction process and enters them to MB-BCD vector storage locations reserved in the same memory 211.
The above-described interframe predictor 210 operates according to an interframe prediction method proposed in the present embodiment. Details of this interframe prediction will be described in the following sections, assuming that the current picture is coded in frame mode.
The memory 211 provides vector storage locations reserved for calculated motion vectors.
The memory 211 provides current-block vector storage locations 310 to accommodate motion vectors calculated for the current macroblock Cu. Also provided are adjacent-block vector storage locations 320, 331, and 332 to store motion vectors of adjacent macroblocks MB-A and MB-BCD. The current-block vector storage locations 310 can accommodate up to sixteen vectors, assuming that the current 16×16 basic macroblock Cu is partitioned into sixteen 4×4 sub-macroblocks, the minimum-size partitions.
The adjacent-block vector storage locations are divided into two classes. One class is MB-A vector storage locations 320 used to store motion vectors of macroblocks to be referenced as adjacent macroblocks MB-A. The MB-A vector storage locations 320 are mapped onto the four vertically-aligned 4×4 sub-macroblocks on the left of the current macroblock Cu. When motion vectors for the current macroblock Cu become available in the current-block vector storage locations 310, the MB-A vector storage manager 215 copies them to those MB-A vector storage locations 320.
Another class is MB-BCD vector storage locations 331 and 332 reserved to store motion vectors of macroblocks to be referenced as adjacent macroblocks MB-B, MB-C, and MB-D. The MB-BCD vector storage locations 331 are mapped onto the bottommost row of 4×4 sub-macroblocks immediately above the current macroblock Cu. This row includes adjacent macroblocks MB-B, MB-C, and MB-D. The MB-BCD vector storage locations 332 are mapped onto the bottommost row of 4×4 sub-macroblocks on the frame line to which the current macroblock Cu belongs. When motion vectors of the current macroblock Cu become available in the current-block vector storage locations 310, the MB-BCD vector storage manager 216 copies the vectors corresponding to the bottom-most sub-macroblocks to the MB-BCD vector storage locations 332. Those motion vectors MB-BCD vector storage locations 332 will be referenced later when the focus reaches the next line of the frame.
Motion vectors stored in the above-described vector storage locations in the memory 211 are updated as the processing focus moves. Suppose, for example, that the focus is moved from the current macroblock to the next macroblock 311. The vector storage locations that have been used to store motion vectors of the preceding macroblock then begins to serve the new current macroblock 311. The MB-A vector storage locations 320 are thus used to store the motion vectors at the rightmost blocks of the previous macroblock. The MB-BCD vector storage locations 331 and 332 are also rewritten in a similar way.
For convenience of explanation, the 4×4 sub-macroblock partitions of a basic macroblock are numbered as follows.
In the following sections, the block numbers b0 to b15 defined above will be used to refer to vector storage locations corresponding to macroblock partitions.
Referring back to
Referring to
Referring to the sequence diagram of
Suppose that the current focus is on the upper 16×8 macroblock Cu0 411. The MVP calculator 212 reads motion vectors from b5 of MB-A, b10 of MB-B, b10 of MB-C, and b15 of MB-D, adjacent to the 16×8 current macroblock Cu0 411. The MVP calculator 212 determines MVP from those reference motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV0 for the current macroblock Cu0 411. The calculated motion vector MV0 is stored in a vector storage location corresponding to block b0 412 at the top-left corner of the current macroblock Cu0 411.
Suppose now that the focus moves to the lower 16×8 macroblock Cu1 413. The MVP calculator 212 reads motion vectors from b13 of MB-A, b2 of Cu0 411 (as MB-B), and b7 of MB-D, adjacent to the 16×8 current macroblock Cu1 413. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV1 for the current macroblock Cu1 413. (Note that there is no calculated motion vector for macroblock X.) The calculated MV1 is then stored in a vector storage location corresponding to block b8 414 at the top-left corner of the current macroblock Cu1 413.
Referring to the sequence diagram of
Suppose that the current focus is on the left 8×16 macroblock Cu0 421. The MVP calculator 212 reads motion vectors from b5 of MB-A, b10 of MB-B, b14 of MB-C, and b15 of MB-D adjacent to the 8×16 current macroblock Cu0 421. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV0 for the current macroblock Cu0 421. The calculated motion vector MV0 is stored in b0 422 corresponding to the top-left corner of the current macroblock Cu0 421.
Suppose now that the focus moves to the right 8×16 macroblock Cu1 423. The MVP calculator 212 reads motion vectors from b1 of MB-A (=previous 8×16 macroblock Cu0), b14 of MB-B, b10 of MB-C, b11 of MB-D adjacent to the 8×16 current macroblock Cu1 423. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV1 for the current macroblock Cu1 423. The calculated motion vector MV1 is stored in b4 424 at the top-left corner of the current macroblock Cu1 423.
Referring to the sequence diagram of
Suppose that the current focus is on the top-left 8×8 macroblock Cu0 431. The MVP calculator 212 reads motion vectors from b5 of MB-A, b10 of MB-B, b14 of MB-C, and b15 of MB-D adjacent to the current macroblock Cu0 431. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV0 for the current macroblock Cu0 431. The calculated motion vector MV0 is stored in b0 432 at the top-left corner of the current macroblock Cu0 431.
Suppose that the focus moves to the top-right 8×8 macroblock Cu1 433. Then the MVP calculator 212 reads motion vectors from b1 of MB-A (=previous 8×8 macroblock Cu0 431), b14 of MB-B, b10 of MB-C, and b11 of MB-D surrounding the current macroblock Cu1 433. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV1 for the current macroblock Cu1 433. The calculated motion vector MV1 is stored in b4 434 at the top-left corner of the current macroblock Cu1 433.
Suppose that the focus moves to the bottom-left 8×8 macroblock Cu2 435. Then the MVP calculator 212 reads motion vectors from b13 of MB-A, b2 of MB-B (=top-left 8×8 macroblock Cu0 431), b6 of MB-C (=top-right 8×8 macroblock Cu1 433), and b7 of MB-D surrounding the current macroblock Cu2 435. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV2 for the current macroblock Cu2 435. The calculated motion vector MV2 is stored in b8 436 at the top-left corner of the current macroblock Cu2 435.
Suppose that the focus moves to the bottom-right 8×8 macroblock Cu3 437. Then the MVP calculator 212 reads motion vectors from b9 of MB-A (=preceding 8×8 macroblock Cu2 435), b6 of MB-B (=top-right macroblock Cu1 433), and b3 (=top-left macroblock Cu0 431) surrounding the current macroblock Cu3 437. The MVP calculator 212 determines MVP from those motion vectors, thus permitting the motion vector calculator 214 to calculate a motion vector MV3 for the current macroblock Cu3 437. The resulting vector MV3 is stored in b12 438 at the top-left corner of the current macroblock Cu3 437.
Referring to the sequence diagram of
Referring to
Referring to
Referring to
Referring to the sequence diagram of
The first embodiment of the invention saves a produced motion vector only in a specific vector storage location (e.g., the one corresponding to the top-left block) that represents the macroblock domain for which that vector is produced. As can be seen from
The above-described process permits motion vectors to be stored in vector storage locations 310, 320, 331, and 332 reserved for the current macroblock Cu and its surrounding macroblocks MB-A and MB-BCD as discussed in
The address translator 213 serves the MVP calculator 212 by translating given information specifying an adjacent macroblock into a specific address of a representative vector storage location corresponding to that macroblock. That is, the address translator 213 receives from the MVP calculator 212 a piece of information specifying a 4×4 adjacent macroblock to be referenced. More specifically, the MVP calculator 212 sends the identifier of a desired 16×16 basic macroblock, along with a specific block number (e.g., b0 to b15 shown in
As discussed earlier in
For explanatory purposes, let us assume that external vectors can be handled in the same way as internal vectors. That is, it is assumed that motion vectors in every macroblock is stored in the same way as the current macroblock Cu of
When the adjacent macroblock of interest is a 16×16 macroblock, the MVP calculator 212 may specify one of the sixteen blocks b0 to b15 in its address request. The representative vector is, however, stored only in one vector storage location corresponding to block b0 in this case, as discussed in
When the adjacent macroblock of interest is an upper 16×8 macroblock, the MVP calculator 212 may specify one of the first eight blocks b0 to b7 in its address request. The representative vector in this case is stored in a vector storage location corresponding to block b0 as discussed in
When the adjacent macroblock is a lower 16×8 macroblock (not shown), the MVP calculator 212 may specify one of the second eight blocks b8 to b15 in its address request. The representative vector in this case is stored in a vector storage location corresponding to block b8 as discussed in
Referring to
When the adjacent macroblock of interest is a top-left 8×8 macroblock, the MVP calculator 212 may specify a block b0, b1, b2, or b3. The representative vector in this case is stored in a vector storage location corresponding to block b0 as discussed in
The motion vector calculator 214 may request an address for a sub-macroblock. The following part of the description explains how the address translator 213 handles such requests, assuming, for simplicity, that the specified sub-macroblock belongs to a top-left 8×8 macroblock. The address translator 213 handles other types of sub-macroblocks in a similar way to determine a representative vector storage location uniquely from specified block size and position.
In the case of an upper 8×4 sub-macroblock of a top-left 8×8 macroblock, the motion vector calculator 214 may specify either block b0 or block b1 as an adjacent macroblock. The representative vector storage location in this case is the one corresponding to block b0. The address translator 213 thus returns the address of that vector storage location to the motion vector calculator 214.
In the case of a left 8×4 sub-macroblock of a top-left 8×8 macroblock, the motion vector calculator 214 may specify either block b0 or block b2 as an adjacent macroblock. The representative vector storage location in this case is the one corresponding to block b0. The address translator 213 thus returns the address of that vector storage location to the motion vector calculator 214.
In the case of a top-left 4×4 sub-macroblock of a top-left 8×8 macroblock, the motion vector calculator 214 specifies block b0 as an adjacent macroblock. The representative vector storage location in this case is the one corresponding to block b0. The address translator 213 thus returns the address of that vector storage location to the motion vector calculator 214.
As can be seen from the above, a relevant representative vector storage location can be determined by looking up a translation table with the macroblock size and sub-macroblock size of a specified adjacent macroblock, as well as with a specified block position (b0 to b15). This feature permits the MVP calculator 212 to reach a motion vector stored in a representative vector storage location by the motion vector calculator 214.
Although what is shown in
In the case where the desired reference motion vector is located outside the current macroblock Cu, the present embodiment operates as follows. Suppose, for example, that the current focus is on a 16×16 basic macroblock. Every reference motion vector in this case has to be found somewhere in vector storage locations 320, 331, and 332 (
The storage address for a representative motion vector depends on the block size. The following will describe a process of storing motion vectors for each block size.
This section describes how address is translated when the MVP calculator 212 refers to outside motion vectors in MB-A or MB-BCD vector storage locations. The MVP calculator 212 gives the address translator 213 a piece of information specifying a specific adjacent 4×4 block. Upon receipt of this information, the address translator 213 determines whether there is an MB-A or MB-BCD vector storage location directly corresponding to the specified block. If not, the translation table discussed earlier in
In the case where the source macroblock size of MB-A is 16×16, only a single motion vector MV0 is stored in A0 out of the four vector storage locations reserved for MB-A. While the MVP calculator 212 may specify A0, A1, A2, or A3, the translation table translates all these locations into the address of A0, thus permitting the MVP calculator 212 to read out MV0 from the same location A0 in this case, no matter which block it specifies.
In the case of 16×8 source macroblocks, two motion vectors MV0 and MV8 are stored in vector storage locations A0 and A2, respectively. When the MVP calculator 212 specifies A0 or A1, the translation table translates it to the address of A0. When the MVP calculator 212 specifies A2 or A3, the translation table translates it to the address of A2. The MVP calculator 212 can therefore obtain MV0 for A0 and A1, and MV2 for A2 and A3.
In the case of 8×16 source macroblocks, one motion vector MV4 is stored in the topmost vector storage location A0. When the MVP calculator 212 specifies A0, A1, A2, or A3, the translation table translates all these locations into the address of A0. The MVP calculator 212 therefore reads out MV0 in this case, no matter which block it specifies.
In the case of 8×8 source macroblocks, two motion vectors MV4 and MV12 are stored in vector storage locations A0 and A2, respectively. When the MVP calculator 212 specifies A0 or A1, the translation table translates it to the address of A0. When the MVP calculator 212 specifies A2 or A3, the translation table translates it to the address of A2. The MVP calculator 212 can therefore obtain MV4 for A0 and A1, and MV12 for A2 and A3.
In the case where the source macroblock is partitioned into 8×4 sub-macroblocks, motion vectors MV4, MV6, MV12, and MV14 are stored in vector storage locations A0, A1, A2, and A3, respectively. When the MVP calculator 212 specifies A0, A1, A2, or A3, the translation table translates those locations to their respective addresses.
In the case of 4×8 sub-macroblocks, motion vectors MV5 and MV13 are stored in vector storage locations A0 and A2, respectively. When the MVP calculator 212 specifies A0 or A1, the translation table translates it to the address of A0. When the MVP calculator 212 specifies A2 or A3, the translation table translates it to the address of A2.
In the case of 4×4 sub-macroblocks, motion vectors MV5, MV7, MV13, and MV15 are stored in vector storage locations A0, A1, A2, and A3, respectively. When the MVP calculator 212 specifies A0, A1, A2, or A3, the translation table translates those locations to their respective addresses.
As can be seen from
A similar address translation process takes place for MB-BCD vector storage locations. Specifically,
In the case where the source macroblock size is 16×16, a single motion vector MV0 is stored in N1 out of the four available vector storage locations of MB-BCD. When the MVP calculator 212 specifies N1, N2, N3 or N4, the translation table translates it to the address of N1, thus permitting the MVP calculator 212 to read out MV0 from the same location A0 in this case, no matter which block it specifies.
In the case of 16×8 macroblocks, a single motion vector MV8 is stored in a vector storage location N1. When the MVP calculator 212 specifies N1, N2, N3 or N4, the translation table translates it into the address of N1, thus permitting the MVP calculator 212 to read out MV8, no matter which block it specifies.
In the case of 8×16 macroblocks, motion vectors MV0 and MV4 are stored in vector storage locations N1 and N3, respectively. When the MVP calculator 212 specifies N1 or N2, the translation table translates it into the address of N1. When the MVP calculator 212 specifies N3 or N4, the translation table translates it into the address of N3. The MVP calculator 212 can therefore obtain MV0 for N1 and N2, and MV4 for N3 and N4.
In the case of 8×8 macroblocks, motion vectors MV8 and MV12 are stored in vector storage locations N1 and N3, respectively. When the MVP calculator 212 specifies N1 or N2, the translation table translates it into the address of N1. When the MVP calculator 212 specifies N3 or N4, the translation table translates it into the address of N3. The MVP calculator 212 can therefore obtain MV8 for N1 and N2, and MV12 for N3 and N4.
In the case of 8×4 sub-macroblocks, motion vectors MV8 and MV12 are stored in vector storage locations N1 and N3. When the MVP calculator 212 specifies N1 or N2, the translation table translates it into the address of N1. When the MVP calculator 212 specifies N3 or N4, the translation table translates it into the address of N3.
In the case of 4×8 sub-macroblocks, motion vectors MV8, MV9, MV12, and MV13 are stored in vector storage locations N1, N2, N3, and N4, respectively. When the MVP calculator 212 specifies N1, N2, N3, or N4, the translation table translates those locations to their respective addresses.
In the case of 4×4 sub-macroblocks, motion vectors MV10, MV11, MV14, and MV15 are stored in vector storage locations N1, N2, N3, and N4, respectively. When the MVP calculator 212 specifies N1, N2, N3, or N4, the translation table translates those locations to their respective addresses.
As can be seen from
The above sections have described how the address translator 213 provides memory address for the MVP calculator 212 to read reference motion vectors for vector prediction. This arrangement of the present embodiment eliminates the need for the motion vector calculator 214 to expand each calculated motion vector in multiple memory locations for the purpose of later reference. By removing unnecessary vector write cycles, the present embodiment reduces the time required to calculate motion vectors. In the case of, for example, a 16×16 macroblock, the conventional vector calculation process takes sixteen write cycles to save a calculated motion vector, four write cycles to fill the MB-A vector storage locations, and another four write cycles to fill the MB-BCD vector storage locations. This means that the process includes 24 memory write cycles. In contrast to this, the present embodiment takes one write cycle to save a calculated motion vector, another write cycle to fill an MB-A vector storage location, and yet another write cycle to fill an MB-BCD vector storage location. That is, the present embodiment only requires three write cycles to process a 16×16 macroblock. This means that the number of write cycles can be reduced by maximum 21 cycles.
Think of 8×8 macroblocks, for another instance. While the conventional process takes 24 write cycles as in the case of 16×16 macroblocks, the present embodiment consumes eight cycles to save calculated motion vectors, two cycles to fill MB-A vector storage locations, and another two cycles to fill MB-BCD vector storage locations. The present embodiment only requires 12 memory write cycles, achieving a reduction of maximum 12 cycles. The effect of this write cycle reduction will be more prominent when the vector calculation process deals with a larger size of macroblocks, or when it processes wider and finer pictures.
The above sections have discussed motion vector prediction using an address translation mechanism, assuming pictures coded in frame mode. The proposed method, however, can also be applied to field mode. Specifically, the proposed method is used to process each set of top-field lines and bottom-field lines that constitute a video picture, which achieves the same effect as in the case of frame mode.
This and subsequent sections describe a video decoding device according to a second embodiment of the present invention and, more particularly, how its interframe predictor decodes motion vectors. The second embodiment is different from the first embodiment in its capability of MBAFF processing. The following description will focus on this difference, using the same reference numerals as in the first embodiment to refer to like elements in the second embodiment.
In MBAFF mode, the term “current macroblock pair” is used to refer to a pair of 16×16 macroblocks that are currently processed, while two macroblocks adjacent to the current macroblock pair are called an “adjacent macroblock pair.” In the following description of the second embodiment, the suffixes “a” and “b” are attached to reference numerals of the top macroblock and bottom macroblock belonging to a specific macroblock pair. According to the second embodiment, the MVP calculator 212 stores motion vectors of a current macroblock pair, regardless of whether it is coded in frame mode or field mode.
The memory 211 provides current-block vector storage locations 310a and 310b to store motion vectors calculated for current macroblocks Cu(top) and Cu(bottom). Also provided are MB-A vector storage locations 320 and MB-BCD vector storage locations 331a, 331b, 332a, and 332b to store motion vectors of adjacent macroblocks MB-A and MB-BCD.
Eight MB-A vector storage locations 320 are reserved to store motion vectors of macroblocks to be referenced as adjacent macroblocks MB-A. They are mapped onto vertically-aligned 4×4 sub-macroblocks immediately to the left of the current macroblocks Cu(top) and Cu(bottom). When motion vectors for the current macroblock Cu(top) and Cu(bottom) become available in their respective vector storage locations 310a and 310b, the MB-A vector storage manager 215 copies motion vectors at the right-most column of 4×4 sub-macroblocks to those MB-A vector storage locations 320. Details of this processing will be discussed in a later section.
MB-BCD vector storage locations 331a, 331b, 332a, and 332b, on the other hand, are reserved for motion vectors of macroblocks to be referenced as adjacent macroblocks including at least MB-B, MB-C, and MB-D. Specifically, an adjacent pair of macroblocks immediately above the current macroblocks Cu(top) and Cu(bottom) are referenced when predicting motion vectors in MBAFF mode. MB-BCD vector storage locations 331a and 331b correspond to the bottommost row of 4×4 sub-macroblocks of the top and bottom halves of that adjacent macroblock pair. Another set of MB-BCD vector storage locations 332a and 332b is mapped onto the bottommost row of 4×4 sub-macroblocks on the two frame lines to which the current macroblocks Cu(top) and Cu(bottom) belong. When motion vectors of the current macroblocks Cu(top) and Cu(bottom) become available in the current-block vector storage locations 310a and 310b, the MB-BCD vector storage manager 216 copies the vectors corresponding to the bottom-most 4×4 sub-macroblocks to the MB-BCD vector storage locations 332a and 332b.
When the MVP calculator 212 makes reference to existing motion vectors, the address translator 213 helps it by providing the address of those reference vectors. With this feature, the second embodiment eliminates the need for nine storage locations b0, b1, b2, b3, b4, b6, b8, b9, and b12 indicated by the hatching in
Likewise,
This section will describe how an address translator provides address of vector storage locations in MBAFF mode, with a focus on reference motion vectors outside the current macroblock. See the first embodiment for reference motion vectors inside the current macroblock.
While not specifically shown in
The address translator 213a first determines whether each given picture is coded in MBAFF mode or non-MBAFF mode. If it is non-MBAFF coded, the address translator 213a uses the first and second translation tables Ta1 and Ta2 to process the given picture. If it is MBAFF coded, the address translator 213a uses the first and third translation tables Ta1 and Ta3 to do the same.
When using the third translation table Ta3, the address translator 213a considers the state of MBFIELD flag, surrounding macroblock type flag, and top/bottom flag. MBFIELD flag indicates the coding type of the current block, i.e., whether the current macroblock is organized in field form or in frame form. Surrounding macroblock type flag indicates the coding type of macroblocks to which adjacent macroblocks MB-A and MB-BCD belong. Top/bottom flag indicates whether the current macroblock is a top field or a bottom field, in the case of field coding.
As mentioned in the previous section, the MBAFF coding allows frame coding and field coding to be combined variously for a current macroblock pair and its left (or upper) adjacent macroblock pair. This section describes various patterns of adjacent macroblock locations in MBAFF, which are derived from multiple combinations of frame coding and field coding. For simplicity, the frame/field patterns will be discussed on an individual macroblock basis, rather than on a macroblock pair basis.
The adjacent macroblock locations of MB-A and MB-BCD shown in
Notice that the pattern shown in
The MBAFF coding also allows various combinations of coding types for a current macroblock and an upper adjacent macroblock pair. The following will describe various patterns of adjacent macroblock locations in MBAFF mode which are derived from combinations of frame coding and field coding.
This section gives details of the aforementioned third translation table Ta3.
As can be seen from
In addition to provide the same functionality as the first embodiment, the video decoding device 200 according to the second embodiment supports motion vector prediction in MBAFF mode. The second embodiment accomplishes this as quickly as in non-MBAFF mode by using a third translation table Ta3 designed to locate reference motion vectors (or adjacent macroblocks MB-A or MB-BCD) outside the current macroblock based on a given MBFIELD flag, surrounding macroblock type flag, and top/bottom flag.
Also the video decoding device 200 of the second embodiment uses a first translation table Ta1, not only to support non-MBAFF coding, but also to locate reference motion vectors (or adjacent macroblocks MB-A or MB-BCD) inside the current macroblock in MBAFF mode. This first translation table Ta1 eliminates the need for reserving nine vector storage locations (specifically, b0, b1, b2, b3, b4, b6, b8, b9, and b12 as indicated by hatching in
The preferred embodiments of the proposed interframe prediction processor, video coding device, and video decoding device have been described. The present invention, however, should not be limited to those specific embodiments. The elements of each embodiment may be replaced with some other functionally equivalent elements. The elements of the present invention may also operate together with other elements or steps. Further, two or more features of the above-described embodiments may be combined in various ways.
The above-described processing mechanisms of the proposed interframe prediction processor are actually implemented on a computer system, the instructions being encoded and provided in the form of computer programs. A computer system executes such programs to provide the intended functions of the present invention. For the purpose of storage and distribution, those programs may be stored in a computer-readable storage medium, which include: magnetic storage devices, optical discs, magneto-optical storage media, and semiconductor memory devices. Magnetic storage devices include hard disk drives (HDD), floppy disks (FD), and magnetic tapes. Optical disc media include digital versatile discs (DVD), DVD-RAM, compact disc read-only memory (CD-ROM), CD-Recordable (CD-R), and CD-Rewritable (CD-RW). Magneto-optical storage media include magneto-optical discs (MO).
Portable storage media, such as DVD and CD-ROM, are suitable for the distribution of program products. Network-based distribution of software programs may also be possible, in which case several master program files are made available on a server computer for downloading to other computers via a network.
A user computer stores necessary software components for interframe prediction in its local storage unit, which have previously been installed from a portable storage media or downloaded from a server computer. The computer executes the programs read out of the local storage unit, thereby performing the programmed functions. As an alternative way of program execution, the computer may execute programs, reading out program codes directly from a portable storage medium. Another alternative method is that the user computer dynamically downloads programs from a server computer when they are demanded and executes them upon delivery.
As can be seen from the above sections, the present invention determines memory addresses of reference motion vectors according to given coding type information about a current block pair and its adjacent block pairs, when performing motion vector prediction with reference to blocks outside the current block pair. The present invention accomplishes this in MBAFF mode as quickly as in non-MBAFF mode, thus reducing the time required to predict motion vectors.
The proposed interframe prediction processor may be used together with a reference picture memory and a reference picture storage manager. If this is the case, the addresses of reference motion vectors can be read out of memory addresses that are calculated automatically. This feature eliminates the need for duplicating a calculated motion vector in multiple storage locations in the memory to allow for the use of the minimum block size. The proposed interframe prediction processor thus reduces the processing time of motion vector calculation.
The foregoing is considered as illustrative only of the principles of the present invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and applications shown and described, and accordingly, all suitable modifications and equivalents may be regarded as falling within the scope of the invention in the appended claims and their equivalents.
| 标题 | 发布/更新时间 | 阅读量 |
|---|---|---|
| 基于子块的预测 | 2020-05-08 | 293 |
| 一种确定运动信息的方法、帧间预测方法及装置 | 2020-05-11 | 777 |
| 视频编解码方法、装置、计算机设备和存储介质 | 2020-05-11 | 345 |
| 从仿射运动预测的非仿射块 | 2020-05-12 | 176 |
| 一种图像预测方法和设备 | 2020-05-12 | 528 |
| 模式依赖的仿射继承 | 2020-05-11 | 679 |
| 仿射的运动矢量访问范围 | 2020-05-11 | 218 |
| 不同视频块尺寸的仿射模式计算 | 2020-05-13 | 331 |
| 两步帧间预测 | 2020-05-13 | 206 |
| 一种帧间预测方法和装置 | 2020-05-12 | 432 |
高效检索全球专利专利汇是专利免费检索,专利查询,专利分析-国家发明专利查询检索分析平台，是提供专利分析，专利查询，专利检索等数据服务功能的知识产权数据服务商。
我们的产品包含105个国家的1.26亿组数据，免费查、免费专利分析。
专利汇分析报告产品可以对行业情报数据进行梳理分析，涉及维度包括行业专利基本状况分析、地域分析、技术分析、发明人分析、申请人分析、专利权人分析、失效分析、核心专利分析、法律分析、研发重点分析、企业专利处境分析、技术处境分析、专利寿命分析、企业定位分析、引证分析等超过60个分析角度，系统通过AI智能系统对图表进行解读，只需1分钟，一键生成行业专利分析报告。
-
4 仿射运动信息导出
