Apparatus for motion estimation using a two-dimensional processing element array and method therefor
专利汇可以提供Apparatus for motion estimation using a two-dimensional processing element array and method therefor专利检索，专利查询，专利分析的服务。并且An apparatus for motion estimation and method therefor are provided. The apparatus includes a processing element (PE) array unit that includes a delay unit array and a PE array. The delay unit array outputs different data flows of current data to the PE array with respect to checking points in one step of an N-step seach algorithm, while a regular data flow of reference data is fed into the PE array. One search step of the N-step search algorithm for motion estimation can be performed while the pixel data of a search area is read in a regular pixel scan order. When the search area is read completely, the search step is completed. In this way, the PE array unit achieves the N-step search algorithm. Further, the PE array unit can be configured to perform half-pel motion estimation with respect to a best point found in a full-pel search.，下面是Apparatus for motion estimation using a two-dimensional processing element array and method therefor专利的具体信息内容。
What is claimed is:
1. Field of the Invention
The invention relates in general to an apparatus for motion estimation and method therefor, and more particularly to an apparatus for motion estimation using a two-dimensional processing element array and a method therefor.
2. Description of the Related Art
Video compression or video encoding is essential to a variety of multimedia applications in electronic devices. Motion estimation is one of the key elements to video compression. MPEG-4, for example, one of the mainstream video compression standards, is widely employed in a variety of applications and devices ranging from high-bit-rate, high quality video devices, such as high definition television (HDTV) or digital versatile disk (DVD) player, to low-bit-rate mobile processing devices, such as mobile phone or digital personal assistant (PDA), with video capability. During MPEG-4 video encoding, motion estimation consumes relatively a large amount of computation time and most of the system resources. For MPEG-4 video encoding, about 60 to 80% of the computation time is consumed in motion estimation. With regard to computation loading and resource usage, motion estimation is a critical factor to implement MPEG-4 encoders in processing devices, particularly in mobile processing devices, which typically have limited resources including limited power capacity, limited memory resource, and limited processing power.
Complexity of the encoders for video compression is dominated by motion estimation. Employing temporal redundancy of adjacent frames in a video sequence, motion estimation is aimed to find a motion vector by which a current macroblock in a current frame can be predicated from a reference macroblock in a reference frame, where the reference macroblock has a minimum error measure as compared with the current macroblock. Many block matching algorithms (BMAs) for motion estimation have been developed for performance improvement and/or reduced hardware complexity. Among the BMAs, step search algorithms, such as three step search (TSS), or four step search (4SS), are developed to reduce computation redundancy and improve performance. However, the data flow employed in these search algorithms are irregular so that hardware implementation of the algorithms is complex. Besides, the overall performance of a processing device performing the step search algorithm cannot achieve the theoretic performance of the algorithm in view of limited resources provided by the processing device, particularly crucially to the mobile processing device.
Many architectural solutions for implementing BMA can be found in the literature. For example, Costa et al., “A VLSI Architecture For Hierarchical Motion Estimation”, IEEE Transactions on Consumer Electronics, Vol. 41, No. 2, May 1995, pp. 248-257, and Kim et al., “A Fast Motion Estimator for Real-Time System”, IEEE Transactions on Consumer Electronics, Vol. 43, No. 1, February 1997, pp. 24-33, proposed hardware architecture based on the TSS algorithm and concentrated on data flow within processing element (PE) array. However, data flows within the PE array employed in these hardware architectures are complex and dedicated to the TSS, causing some problems outside the PE array.
First, complex data flow within the PE array results in complex implementation of the PE array control circuit. Secondly, complex data flow within the PE array inherently leads to repetition of memory read operations for the pixel data during motion estimation. In a typical encoder, a memory bus coupled to the motion estimation architecture and frame memory and other units of the encoder will be busy for those repeated read operations for the pixel data, and the overall performance would thus be degraded. Although this problem can be straightforwardly resolved by providing additional pixel data memory blocks for buffering pixel data from the frame memory and loading the required pixel data into the memory blocks before each search step of the TSS algorithm, overall performance of motion estimation would still be reduced and higher hardware cost for memory is required. In addition, elaborate design of data flow dedicated to the TSS algorithm hinders the utilization of the architectures for other step search algorithms, such as the FSS algorithm. With respect to a limited resource environment, such as mobile processing devices, the above described problems outside the PE array are crucial to hardware implementation and must be carefully considered in order to make the device successful and possible for end users of the devices.
Therefore, it is desirable to provide a motion estimation architecture to resolve the above described problems and to provide expandability and flexibility in view of circuit design.
It is therefore an object of the invention to provide an apparatus for motion estimation with a two-dimensional processing element (2D PE) array and a method therefor. According to the invention, a data flow scheme for within the PE array is provided to reduce hardware complexity of the control hardware of the 2D PE array. With the data flow scheme, number of times of memory access is reduced and a reduced computation time can be achieved, thereby achieving less power consumption. The 2D PE array can also benefit from its structure and the data flow scheme. Control of the 2D PE array is regular and simple, and a reduced circuit area for the motion estimation system is achieved. A motion estimation system using the 2D PE array unit is therefore suitable for a mobile processing device, such mobile phone or PDA, which is with a limited power supply.
According to one of the objects of the invention, an apparatus for motion estimation is provided to include a processing element (PE) array unit. The PE array unit includes a delay unit array and a processing element (PE) array. The delay unit array includes a plurality of horizontal delay units (HDUs) and a plurality of vertical delay units (VDUs). There are 3 rows of HDUs, each row having a first HDU and a second HDU, each HDU including an input terminal and an output terminal, wherein in each row, the output terminal of the first HDU is connected to the input terminal of the second HDU. There are a first VDU and a second VDU, each having an input terminal and an output terminal, wherein the input terminal of the first VDU is connected to the input terminal of the first HDU of the first row, the output terminal of the first VDU is connected to the input terminal of the first HDU of the second row and the input terminal of the second VDU, the output terminal of the second VDU is connected to the input terminal of the first HDU of the third row. The PE array includes 3 rows of processing elements (PEs), each row having first, second, and third PEs, each PE including a first input terminal and a second input terminal, an error measure output terminal, and a control terminal. In each row, the second input terminal of the first PE is connected to the input terminal of the first HDU; the second input terminal of the second PE is connected to the output terminal of the first HDU; the second input terminal of the third PE is connected to the output terminal of the second HDU; wherein each PE calculates an error measure accumulatively between reference data at the first input terminal and pixel data at the second input terminal when the control terminal is enabled.
In one embodiment, the PE array unit is configured to perform a search step of N-step search algorithm for motion estimation while the pixel data of the pixels in a search area is reading in a regular pixel scan order, wherein a number of macroblocks of the search area are compared to a current macroblock in parallel. When the reading of the search area is completed, the search step is completed and a minimum error measure can be determined.
In one embodiment, a configuration of the 2D PE array unit for performing full-pel motion estimation is provided to perform FSS algorithm for motion estimation in a second embodiment of the invention.
According to one of the objects of the invention, a method for full-pel motion estimation is provided. A search step of N-step search algorithm for motion estimation is completed while the pixel data of the pixels in a search area is reading in a regular pixel scan order, wherein a number of macroblocks of the search area are compared to a current macroblock in parallel.
In another embodiment of the invention, a motion estimation system architecture is shown by which motion estimation is achieved and integrated in a circuit.
Based on the configuration of the motion estimation method, regular data flows from a current memory and a reference memory are in a sequential, line after line, manner and control circuit for controlling the PE array unit can thus be implemented in a simplified manner.
According to other object of the invention, the 2D PE array unit is expandable and flexible in design and can be further utilized to perform motion vector refinement with fractional pixel accuracy, such as half-pel or quarter-pel motion estimation.
Other objects, features, and advantages of the invention will become apparent from the following detailed description of the preferred but non-limiting embodiments. The following description is made with reference to the accompanying drawings.
A two-dimensional processing element (2D PE) array unit is provided in a first embodiment of the invention. This array unit can be configured to perform a search step of N-step search algorithm for motion estimation while the pixel data of the pixels in a search area is reading in a regular pixel scan order, wherein a number of macroblocks of the search area are compared to a current macroblock in parallel. A configuration of the 2D PE array unit for performing full-pel motion estimation is provided to perform FSS algorithm for motion estimation in a second embodiment of the invention. Notably, 9 macroblocks of the search area are compared to a current macroblock in parallel while the pixel data of the pixels in a search area is being read in a pixel scan order according to the invention. Based on the configuration, regular data flows from a current memory and a reference memory are designed and control circuit for controlling the PE array unit can be implemented in a simplified manner. In a third embodiment of the invention, a motion estimation system architecture is shown by which motion estimation is achieved and integrated in a circuit. The 2D PE array unit is expandable and flexible in design. In other embodiments, the 2D PE array can be further utilized to perform half-pel motion estimation.
Two-Dimensional processing Element (2D PE) Array Unit
Referring to
In
The connection between the PE array and the delay unit array is illustrated in
The 2D PE array unit shown in
In practical applications, a specified error measure is chosen to be performed in implementing the 2D PE array unit. Any error measures, for example, sum of absolute differences (SAD), mean squared error (MSE), or mean absolute error (MAE), can be adopted in the 2D PE array unit, and one or some of error measure schemes can be embedded or used in the 2D PE array unit selectively. Preferably, SAD is adopted in the following embodiments for sake of illustration. Referring to
Configuration of the 2D PE Array Unit for Performing Full-Pel Motion Estimation
Referring to
Four Step Search Algorithm
In this embodiment, the four step search (FSS) algorithm for motion estimation is to be performed by the 2D PE array unit in
In the FSS algorithm, a step indicates a search for a minimum BDM point within a search area. In practical applications, a current memory is required to store pixel data of a frame currently to be decoded, and a reference memory is employed to store pixel data of a reconstructed frame obtained by decoding a previous decoded frame, wherein the reconstructed frame is used as a reference frame for the current frame to be decoded. In the reference memory, the pixel data, called reference data (Ref_Data), corresponding to a pixel in the reconstructed frame is a luminance pixel value of 8 bits. In the current memory, the pixel data, called current data (Curr_Data), corresponding to a pixel in the current frame is a luminance pixel value of 8 bits. In one step, a search area, as shown in
Pixel Scan Order
Referring to
Likewise, the pixel scan order for the current macroblock, or the reading of pixel value of the current macroblock, is sequential, pixel by pixel, line after line. If PE0 is enabled, i.e. when the enabling signal applied to the control terminal PE0en of PE0 indicates “enabled”, the pixel values of the current macroblock are read in the pixel scan order for the current macroblock. In one embodiment, when PE0 is enabled, a piece of current data is read immediately before a piece of reference data is read. In
Delay Unit Array
The scanning of the search area and that of the current macroblock are similarly in a sequential, pixel by pixel, line after line manner. In the second embodiment, while the scanning of the search area is completed, 9 error measures associated with 9 checking points are determined as well as the MBDM in the step. With the pixel scan order for the current macroblock above described, the delay unit array provides 8 different data flows with specific delay times to the respective second input terminals of the PEs in order that the pixel values from the search area and those from the output terminals of the delay unit array are correctly fed into the PEs.
In the FSS algorithm, step size is 2 in the first, second, and third steps, and step size changes to 1 in the final step. Each of the HDUs has a delay time of STEP_SIZE time units while each of the VDUs has a delay time of STEP_SIZE×P, wherein P is the width (number of pixels) of the macroblock, and P=16 in the embodiment. Referring to
For example, in the first step of the FSS algorithm, step size is 2. Take PE1 as example. PE1 is responsible for determining the error measure between the current macroblock and the macroblock in the search area with a starting point at (2,0) of the search area. Thus, PE1 is enabled when Ref_Data corresponding to (2, 0) to (17, 0) of the search area is sequentially fed into the first input terminal A1 of PE1. At the same time, Curr_Data corresponding to (0, 0) to (15, 0) of the current macroblock is required to be sequentially fed into the second input terminal B1 of PE1. Referring to
In addition, the HDU and VDU are also called delay lines and can be implemented by another logic circuits. Notably, if a step search algorithm to be performed by the 2D PE array unit has different step sizes during different search step, the number of FFs, for example, of the HDUs and VDUs can be modified according to the requirements for the step search algorithm.
Control of the PE Array
Each PE of the PE array has a control terminal PEZen, where Z indicates a number from 0 to 8. Referring to
Referring to
For example, when Ref_Data corresponds to (2, 2) is read, the control logic circuit determines that PE0_enable_cycle, PE1_enable_cycle, PE3_enable_cycle, and PE4_enable_cycle, (4 sub-areas) include pixel (2, 2). For the 4 sub-areas that are determined to include pixel (2, 2), enabling signals, i.e. PE0_Enable, PE1_Enable, PE3_Enable, PE4_Enable, are enabled and applied to the corresponding control terminals, i.e. PE0en, PE1en, PE3en, PE4en, of the PEs that corresponds to the corresponding checking point, i.e. (0, 0), (2, 0), (0, 2), (2, 2).
Consistent with the second embodiment for performing the FSS algorithm, TABLE 1 lists enabling conditions for the 9 PEs specifically. TABLE 1 specifies the conditions that enable the enabling signals, denoted by PEZ_Enable (Z=0 to 8); when Ref_Data corresponding to a pixel (X, Y) of the search area is included in the sub-areas. It should be noted that conditions in the second column of TABLE 1 defines the sub-areas for the first n-1 steps for full-pel motion estimation, while conditions in the third column defines the sub-areas for the final step for full-pel motion estimation. In addition, the enabling signals, PEZ_Enable (Z=0 to 8), in the second embodiment, are fed into the control terminals PEZen (Z=0 to 8), of PE0 to PE8, respectively.
Architecture of a Motion Estimation System
Referring to
The motion estimation unit 1100 includes a 2D PE array unit 100, a multiplexer 1150, a register unit 1160, and a minimum SAD determination unit 1170.
The memory reading unit 1500 is a memory reading interface for the motion estimation system 1000, wherein the memory reading interface can be implemented to be compliant with at least one communication protocol that is employed by a memory bus 10 coupled to the motion estimation system 1000. The memory bus 10, for example, is coupled to a reference memory and a current memory, and thus the motion estimation system 1000 can read current data and reference data from the current memory and the reference memory via the memory reading unit 1500.
The control unit 1600 is used to count for a step search. The control unit 1600 can be a finite state machine, for example, including two counter circuits, X counter and Y counter, to count for a step search. The X counter counts how many pixels whose pixel value is read in a row of a search area. The Y counter counts how many pixel columns whose pixel values are read in the search area. The X counter increases by one when a piece of Ref_Data, corresponding to a pixel in the search area, is read. The Y counter increases by one when X counter reaches a predetermined value, denoted by X_max_count, and then X counter is reset to 0. When Y counter reaches y_range, the step of the step search algorithm is ended. X_max_count is the width of the search area (number of pixels), i.e. X_max_count=x_range. In step 1 to step n-1 of full-pel motion estimation, X_max_count=x_range=macroblock_size+STEP_SIZE×2. For example, in FSS algorithm, X_max_count=16+2×2=20, where STEP_SIZE=2 except for the final step. For the final step of full-pel motion estimation, X_max_count=x_range but the SIZE_SIZE may be changed to a reduced value. In the final step of the FSS algorithm, X_max_count=16+1×2=18, where STEP_SIZE is 1. The memory reading unit 1500 generates a memory read signal, denoted by Ref_ready, to the control unit 1600. The memory read signal is used to inform the X and Y counters to update their count values. For example, Ref_ready is set to be enabled, e.g. a high level, when a piece of Ref_Data, corresponding a pixel of the search area, is read from a memory, e.g. the reference memory. The PE enabling cycles are determined according to the current count values X and Y from the X and Y counters.
The address generation unit 1700 includes a PE enabling logic circuit 1750 and a motion vector (MV) generation logic circuit 1770. The PE enabling logic circuit 1750 receives the current count values X and Y from the X and Y counters of the counter unit 1600; generates enabling signals according to the current count values X and Y and TABLE 1; and outputs the enabling signals to the 2D PE array unit 100 of the motion estimation unit 1100. As above described, after the scanning of the search area, 9 error measures, e.g. 9 SADs corresponding to the nine checking points in the second embodiment, are obtained, and a minimum error measure is determined and outputted by the minimum SAD determination unit 1170. The address generation unit 1700 receives the minimum error measure outputted by the minimum SAD determination unit 1170. The MV generation logic circuit 1770 generates a motion vector in the final step of the search algorithm. In addition, the address generates unit 1700 generates memory addresses to the memory reading unit 1500 so that reference data and current data are read from the memory reading unit 1500 and fed into the motion estimation unit 1100.
The operation of the motion estimation system 1000 is illustrated to perform an N-step search algorithm for motion estimation. Suppose that the motion estimation system 1000 operates with a clock signal, CLK. First, the 2D PE array unit is configured, as shown in
Specifically, during configuration of the 2D PE array unit 100, the HDUs and VDUs of the 2D PE array unit 100 are configured according to the step size of the current step of the step search algorithm. For example, when STEP_SIZE is set to 2 in the first step of the FSS algorithm for full-pel motion estimation, the HDUs, as shown in
During full-pel motion estimation, Ref_Data of the search area is being read, sequentially, line after line. In this embodiment, when PE0_Enable indicates “enabled”, or is asserted, a piece of current data, corresponding to a pixel of the current macroblock, is read before a piece of reference data, corresponding to a pixel of the search area, is read.
In one embodiment, efficient power reduction is achieved by using gated clock technique in the HDUs and VDUs of the 2D PE array unit 100 to control the shift registers. The memory read signal, Ref_ready, generated by the memory reading unit 1500 is applied in controlling the delay unit array of the 2D PE array unit 100. For example, in a full-pel motion estimation, the HDU enabling signals are set to a logic state corresponding to that of the memory read signal Ref_ready, and the VDU enabling signals is set a logic state equal to the result of the logic expression (Ref_ready & (X_count<16)), wherein Ref_ready is set to a high state when Ref_Data of a pixel of a search area is read from the reference memory. The HDU enabling signal is fed into the HEN terminal of the HDU, as shown in
Performance
In a MPEG-4 environment, for example, macroblock size is 16×16 pixels. It is assumed that a piece of reference data, corresponding to a pixel in the search area, Ref_Data, which is byte aligned, is read in one cycle, and 4 pieces of current data, corresponding to 4 consecutive pixels in the macroblock, Curr_Data, which are word aligned, is read in one cycle. In one embodiment, a modification of the motion estimation unit 1100 is illustrated in
Advantages The 2D PE array unit in the above embodiments is constructed with 9 PEs working in parallel, being supplied with data flows in simple orders, and being controlled correspondingly.
Since the pixel scan order, as shown in
The reference data and current data that are fed into the 2D PE array unit are suitably reused during the computation of a motion estimation. The computation speed of the 2D PE array unit is 9 times faster than that of a conventional one with only one PE.
In addition, the number of times of memory access of the 2D PE array unit is 9 times less than that of a conventional one with only one PE. Since power dissipation is proportional to the number of times of memory access and a reduced number of times of memory access is achieved, the 2D PE array unit efficiently saves power. The motion estimation system using the 2D PE array unit is therefore suitable for a mobile processing device, such mobile phone or PDA, which is with a limited power supply.
Further, in the motion estimation system according to the invention, a reduced number of access times to the memory bus is achieved, the utilization of the memory bus is increased.
Memory resource is also saved because additional large memory blocks, as used in some conventional approaches, for buffering reference data and current data are not needed. According to the embodiments of the invention, the computation of a motion estimation is performed while the reference data is being fed into the 2D PE array unit.
Furthermore, the 2D PE array unit is a flexible architecture that can be adaptable to different motion estimation algorithms and can be extendable its utilization. In particular, as disclosed in the above embodiments of the invention, the 2D PE array unit is configured to perform N-step search algorithm for motion estimation. The 2D PE array unit can be utilized in a motion estimation system supporting a specific type of algorithm. In addition to the FSS algorithm, any N-step search algorithm, such as three step search or 3-3-3-1 search algorithms for motion estimation can therefore be performed using the 2D PE array unit, where the first to fourth steps of the 3-3-3-1 search algorithm have step sizes of 3, 3, 3, and 1, respectively. A motion estimation system with the 2D PE array unit can also support a variety of algorithms, e.g. FSS and TSS algorithms, selectively.
Although originally configured for full-pel motion estimation, the 2D PE array unit shown in
In the following, a configuration of the 2D PE array unit in
In order to obtain an optimal benefit from the parallelism and pipelining that are inherent in the configuration of the 2D PE array unit in
Configuration of the 2D PE Array Unit for Performing Half-Pel Motion Estimation
Referring to
In
Half-Pel Values Generation
In order to provide a group of four half-pel values at one time when a full-pel value is read, a half-pel values generation unit including two additional circuits is employed with the 2D PE array unit configured in
A=(a+b+c+d+2-rounding)>>2,
B=(b+d+1-rounding)>>1,
C=(c+d+1-rounding)>>1,
D=d,
where A, B, C, D are half-pel values, and a, b, c, d are full-pel values.
Search Area and Checking Points
In the half-pel motion estimation, the search area is defined differently from that in full-pel motion estimation: search area=x_range·y_range, wherein x_range=16+STEP_SIZE×2=18, and y_range=16+STEP_SIZE×2=18, STEP_SIZE is 1. Particularly, checking points in the half-pel search are defined around a best point, regarded as R(0, 0), found in a full-pel search. Referring to
Half-Pel Motion Estimation Operation
The operation of the half-pel motion estimation is described as follows.
First, a 2D PE array unit is configured as shown in
Secondly, a pre-fetching cycle begins for generating the first group of four half-pel values. In the pre-fetching cycle, full-pel values, Ref_Data, corresponding to the pixels of the search area, DR(−1, −1) to DR(16,16), are sequentially read and fed into the half-pel values generation unit. In this embodiment, a full-pel value of the search area is fed into the input terminal Ref_In of the preparation delay unit 2200. When the 20-th full-pel value DR(0, 0) from the search area is applied to the delay unit 2200, the full-pel values a, b, c, d can be outputted at the same time and then be applied to the half-pel generating circuit 2300. Four half-pel values A, B, C, D are generated by the half-pel generating circuit 2300 at the same time and are fed into the 2D PE array unit in
Thirdly, the 2D PE array unit in
In order to fulfil the requirements of the operation, pixel scan order, delay units, and the control of the PE array are required to be specified with respect to the half-pel motion estimation to be performed by the 2D PE array unit in
Pixel Scan Order for Half-Pel Motion Estimation
The pixel scan order for the search area in half-pel motion estimation is similar to that in full-pel motion estimation as illustrated in
Likewise, pixel values of the current macroblock are read sequentially, line after line, from the starting point C(0, 0) to the ending point C(15, 15). However, it should be noted that a prefetch cycle, as above described, is to be elapsed before the scanning of the current macroblock begins, wherein the first group of four half-pel values, i.e. A, B, C, D as indicated in
In
In order for each of the PEs in
With respect to PE0, after the full-pel value of the last pixel in a row of the current macroblock is read and applied to the second input terminal B0, PE0 is disabled. In this time, the scanning of the search area continues. In addition, the scanning of the current macroblock pauses until the first pixel of the next row of the search area is scanned. When the first pixel of the next row of the search area is to be scanned, PE0 is enabled again and the scanning of the current macroblock continues. In this way, the half-pel value of the next row of the current macroblock and the half-pel values of the next row of the search area can be correctly, e.g. synchronously, applied to PE0. The scanning of the current macroblock is done in the above manner so that the other PEs can receive correct pixel values for determining error measures, correspondingly. The other PEs have pixel values inputted correctly by the help of the delay unit array.
Delay Units for Half-Pel Motion Estimation
With the pixel scan orders for the search and the current macroblock, delay units are required to have respect delay times in order to reuse current data, i.e. full-pel values from the current macroblock. As above discussed, half-pel values A, B, C, D provided at a time are fed into PE0, PE1, PE3, and PE4, for example, synchronously with full-pel value DC(i, j), so that the PE0, PE1, PE3, and PE4 determine respective error measures correspondingly. Therefore, the HDUs 140, 160, and VDU 150 are set to have no delay time in this embodiment. With regard to the other PEs, settings are done in view of data reuse of the current data as follows.
Referring to
Referring to
Control of the PE Array for Half-Pel Motion Estimation
Consistent with the above discussions, the 9 PEs as shown in
With the definition of enabling cycles, the control of the PE array is convenient and the implementation is less complexity. For example, a PE enabling logic circuit can be implemented to determine which one of the sub-areas includes a pixel R(i, j) when a full-pel value DR(i, j) is read. For each sub-area that is determined to include the pixel R(i, j), a corresponding one of the enabling signals is enabled and applied to the corresponding control terminal of the PE associated with the sub-area (or enabling cycle), whereby the PE array is controlled.
For instance, when DR(1, 0) is read, the PE enabling logic circuit determines that PE0_enable_cycle and PE2_enable_cycle, (two sub-areas) include pixel R(1, 0). For the two sub-areas that are determined to include pixel R(1, 0), enabling signals, i.e. PE0_Enable, PE1_Enable, PE3_Enable, PE4_Enable, and PE2_Enable, PE5_Enable, are enabled and applied to the corresponding control terminals, i.e. PE0en, PE1en, PE3en, PE4en, and PE2en, PE5enof the PEs associated with the enabling cycles PE0_enable_cycle and PE2_enable_cycle.
Consistent with the fourth embodiment for performing half-pel motion estimation, TABLE 2 lists enabling conditions for the 9 PEs in
Architecture of a Motion Estimation System for Full-Pel and Half-Pel Motion Estimation
Referring to
The motion estimation unit 2100 includes a 2D PE array unit 100, a multiplexer 1150, a register unit 1160, and a minimum SAD determination unit 1170. In addition, the motion estimation unit 2100 includes a half-pel values generation unit for outputting a group of half-pel values, in parallel to the 2D PE array unit 100. h includes a preparation delay unit 2200 and a half-pel generating circuit 2300. Examples of the preparation delay unit 2200 and half-pel generating circuit 2300 are shown in
The memory reading unit 1500 is a memory reading interface for the motion estimation system 2000, wherein the memory reading interface can be implemented to be compliant with at least one communication protocol that is employed by a memory bus 10 coupled to the motion estimation system 2000.
The control unit 1600 is used to count for a step search. The control unit 1600 can be a finite state machine, for example, including two counter circuits, X counter and Y counter, to count for a step search. The X counter counts how many pixels whose pixel value is read in a row of a search area. The Y counter counts how many pixel columns whose pixel values are read in the search area. The X counter increases by one when a piece of Ref_Data, corresponding to a pixel in the search area, is read. The Y counter increases by one when X counter reaches a predetermined value, denoted by X_max_count, and then X counter is reset to 0. When Y counter reaches y_range, the step of the step search algorithm is ended. Because the motion estimation system 2000 can operate in two different stages for full-pel motion estimation and half-pel motion estimation selectively. The X and Y counters are required to reach different predetermined values for full-pel motion estimation and half-pel motion estimation.
At the first stage for full-pel motion estimation, X_max_count is the width of the search area (number of pixels), i.e. X_max_count=x_range. In step 1 to step n-1 of full-pel motion estimation, X_max_count=x_range=macroblock_size+STEP_SIZE×2. For example, in FSS algorithm, X_max_count=16+2×2=20, where STEP_SIZE=2 except for the final step. For the final step of full-pel motion estimation or half-pel motion estimation, X_max_count=x_range but the SIZE_SIZE may be changed to a reduced value. In half-pel motion estimation, X_max_count=16+1×2=18, where STEP_SIZE is 1. The memory reading unit 1500 generates a memory read signal, denoted by Ref_ready, to the control unit 1600. The memory read signal is used to inform the X and Y counters to update their count values. For example, Ref_ready is set to be enabled, e.g. a high level, when a piece of Ref_Data, corresponding a pixel of the search area, is read from a memory, e.g. the reference memory. The PE enabling cycles are determined according to the current count values X and Y from the X and Y counters, and the enabling conditions, as specified in TABLE 1 for full-pel motion estimation and TABLE 2 for half-pel motion estimation.
The address generation unit 1700 includes a PE enabling logic circuit 1750 and a motion vector (MV) generation logic circuit 1770. The PE enabling logic circuit 1750 receives the current count values X and Y from the X and Y counters of the counter unit 1600; generates enabling signals according to the current count values X, Y and either TABLE 1 for full-pel motion estimation or TABLE 2 for half-pel motion estimation; and outputs the enabling signals to the 2D PE array unit 100 of the motion estimation unit 2100. As above described, after the scanning of the search area, 9 error measures, e.g. 9 SADs corresponding to the nine checking points in the second embodiment, are obtained, and a minimum error measure is determined and outputted by the minimum SAD determination unit 1170. The address generation unit 1700 receives the minimum error measure outputted by the minimum SAD determination unit 1170. At the first stage, the MV generation logic circuit 1770 generates a motion vector in the final step of the search algorithm. If a half-pel motion estimation is to be performed at the second stage, the motion vector obtained in the first stage will be used as a basis to determine its refinement with half-pel accuracy. In addition, the address generates unit 1700 generates memory addresses to the memory reading unit 1500 so that reference data and current data are read from the memory reading unit 1500 and fed into the motion estimation unit 1100.
Operation of the Motion Estimation System During Half-Pel Motion Estimation
The operation of the motion estimation system 2000 is illustrated to perform a half-pel motion estimation. Suppose that the motion estimation system 2000 obtains a best point found in a full-pel motion estimation at the first stage and operates with a clock signal, CLK. First, the 2D PE array unit 100 is configured, as shown in
Specifically, during configuration of the 2D PE array unit 100, the HDUs and VDUs of the 2D PE array unit 100 are configured, for example, as described in the section “DELAY UNITS FOR HALF-PEL MOTION ESTIMATION”.
During full-pel motion estimation, Ref_Data of the search area is being read, sequentially, line after line. In this embodiment, when PE0_Enable indicates “enabled”, or is asserted, a piece of current data, corresponding to a pixel of the current macroblock, is read before a piece of reference data, corresponding to a pixel of the search area, is read.
In one embodiment, efficient power reduction is achieved by using gated clock technique in the HDUs and VDUs of the 2D PE array unit 100 to control the shift registers. In half-pel motion estimation, the memory read signal, Ref_ready, generated by the memory reading unit 1500 is applied in controlling the delay unit array of the 2D PE array unit 100 and the preparation delay unit 2200. For example, in a half-pel motion estimation, the HDU enabling signals, HEN_CS, are set to a logic state equal to a logic expression: HEN_CS=Ref_ready & (X_count>0) & (Y_count>0) for matching the timing of PE1_Enable. The VDU enabling signals, VEN_CS, are set a logic state equal to a logic expression: VEN_CS=HEN_CS & (X_count<17). The enabling signal for the preparation delay unit 2200, SEN_CS is set by: SEN_CS=Ref_ready. It is noted that Ref_ready is set to a high state when Ref_Data of a pixel of a search area is read from the reference memory. The HDU enabling signal is fed into the HEN terminal of the HDU, as shown in
Performance for Half-Pel Motion Estimation
The assumptions as described in the section “PERFORMANCE” for full-pel motion estimation are taken. A half-pel motion estimation requires reading (16+1×2)×(16+1×2)=324 pieces of reference data and reading 16×16=256 pieces of current data in the above embodiment. Thus, under the assumptions, the half-pel motion estimation with respect to a current macroblock takes 324+256/4=388 cycles to complete.
Advantages
In the fourth and fifth embodiments of the invention, the configuration of the 2D PE array unit for performing half-pel motion estimation and an architecture of a motion estimation system for full-pel and half-pel motion estimation are disclosed. According to the embodiments, a half-pel values generation unit with a preparation delay unit and a half-pel generating circuit is disclosed to operate with the 2D PE array unit configured to perform half-pel motion estimation. The 2D PE array unit as shown in
An optimal benefit from the parallelism and pipelining that are inherent in the 2D PE array unit in
The pixel scan orders for the search area and macroblock are regular and simple so that the control logic and enabling signals can be implemented without using complicated hardware. The 2D PE array unit is expandable and flexible in circuit design. In addition to half-pel motion estimation, the 2D PE array unit can be further adapted for any sub-pixel motion estimation, e.g. quarter-pel motion estimation, 1/8 -pel motion estimation and so on.
While the invention has been described by way of example and in terms of a preferred embodiment, it is to be understood that the invention is not limited thereto. On the contrary, it is intended to cover various modifications and similar arrangements and procedures, and the scope of the appended claims therefore should be accorded the broadest interpretation so as to encompass all such modifications and similar arrangements and procedures.
| 标题 | 发布/更新时间 | 阅读量 |
|---|---|---|
| 一种仿鸽子智能的无人机自主导航系统及方法 | 2020-05-23 | 4 |
| 基于固定积分时间的嵌入式红外图像超帧处理方法 | 2020-06-25 | 3 |
| 一种具有类框架小波结构的小波域运动估计方法 | 2021-08-06 | 5 |
| 参考数据载入的方法及装置 | 2021-08-27 | 5 |
| Reducing and correcting motion estimation artifacts during video frame rate conversion | 2020-12-10 | 1 |
| Moving image coder and control method thereof, and program | 2022-01-19 | 3 |
| Method of multi-resolution based motion estimation and recording medium storing program to implement the method | 2021-12-23 | 4 |
| Method for non exhaustive motion estimation which times out | 2023-02-09 | 3 |
| IMAGE DISPLAYING APPARATUS HAVING FRAME RATE CONVERSION AND METHOD THEREOF | 2021-10-09 | 1 |
| STANDARDS-COMPLIANT MODEL-BASED VIDEO ENCODING AND DECODING | 2020-10-25 | 3 |
高效检索全球专利专利汇是专利免费检索,专利查询,专利分析-国家发明专利查询检索分析平台，是提供专利分析，专利查询，专利检索等数据服务功能的知识产权数据服务商。
我们的产品包含105个国家的1.26亿组数据，免费查、免费专利分析。
专利汇分析报告产品可以对行业情报数据进行梳理分析，涉及维度包括行业专利基本状况分析、地域分析、技术分析、发明人分析、申请人分析、专利权人分析、失效分析、核心专利分析、法律分析、研发重点分析、企业专利处境分析、技术处境分析、专利寿命分析、企业定位分析、引证分析等超过60个分析角度，系统通过AI智能系统对图表进行解读，只需1分钟，一键生成行业专利分析报告。
