This is a continuation of application Ser. No. 09/336,522 filed Jun. 18, 1999 now U.S. Pat. No. 6,407,736.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to computer graphics architecture and processing. More particularly, it relates to scan conversion of triangle-based polygon data into pixels.

2. Description of Related Art

Introduction

Three-dimensional (3-D) computer graphics systems display images, which represent real or imaginary objects in real or imaginary settings, on a two-dimensional (2-D) monitor or other output device. As a result, the user “believes” that he is seeing 3-D objects in a 3-D world. A typical computer graphics system stores such objects in one of the many existing object file formats, using 3-D coordinates to represent spheres, vectors, curves, polygons, and other simpler component objects, along with their associated object properties, such as color, texture, intensity, transparency and/or reflectivity. Environmental data such as the number, color, location, intensity, and other properties of illumination sources, as well as atmospheric properties, are included to add richness in detail to a scene containing one or more objects.

To render such a scene from a particular viewing angle onto a 2-D screen, the “front end” of a typical computer graphics system transforms the collection of objects in the scene into a set of primitives (typically polygons, such as triangles, that are independent of scale), taking into account any movement of objects over time, as well as the scene's environmental data and the user's desired viewing angle. Triangles frequently are used as the “building blocks” for 3-D objects with complex curved surfaces, because they are simple primitive objects that effectively can “cover” or represent each surface of virtually any complex object in a tiled manner. Relatively simple images might be represented with a few, relatively large triangles, whereas more complex images might require a greater number of smaller triangles. Regardless of their size, triangles typically are represented as three 3-D (x,y,z) vertices, along with color (RGB) and texture information. Of course, given sufficient memory and computational resources, pixels could be used in lieu of triangles to represent complex images even more precisely.

Front-end processing typically still is handled in software on the host system (e.g., a PC), and does not itself require hardware acceleration for most applications. The host system provides a stream of triangles to the “back end” of the computer graphics system. The order in which the host system provides these triangles does not necessarily bear any relationship to the screen location at which such triangles might be visible.

The back end of the system is responsible for “rasterizing” this set of triangles—i.e., transforming them into the particular pixels that will be displayed on the screen. It projects these 3-D triangles onto a 2-D screen, removes “hidden surfaces” to prevent portions of triangles that are obscured by other triangles from being displayed, and generates individual pixels (to be displayed on the screen) that “fill in” the visible portions of these triangles with their associated color or texture information. Back-end processing typically is relatively time-intensive, and thus often requires hardware acceleration to maintain sufficient performance.

The performance of 3-D graphics systems typically is measured by the number of triangles per second they can process. A key problem therefore is how to architect the back-end of a computer graphics system to process a stream of 3-D triangles as quickly as possible. Ideally, the back end of a system will rasterize, within the time required for one frame to be displayed on the screen (e.g., {fraction (1/60)} of a second for a monitor with a 60 Hz refresh rate), all of the triangles generated by the system's front end. This is not, however, always possible.

For example, even a moderately complex screen object, such as a person, may be represented by a sufficiently large number of triangles to cause the back end of a typical computer graphics system to take multiple “frame times” to render that object completely. If the scene is static and the person is standing still, the back end may, for example, require 120 frames or 2 seconds to render that scene. If, however, the scene changes frequently, e.g., if that person moves across the screen, the back end would have to rasterize a greater number of triangles per second, because it would have to render, within those same 2 seconds, multiple variations of the same object—i.e., the same person in different poses and at different locations on the screen. Alternatively, to render an even more complex static image (e.g., a scene with three people together at one time) within those same few seconds would also require the back end to rasterize a greater number of triangles per second. Thus, by processing a greater number of triangles per second, a system is able to render more complex images and/or update images more frequently to reflect changes over time, even though it may not be able to render every image within a single “frame time.”

Many of today's computer graphics applications handle very complex images and/or images that change very frequently. For example, digital imaging applications often require images of near-photographic quality which are represented by a large number of relatively small triangles. A computer graphics system must process many triangles relatively quickly in order to render such images within a reasonable period of time. Computer animation and virtual reality applications, on the other hand, may not require images of such complexity; but, they may require that frames be updated very frequently to reflect, for example, the many changes in a scene that result from a slight movement of a user's virtual reality headset. In either case, the system must process a larger number of triangles per second than if the images were less complex or changed less frequently.

To obtain adequate performance and process a sufficient number of triangles per second, most current computer graphics systems employ one of two general types of back-end architectures—(1) frame buffer architectures, which operate on a frame-by-frame basis, generating and writing into a buffer the pixels of each frame of an image to be displayed on the screen, and scanning out those pixels to the screen; and (2) display list architectures, which operate on a scanline-by-scanline basis, generating in scan order (and possibly writing into a buffer) the pixels of each scanline of an image to be displayed on the screen, and scanning out those pixels to the screen.

Frame Buffer Architectures

Systems based on frame buffer architectures, like all back end systems, receive 3-D triangles from the system's front end. These systems generate pixels to fill in each triangle (or at least the visible portion of each triangle), and store those pixels in a frame buffer that contains memory locations corresponding to each pixel on the screen. Typically, the order in which these systems generate pixels and store them in the frame buffer corresponds to the order in which triangles are received from the system's front end, and not necessarily the location of such triangles on the screen.

Typical frame buffer architectures employ a double-buffered approach, particularly for animation, in which two frame buffers are utilized. While the system is scanning out to the screen the pixels from the first frame buffer (containing the current image), it simultaneously is writing into the second frame buffer the pixels generated by rasterizing each triangle (for the next image). Once the system finishes processing the triangles for this second frame buffer (even if such processing requires multiple “frame times”), the system can switch buffers (on the next vertical retrace) and begin scanning out to the screen this next image from the second frame buffer, while generating a subsequent image in the first frame buffer.

If the system's Back end cannot generate and store pixels in a frame buffer quickly enough (i.e., cannot process a sufficient number of triangles per second), then the system scans out the same image to the screen for too many “frame times” before switching buffers and displaying the next image. As a result, images are not updated frequently enough to produce the desired animation effect.

If only a single buffer is used (e.g., for rendering a complex static 3-D object in a CAD program), the system displays the image as it is being generated. In this case, if the back end processes too few triangles per second, then the system will take too long to fully render the complete image.

Although all computer graphics systems can process only a limited number of triangles per second, systems based on frame buffer architectures are further limited by the nature of their design. Because they do not necessarily generate pixels in scan order, they cannot begin scanning out to the screen a complete image until after they generate all of the pixels representing that image and store those pixels in a frame buffer. Their overall performance therefore is limited by the time required to generate every pixel necessary to fill in each triangle (or at least the visible portion of each triangle), and write each of these pixels into the frame buffer or some other temporary memory. Further exacerbating this problem are the additional memory accesses made on a per-pixel basis, e.g., to a “z buffer” that stores pixel depth information.

Although a computer graphics system must generate a pixel for each location on the screen, it is not necessarily the case that it must write every such pixel (or even every visible pixel from each triangle) into a frame buffer in order to scan out such pixels to the screen. If, for example, a scene contains a large triangle that covers much of the screen, it is wasteful to take the time to store the same pixel value in many locations of the frame buffer memory, merely because that pixel must be displayed at many pixel locations on the screen (as is illustrated below with respect to the present invention).

Moreover, in a typical scene, many triangles may be partially or completely obscured by other triangles. As a result, the system may perform many redundant computations, as well as redundant writes to the frame buffer or other temporary memory, for pixels that ultimately will not be visible on the screen. Some systems, however, implement “hidden surface removal” algorithms to avoid writing these hidden pixels into the frame buffer, which may reduce this additional performance penalty to some extent.

For a description of a typical frame buffer architecture, see Kurt Akeley, “Reality Engine Graphics,” Proceedings of SIGGRAPH '93 (Anaheim, Calif.; Aug. 1-6, 1993), published in COMPUTER GRAPHICS Proceedings, Annual Conference Series 1993, pp. 109-116. Although the Reality Engine system dedicates parallel hardware units to selected subsets of its frame buffer pixel locations, it still suffers from the above-mentioned disadvantages within each hardware unit.

The architecture of Oak Technology's 64-bit 3-D “WARP 5” graphics accelerator is a slight variation of a traditional frame buffer architecture. The WARP 5 first sorts the triangles into regions of the screen where they might generate visible pixels. Individual triangles can, of course, affect multiple regions. Upon completion of this “X-Y sort” of the entire set of triangles, the WARP 5 then rasterizes the triangles on a region-by-region basis, one region at a time, generating pixels for the current region and writing them into an on-chip “mini” frame buffer corresponding to that region of memory. It then writes the contents of each “mini” frame buffer into a single external (off-chip) frame buffer.

This process, though performed sequentially for each region, is similar in nature to the process employed by more traditional frame buffer architectures, and thus suffers from many of the same disadvantages. The WARP 5 still does not generate pixels in scan order. Although it implements a “hidden surface removal” algorithm that reduces the redundant pixel computations and writes for obscured triangles, it still generates and writes to a frame buffer (albeit a smaller, on-chip frame buffer) the many pixels necessary to fill in at least the visible portions of every desired triangle within each region before scanning out to the screen any of these pixels. Moreover, it suffers an additional performance penalty by serially (one region at a time) generating and writing pixels. This disadvantage, however, is a tradeoff for the relatively simple hardware necessary to handle only a single region at a time.

Display List Architectures

As an alternative to frame buffer architectures, display list architectures attempt to reduce the time required to generate and write every pixel (or at least every visible pixel from each triangle) into a frame buffer. Such architectures typically employ a pipeline of massively parallel processors, in which each processor is associated with an individual pixel or triangle (usually within a single scanline), to generate pixels very quickly, and in scan order. These pipelined processors enable the system to generate multiple scanlines in parallel, and thus to begin generating scanlines of a subsequent image before it has finished generating all of the scanlines of the current image, thereby reducing the average number of “frame times” required to generate a complete image.

Display list systems, although they pipeline the pixel generation process, typically cannot generate pixels sufficiently quickly to enable them to be scanned out to the screen “on the fly”—i.e., immediately as they are generated. A temporary frame buffer therefore still is necessary to buffer at least some number of generated scanlines before the process of scanning them out to the screen can begin.

For a description of a typical display list architecture, see Michael Deering, Stephanie Winner, Bic Schediwy, Chris Duffy and Neil Hunt, “The Triangle Processor and Normal Vector Shader: A VLSI System for High Performance Graphics,” COMPUTER GRAPHICS, Vol. 22, No. 4, pp. 21-30 (August 1988). This system employs a pipeline of 1024 triangle processors, each associated with a single triangle at any one time, to generate scanlines of pixels in scan order.

The Deering et al. System pre-sorts the triangles into a Y-buffer that associates each scanline with a set of those triangles which intersect that scanline, and thus potentially might include pixels visible on that scanline. Each of these triangles is then assigned to one of the triangle processors in the pipeline, and “blank” pixels (representing actual pixel locations, processed in scanline order) are sent through the pipeline. Each triangle processor determines whether the current pixel location it receives is visible within its associated triangle—i.e., whether the pixel location falls within that triangle, and whether the interpolated depth of that triangle for that pixel location is “closer” than that generated by any previous triangle processor in the pipeline. If not, it merely passes that pixel onto the next triangle processor. If it is visible (thus far in the pipeline), it replaces the pixel with one having its interpolated depth. At the end of this pipeline, the “winning” pixel is sent through a smaller pipeline to generate RGB pixels that are stored in a temporary RGB frame buffer before being scanned out to the screen.

By employing a pipeline of massively parallel processors to generate pixels quickly and in scan order, display list systems are able to reduce the average number of “frame times” required to generate a complete image. Yet, such systems typically are “unbounded” in that they cannot guarantee that every scanline will be generated within a predefined period of time, i.e., because the performance of their pixel-generation process is dependent upon the concentration of triangles within particular regions of the screen.

For example, although the system described above has a fixed number of triangle processors, the number of triangles per scanline (in the image to be rendered) is not fixed. Even though a triangle processor can be associated with a new triangle once it has finished processing the last pixel location within its current triangle, there is no guarantee that a triangle processor will be available when a new triangle is ready to be loaded. If this “overflow” condition is detected, one or more addition “passes” through the triangle processor pipeline will be necessary to handle the “overflowed triangles” for a particular scanline. Only when the system completes these additional passes can it generate the correct scanline. Thus, congestion of triangles within a particular region of the screen may impact the overall performance of the system, and effectively increase the average number of “frame times” required to generate a complete image.

Moreover, these pipelined triangle processors cannot necessarily generate pixels sufficiently quickly to enable them to be scanned out to the screen “on the fly”—i.e., immediately as they are generated. In addition, the system's circuitry is made more complex by the fact that the pipeline of triangle processors may be processing pixel locations on multiple scanlines at any given point in time, not to mention the complexity and associated performance penalty of having to detect and handle “overflow” conditions when triangles are congested within a region of the screen.

Display list architectures also have a number of other disadvantages, such as the higher cost and greater complexity of massively parallel hardware. It generally is not feasible, for example, to include a single processor for every pixel on the screen. Moreover, even if the number of processors is limited, for example, to one per pixel on a single scanline, this may result in little overall performance benefit, due to the large number of triangles that have to be processed by each pixel processor, as well as any pre-sorting of triangles by the system.

Semiconductor Industry Trends

To approach the ideal of rasterizing all of the triangles generated by the front end of a computer graphics system within a single “frame time,” the system's back end architecture must be optimized to avoid the bottlenecks while leveraging the benefits resulting from current trends in the semiconductor industry. For example, both logic and memory are increasing in density and decreasing in cost at an exponential rate. Based upon current predictions, by the year 2000, a single ASIC logic chip will contain over 100 million transistors, and mass production of 1 Gbit DRAMs will have begun, with each 1 Gbit DRAM (128 Mbytes) chip being capable of storing a 2 Mpixel image having 64 bytes of storage per pixel. Yet, ASIC pin counts are not increasing, instead remaining relatively constant at about 200-500 pins per ASIC. It is thus apparent that inter-chip bandwidth is likely to remain a significant bottleneck.

This bottleneck underscores the disadvantages noted above, particularly with respect to frame buffer architectures, which suffer performance penalties due in part to the many off-chip memory accesses that result from generating and writing many pixels to a frame buffer, and frequently accessing a “z buffer” and other temporary memory. Display list architectures also suffer from similar disadvantages, though they increase overall performance somewhat by pipelining the pixel-generation process. Yet, neither frame buffer nor display list systems can generate pixels sufficiently quickly to enable them to be scanned out to the screen “on the fly” as they are generated, which would eliminate the need for a frame buffer entirely.

SUMMARY OF THE INVENTION

The present invention provides a solution to the above-described problems by employing an architecture attuned to the current trends in the semiconductor industry. Various embodiments of this architecture are optimized to utilize one or a small number of ASICs, each containing a large number of transistors with relatively few interconnects. One embodiment of the present invention can be implemented in a single-chip ASIC which includes all the functionality necessary to perform the triangle buffer writing and rasterization/scan-out duties. Other embodiments may provide for two chips. One chip performs triangle buffering, while the other chip performs rasterization/scan-out functions.

One embodiment of this architecture is a real-time system that implements a two-step process. The first step in this process identifies which triangles are in competition to be rendered at a given pixel location, and stores them in a triangle buffer. The number of competing triangles is bounded in this first step to the “closest” N triangles associated with each pixel location to simplify the pipelined pixel generation implementation in the second step. The second step generates pixels based on the contents of the triangle buffer by resolving the competition, and renders each pixel (e.g., scans it out to the screen) “on the fly” as it is generated. Specifically, for each pixel location, this second step selects the relevant competing triangles, determines whether that pixel location is inside or outside these competing triangles, determines z depth values for each triangle, resolves the competition to identify the winning triangle, and generates the pixel color/texture associated with that winning triangle.

Triangle Buffer Write

By first storing triangle information for each triangle into a relatively few key locations in the triangle buffer, the system generally performs far fewer writes per triangle than there are potentially visible pixels within that triangle. It also defers scan conversion until after all triangles have been considered, at which point the system has sufficient information in the triangle buffer to generate each pixel in scan order, and scan that pixel out to the screen “on the fly” immediately as it is generated.

Writing the triangle information into a particular location of the triangle buffer guarantees “coverage competition” within a fixed-size region of the screen proximate to that location—i.e., it guarantees that the triangle will compete to be scan-converted at each of the pixel locations within that region. Triangle information may of course be written into multiple locations of the triangle buffer (each associated with a fixed-size region proximate to that location) to ensure sufficient “coverage competition” for at least all pixel locations at which that triangle may be visible. Thus, larger triangles may necessitate more writes to the triangle buffer than will smaller triangles.

In one embodiment, the triangle information includes 3-D coordinates and RGB color or texture information for each of three triangle vertices, as well as certain coefficients of “z-plane” and “slope” equations. This information can be used to determine, for any given pixel location on the screen, whether the triangle is “visible” at that location and, if so, at what depth in the scene.

Prior to writing this triangle information into a selected location of the triangle buffer, the system calculates a “z depth” value for the triangle at that location, using an artificial “maximum” value if the triangle is not visible at that location. The system compares the triangle's calculated z depth value to the z depth value stored at the corresponding location in a separate z buffer (e.g., to determine which of two triangles is “closer” at that pixel location). Initially, all locations in the z buffer are set to the artificial maximum value. Assuming, in one embodiment, that no objects are transparent and no anti-aliasing techniques are employed, then there will exist only one visible surface, and thus only one “winning” triangle, at any given pixel location on the screen. Whenever the system writes triangle information into a selected location of the triangle buffer, it also writes this z depth value into the corresponding location of the z buffer.

For each triangle being processed, the system determines how many fixed-size “coverage masks” are needed to sufficiently cover the triangle's bounding box. The system first attempts to store the triangle information for a triangle in the triangle buffer memory locations corresponding to the top left corner of each coverage mask. For each coverage mask, if the triangle information for an existing (previously processed) triangle already has been stored at that selected location in the triangle buffer, and is “closer” than (or at the same depth as) the current triangle, then the system attempts to store the triangle information for the current triangle at the next location within that coverage mask. Alternatively, if the current triangle wins, then its triangle information displaces the triangle information for the existing triangle, and the system attempts to relocate the triangle information for the displaced triangle to the next location within the particular original coverage mask associated with that displaced triangle.

In either case, the same process of comparing z depth values continues at each of these next selected locations until the triangle information for each “losing” triangle has been stored at a selected location within that triangle's particular associated coverage mask in the triangle buffer, or until such triangle “loses” at all such locations. In this latter case, its triangle information need not be stored anywhere within that coverage mask area of the triangle buffer because the triangle is not visible (based upon the prior z depth comparisons) at any pixel location on the screen corresponding to any of the fixed-size “coverage competition” regions associated with each location within that coverage mask area of the triangle buffer—i.e., because the triangle at each such pixel location either is outside the user's viewing angle or is obscured by a “closer” triangle.

This process of writing triangle information into selected locations of a triangle buffer requires far fewer writes, and far less time, than a frame buffer or display list system would require to generate pixels and store them in a frame buffer. This is due in part to the fact that this process is performed on a per-triangle, not a per-pixel, basis. By employing fixed-size “coverage competition” areas, the triangle information for each triangle need only be stored at one or a few selected locations in a triangle buffer, as opposed to the far greater number of frame buffer locations corresponding to the number of pixels necessary to fill in the visible portion of each triangle. Moreover, a great deal of time has been saved by deferring the process of scan-converting triangles into pixels.

Pixel Generation and Scan-out

Once the system has considered all triangles, and stored all relevant triangle information in the triangle buffer, it then generates a pixel for each pixel location on the screen, one at a time in scan order, and immediately scans each pixel out to the screen “on the fly” as it is generated. This is possible not only because the system's pixel generation process is heavily pipelined, but also because it is “bounded,” in that a fixed maximum number of triangles will compete to be visible at each pixel location on the screen. This maximum number of triangles corresponds to the number of memory locations within the fixed-size “coverage competition” region associated with each pixel location on the screen. Those triangles whose triangle information was stored within any such region in the triangle buffer are guaranteed to be the “closest” triangles at the pixel location on the screen associated with that region. The prior z depth comparisons effectively discarded other “losing” triangles having greater depths at that pixel location.

The processes of generating pixels and scanning them out to the screen are performed in parallel via a pipeline that processes the contents of the triangle buffer and generates pixels in scan order. Because this process is “bounded,” the system can guarantee that each pixel will be generated in the fixed period of time required to scan that pixel out to the screen—e.g., {fraction (1/60)} of a second, divided by the number of pixels on the screen. Thus, the system incurs no additional overhead to scan-convert triangles into pixels. Its performance (triangles per second) is limited only by the time required to process each triangle and write triangle information into the triangle buffer.

Moreover, by “bounding” this process, the hardware required to implement this pipeline is greatly simplified. Compared with massively parallel display list architectures, for example, this pipeline uses far fewer and far simpler processors. Yet, it generates pixels faster and at regular intervals, enabling each pixel to be scanned out to the screen “on the fly” as it is generated. This system also can operate in a “double-buffered” manner. In that case, it utilizes the contents of a first triangle buffer and z buffer to generate pixels and scan them out to the screen for the current frame, while simultaneously storing triangle information for the next frame into a second triangle buffer and z buffer.

In either case, the system transfers the contents of the triangle buffer in scan order into a multi-stage pipeline that includes a “triangle cache,” a column of “coefficient evaluators,” an array of “z interpolation” processors, an “image composition network,” and a “shading unit.” In effect, this pipeline implements a “sliding coverage competition window”, which slides across the triangle buffer determining the “winning” triangle for each pixel location on the screen, in scan order. At each moment in time, the z interpolation processors are calculating z depth values for all competing triangles within that “sliding coverage competition window,” and then providing them in parallel to the image composition network, which determines the “winning” triangle.

At the beginning of the pipeline, the triangle cache receives and caches the most recent “N” rows from the triangle buffer, where N is equal, in one embodiment, to the number of rows in a fixed-size “coverage competition” region (e.g., 16 rows). The triangle cache wraps around to overwrite the first row after the last row of the cache is filled.

At the next stage of the pipeline, the triangle cache provides a column of triangle information in parallel to the coefficient evaluators, each of which determines certain depth-related components for each triangle stored in that column. After providing the coefficient evaluators the rightmost column of triangle information in the cache, the triangle cache wraps around to provide the leftmost column for the next N rows from the triangle buffer. Because the pipeline generates pixels in scan order, these depth-related components are limited to the row/scanline of the triangle buffer in which the triangle information for each triangle is stored. They enable the next stage of the pipeline to calculate, for any pixel location within that row/scanline, whether the triangle encompasses that pixel location and, if so, the triangle's interpolated z depth at that pixel location.

These depth-related components include “2-D span” information, which identifies the left and right edges of the triangle intersected by that row/scanline, z depth information for the current pixel being processed on that row/scanline (or for the left edge of the triangle intersected by that row/scanline if the current pixel is not within the triangle), and “dz slope” information which indicates the slope, or change in z depth, of the triangle from left to right.

At the next stage of the pipeline, the coefficient evaluators provide a column of triangle information in parallel to a “sliding window” or array of z interpolation processors (e.g., M processors, where M is equal to the number of columns in each fixed-size “coverage competition” region, e.g., 32). Each of these z interpolation processors calculates a z depth value, at the current pixel location being processed, for one of the triangles stored within this (e.g., 32×16) sliding window of locations in the triangle buffer. As each new column of triangle information is received from the coefficient evaluators, the sliding window of z interpolation processors calculate z depth values for the next pixel location, using a set of competing triangles within the “coverage competition” region one column to the right of the previous region.

In other words, the coefficient evaluators and z interpolation processors together enable the system to calculate, for the current pixel location being processed, z depth values for all competing triangles within a “coverage competition” region—e.g., the 32×16=512 triangles stored at the locations in the triangle buffer within this region. These z depth values are calculated simultaneously by the array of z interpolation processors for the current pixel location, and provided to an “image composition network” to determine the “winning” triangle.

At the next stage of the pipeline, the array of z interpolation processors provides all of the z depth values in parallel to the image composition network, which includes a “tree” of comparators to compare the z depth values within the current “coverage competition” region, and determine the “winning” triangle that is visible at the current pixel location being processed. A “shading unit” then determines the RGB color or texture for that pixel from the triangle information stored in the triangle buffer for that triangle—e.g., by interpolating from RGB information for each vertex of the triangle.

The calculations at each stage of this pixel generation pipeline are synchronized such that the final pixel data for each pixel is provided by the last stage of the pipeline when the “video clock” actually scans that pixel out to the screen. As noted above, this is possible because these calculations are “bounded” to a relatively small fixed number of triangles. This pipeline also benefits by making efficient use of very wide on-chip “embedded DRAM” busses for parallel data transfers between stages of the pipeline, which improves performance significantly and avoids time-consuming off-chip memory accesses.

Another embodiment of the present invention uses micro-polygons instead of polygons (i.e., triangles). Of course, the front end graphics system delivers micro-polygons, which can be conceptually viewed as polygons of higher resolution. The vertices of the micro-polygons are associated with samples or sub-pixels and the micro-polygon is any grouping of a plurality of samples or sub-pixels. A buffer at the output image composition network sums the sub-pixel values per pixel, calculates an average of the sub-pixels per pixel, and associates the average to that pixel. This feature results in smoother edges and improved anti-aliasing effects. One embodiment of the present invention uses micro-polygons in a real-time graphics system.

BRIEF DESCRIPTION OF THE FIGURES

The above objects and description of the present invention may be better understood with the aid of the following text and accompanying drawings.

FIG. 1

shows a high level block diagram view of the deferred scanline converter system architecture (or graphics system) in accordance with one embodiment of the present invention.

FIG. 2

shows a graphics system implementation with a computing system, a back end graphics processor containing the deferred scanline converter, and an output device in accordance with one embodiment of the present invention.

FIG. 3

shows a flow diagram of the main triangle buffer write process in accordance with one embodiment of the present invention.

FIG. 4

shows a flow diagram of reordering each triangle data to top, middle, and bottom vertices in accordance with one embodiment of the present invention.

FIG. 5

shows a flow diagram of determining the bounding box for the triangle data.

FIG. 6

shows the structure of the triangle buffer and one of its cells in accordance with one embodiment of the present invention.

FIG. 7

shows a flow diagram of the write subroutine portion of the main triangle buffer write process in accordance with one embodiment of the present invention

FIG. 8

shows a flow diagram of the comparison subroutine portion of the main triangle buffer write process in accordance with one embodiment of the present invention.

FIG. 9

shows a flow diagram of finding a new buffer location for the “failed” or “displaced” triangle data in accordance with one embodiment of the present invention.

FIG. 10

shows one example of received triangles mapped onto an output device such as, for example, a computer monitor, to illustrate the reception order and varying sizes of the triangles received by the system.

FIGS.

11

(A)-

11

(E) illustrate the bounding box concept.

FIGS.

12

(A)-

12

(C) illustrate the coverage mask concept with respect to a triangle and its respective bounding box in accordance with one embodiment of the present invention.

FIGS.

13

(A)-

13

(C) illustrate the concept of corresponding triangle buffer locations to triangles, its bounding box, and the coverage mask.

FIGS.

14

(A)-

14

(C) illustrate the concepts of assigning specific triangle buffer locations to triangles based on coverage mask placement and size in accordance with one embodiment of the present invention. These figures also illustrate the concept of visible v. non-visible regions of the bounding box.

FIG. 15

shows a triangle with its vertices.

FIG. 16

illustrates the concept of determining bounding box boundaries.

FIG. 17

illustrates the concept of writing triangle data to specific triangle buffer locations in accordance with one embodiment of the present invention.

FIG. 18

illustrates the concept of writing “failed” new triangle data to specific triangle buffer locations in accordance with one embodiment of the present invention.

FIG. 19

illustrates the concept of writing “displaced” old triangle data to specific triangle buffer locations in accordance with one embodiment of the present invention.

FIG. 20

shows the hardware implementation of the triangle buffer write scheme in accordance with one embodiment of the present invention.

FIG. 21

shows the scan-out subsystem of the pipelined system from the triangle cache to the image composition network in accordance with one embodiment of the present invention.

FIGS. 22-25

illustrate the operation of the scan-out subsystem shown in

FIG. 21

in accordance with one embodiment of the present invention.

FIG. 26

shows a single z-interpolator processor in the array of z-depth interpolator processors in accordance with one embodiment of the present invention.

FIG. 27

shows an image composition network in accordance with one embodiment of the present invention.

FIG. 28

shows the double-buffered system implementation in accordance with one embodiment of the present invention.

FIG. 29

shows a chip-level view of the deferred scanline converter architecture in accordance with one embodiment of the present invention.

FIG. 30

shows performance improvement specifications at various key points in the chip-level diagram of FIG.

29

.

FIG. 31

illustrates the fill factor concept in graphical form.

FIG. 32

shows a graph of triangle buffer write attempts need for a successful triangle write given various capacity specifications of the triangle buffer.

FIG.

33

(A) shows a teaching example of a grid for the output monitor and its pixel locations. The grid can also represent the triangle buffer locations.

FIG.

33

(B) shows three triangles representing the background of a scene covering the output monitor of FIG.

33

(A).

FIG.

34

(A) shows a graphical depiction of a top-open box that will be rendered by the graphics system in accordance with one embodiment of the present invention.

FIG.

34

(B) shows a simplified triangle representation of the same box of FIG.

34

(A).

FIG.

34

(C) shows the same box depicted against the background by the output monitor of FIGS.

33

(A) and

33

(B).

FIG. 35

expressly shows the pixel locations superimposed on the box and background of FIG.

34

(C).

FIG.

36

(A) shows the triangle buffer contents after the first three triangles (i.e., the background) of FIG.

33

(B) have been received by the graphics system and the triangle buffer write scheme has been performed for these triangles.

FIG.

36

(B) shows the triangle buffer contents after all thirteen triangles (i.e., the entire image) of FIG.

34

(C) have been received by the graphics system and the triangle buffer write scheme has been performed for these triangles.

FIGS.

37

(A)-

37

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

0

,

0

) to (

5

,

0

), respectively.

FIGS.

38

(A)-

38

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

6

,

0

) to (

11

,

0

), respectively.

FIGS.

39

(A)-

39

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

12

,

0

) to (

17

,

0

), respectively.

FIGS.

40

(A)-

40

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

18

,

0

) to (

23

,

0

), respectively.

FIGS.

41

(A)-

41

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

0

,

1

) to (

5

,

1

), respectively.

FIGS.

42

(A)-

42

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

6

,

1

) to (

11

,

1

), respectively.

FIGS.

43

(A)-

43

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

12

,

1

) to (

17

,

1

), respectively.

FIGS.

44

(A)-

44

(B) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

18

,

1

) to (

19

,

1

), respectively.

FIGS.

45

(A)-

45

(C) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

20

,

3

) to (

22

,

3

), respectively.

FIGS.

46

(A)-

46

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

23

,

3

) to (

4

,

4

), respectively.

FIGS.

47

(A)-

47

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

5

,

4

) to (

10

,

4

), respectively.

FIGS.

48

(A)-

48

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

11

,

4

) to (

16

,

4

), respectively.

FIGS.

49

(A)-

49

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

17

,

4

) to (

22

,

4

), respectively.

FIGS.

50

(A)-

50

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

23

,

4

) to (

4

,

5

), respectively.

FIGS.

51

(A)-

51

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

5

,

5

) to (

10

,

5

), respectively.

FIGS.

52

(A)-

52

(F) show the contents of the triangle cache and the array of z-interpolator processors when the pipelined graphics system of the present invention is generating pixels for pixel locations (

11

,

5

) to (

16

,

5

), respectively.

FIG. 53

shows the winning pixels as determined by the graphics system in accordance with one embodiment of the present invention.

FIG. 54

shows the z-plane finite state machine that calculates the z values given the various coefficients x, x

0

, y, y

0

, z

0

, dz, and b

coeff

in accordance with one embodiment of the present invention.

FIG. 55

shows a coefficient evaluator in accordance with one embodiment of the present invention.

FIG. 56

shows a coefficient evaluator in accordance with another embodiment of the present invention.

FIG. 57

shows in graphical form a triangle, its vertices, its slopes, and some left and right edges on a grid to illustrate the operation of the coefficient evaluator.

FIG. 58

shows the sliding mask superimposed on an exemplary triangle buffer to teach the basic concept of the sliding mask.

FIG. 59

shows the sliding mask superimposed on an exemplary triangle buffer to show the operation of the sliding mask in accordance with one embodiment of the present invention.

FIGS.

60

(A) and

60

(B) illustrate the sliding mask superimposed on an exemplary triangle buffer to illustrate why the embodiments of the present invention utilize the out of phase loading (or delayed pixel generation) scheme in accordance with one embodiment of the present invention.

FIG. 61

illustrates the region where a given triangle is guaranteed coverage competition.

FIG. 62

illustrates micro-polygons and sub-pixels (samples).

FIG. 63

shows the single line buffer used to temporarily accumulate and store the sub-pixel values for each pixel location so that they can be processed for presentation to the display screen.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The many embodiments of the present invention provide a system and method of rendering a description of objects, such as triangle data, to some output device, such as a printer or display monitor; that is, 3-D triangles are converted to pixels. These embodiments include a scanline converter architecture and method for processing multi-dimensional computer graphics data and displaying processed pixel data with a monitor or printer in real-time without the use of a full-framed pixel buffer. A two-stage process enables the rendering function by receiving the triangle data, determining those triangle data that are in competition for a given pixel location with a bounded writing scheme, and then, during scan-out to the output device, determining the winning triangle data for a given pixel location from among those triangles that are in competition. The system then applies shading/texture mapping to the winning triangles to generate pixels. These embodiments work well with current chips for optimum use of on-chip and off-chip bandwidth.

This patent specification will discuss the many embodiments of the present invention. First, some definitions of general terms and phrases will be provided. Second, an overview of the general architecture of the present invention will be discussed with specific details to be presented later. Third, the triangle buffer writing scheme in accordance with one embodiment of the present invention will be discussed, including a description of the coverage mask, bounding box, and the visible v. non-visible regions of the bounding box. Fourth, the scan-out logic including the triangle cache, the column of coefficient evaluators, the array of z-interpolator processors, and the image composition network will be discussed in detail. Alternative scan-out logic embodiments will also be presented. Fifth, the patent specification will then discuss a chip level implementation of one embodiment of the present invention. Sixth, the patent specification will illustrate the operation of one embodiment of the present invention with an example that guides the reader from the desired scene/object to the reception of the triangle data and the scan-out of the winning triangles for pixel generation to display the scene and the object. Seventh, the patent specification will conclude with some practical industrial applications that are themselves further embodiments of the present invention.

Definitions

Some definitions of terms and phrases used in this specification are in order. In the specification, the embodiments of the present invention are called “graphics system,” “system,” “deferred scanline converter,” “deferred scan converter,” or “back-end processor.” These terms and phrases generally refer to the set of embodiments of the present invention. The scope of these terms is dependent on the context.

The specification uses the terms “line,” “wire,” “wire line,” “wire/bus line,” and “bus.” These terms refer to various electrically conducting lines. Each line may be a single wire between two points or several wires between points. These terms are interchangeable in that a “wire” may comprise one or more conducting lines and a “bus” may also comprise one or more conducting lines.

The word “user” refers to the user of the graphics program who is observing the objects on the computer display screen. The “user” can select any “user viewing angle” or “viewing angle” from any three-dimensional location to view the object(s) on the display screen. Thus, if the object on the display screen is a typical open football stadium, one viewing angle may show the stadium from directly above where the stands in their entirety and the football field can be seen. Looking at the same stadium from ground level from the outside of the stadium, the stands and the football field are no longer visible but the sides of the stadium where the entrances are located can be seen. Other viewing angles will show other features of the stadium while not showing obscured or hidden features. Thus, depending on the viewing angle, certain features may or may not be hidden by obstructions. Of course, certain features may be partially visible and partially obstructed.

“Frame period,” “vertical synchronization pulse period,” and “screen refresh period” all generally refer to the time period during which data in the scan-out triangle buffer is used for scan-out purposes. This period is also the same period where the graphics system of the present invention writes triangle data into the triangle buffer that is not used for scan-out in a double buffered design. In one embodiment, the frame period may range from 30 Hz to 70 Hz with a typical value of 60 Hz. In the prior art, each frame period coincided with the scan-out of pixels in the frame buffer. In the embodiments of the present invention, the frame period coincides with the time between each screen refresh to enable the scan-out of triangle data in the triangle buffer.

Although the discussion below will generally be directed to triangles, it is equally applicable to micro-polygons. As known to those skilled in the art, micro-polygons are a collection or grouping of sub-pixels or samples. In one embodiment, a pixel includes sixteen sub-pixels or samples in a 4×4 matrix. Micro-polygons will be discussed in greater detail later in this patent specification.

General Architecture

Generally, one embodiment of the present invention is a graphics system that converts the three-dimensional polygons (e.g., triangles, micro-polygons) received from the front end of the system to pixels so that an image or series of images represented by these polygons can be rendered to some output device, such as a printer or monitor. This system is capable of generating 16 billion pixels per second, or 384 Gbytes per second, at the point in the architecture where the z values are computed and compared, which by itself represents a significant performance improvement over known prior art systems. It also has substantial expansion capabilities for fitting larger chips and newer processes such as embedded DRAM.

The particular designs substantially eliminate the many bottlenecks that had existed in prior art systems when low memory-logic interface bandwidth diminished and negatively offset the capabilities of high on-chip bandwidth. Accordingly, the embodiments of the present invention can generate an exemplary computer graphics scene of medium complexity with under 4,000 writes to memory whereas prior art systems may require approximately 300,000 pixel writes to the frame buffer for the same scene. For more complex scenes, the system needs more memory writes but still represent several orders of magnitude improvement in performance over the prior art. For simpler scenes, the system of the present invention represents an even greater order of magnitude improvement over the prior art. Such reduction in memory accesses by the system of the present invention reduces the need for off-chip bandwidth, increases throughput, and allows the system to process more triangles than ever before to increase rendering accuracy.

Other embodiments of the present invention include the many processes of converting these triangle data to pixels. Instead of the traditional frame buffer, the system uses a triangle buffer for storing triangles and defers scan conversion into pixels until scan-out. Thus, no pixels are generated and stored in memory; rather, pixels are generated “on the fly” from data in the triangle buffer during scan-out. In other words, one embodiment of the present invention can generate pixels in pipelined fashioned for immediate output to an output device (e.g., monitor) as the output device needs them during a frame period without any buffering between the pixel generation logic and the output device.

At a high level, the system utilizes a particular writing scheme to the triangle buffer which limits the number of triangles in competition to the size of a coverage mask. During pixel generation scan-out, one embodiment of the present invention uses a variation of the same coverage mask to identify competing triangles, determine visibility of triangles at particular pixel locations, determine z values, and resolve the z depth competition among the competing triangles so that a pixel from the winning triangle can be generated to the output device without any pixel writes to memory.

Referring to FIG. A, a high level view of the graphics system in accordance with one embodiment of the present invention is shown. A user decides to depict a scene and/or object(s) with computer-generated graphics. The user programs the appropriate data with a computer graphics software package, whether commercially purchased or custom-built, and provides the data to a geometry processor

15

. The geometry processor

15

performs coordinate transformations and provides the data to a triangle buffer logic

20

via line

50

. The data is in the form of triangles which may be of varying sizes and shapes, depending on the complexity of the scene or object(s) depicted. Ordering of the triangle data sent by the geometry processor

15

is not assumed; that is, the triangles may be sent by the geometry processor in random order or a specific order. The system of the present invention also does not assume any particular order of the triangles. Triangle data in the form of three-dimensional (3D) triangle data with z-plane equations are written by the triangle buffer logic

20

to a triangle buffer in accordance with a triangle buffer writing scheme. In one embodiment, the triangle buffer is implemented in a double buffered manner in which one of the buffers is used for writing data while the other is used for scan-out and the roles reverse after every vertical synchronization pulse (i.e., every 30-70 Hz, typically 60 Hz).

During scan-out, rasterization unit

30

receives the triangle data via line

51

. The rasterization unit identifies triangles that are in competition for a given pixel location, determines visibility of the triangles at that pixel location, interpolates z values for the visible triangles, compares z values of the competing triangles, and resolves the competition by selecting a winning triangle for the given pixel location. The winning triangle represents the closest triangle to the user that is not obscured by any other triangle at that pixel location. The winning triangle is provided to shading/texture mapping unit

40

via line

52

where pixels are then generated to an output system. At one level, the rasterization unit

30

receives the data in the form of 3D triangles and z-plane equations at its input on line

51

and converts them to two-dimensional (2D) spans, z, and dz information and then finally to one-dimensional (1D) pixels after shading/texture mapping. To provide a fast and efficient system, the system depicted in

FIG. 1

is pipelined.

FIG. 2

shows another high level overview of one embodiment of the present invention in the context of a computing environment. The computer graphics environment includes a host computer

60

(which includes a front end graphics processor

64

), back end graphics processor

70

, and an output system or device

80

, coupled together via several electrically conductive bus systems

90

-

92

.

Host computer

60

includes a host microprocessor

62

, a memory

61

, input/output (I/O) devices

63

, and a front end graphics processor

64

(which may be software or hardware). The host microprocessor

62

communicates with memory

61

and I/O devices

63

via bus

65

and bus

66

, respectively. The host microprocessor

62

can be any relatively fast microprocessor such as the series of Intel Pentium processors, Digital's Alpha processors, Sun MicroSystem's SPARC processors or any number of processors known to those skilled in the art that is fast enough to process digital graphics data sufficiently.

Memory

61

includes main random access memory (DRAM or SRAM), read-only memory (ROM), and possibly one of several types of on-chip or off-chip cache subsystems. Memory

61

can also include a database of image data.

The processor-memory interface may include typical connections involving a local bus and a bus controller where memory accesses occur through the local bus like some of the processors manufactured by Intel. In another embodiment, the processor communicates with memory via dedicated lines. For example, the host computer system

60

can be a Sun Microsystems Enterprise 450 system which employs UltraSPARC II processors. Instead of the memory access via the local bus, the Sun 450 system allows the multiprocessors to access the memory via dedicated buses to the memory through a crossbar switch. Thus, multiple processes can be running with multiple microprocessors executing their respective instructions and accessing the memory without going through the local bus. The Sun 450 system along with the Sun UltraSPARC multiprocessor specifications are incorporated herein by reference. The Sun Ultra 60 system is another example of a microprocessor system although it allows only two processors. In still a further embodiment, the host computer system

60

may be a massively parallel processor system.

I/O devices

63

include keyboards, mouse, display monitors, printers, disk drives, tape drives, sensors, controllers, actuators, line drivers, modems, and any other devices that need to communicate with the host computer for any number of reason. Each of these elements in the host computer

60

, including the host computer

60

itself, is commercially available and those skilled in the art knows which specific devices and brands to acquire for his/her specific application.

Computer graphics data is provided from the host computer

60

to the front end graphics processor

64

via bus

67

. After processing the data, the front end graphics processor

64

provides the processed data to the back end graphics processor

70

via bus

90

. In one embodiment, the processed data is a stream of polygons (i.e., triangles) output from the front end graphics processor at a rate of I triangle per clock on bus

90

.

Computer graphics hardware (with or without related software) are designed for either front end or back end processing. The computer graphics industry realizes that a fine line exists between front end and back end processors that resists such simple categorization. For the sake of describing the present invention, however, front end processor

64

is provided herein as an element in the computing environment of

FIG. 2

that provides triangle data and which may reside in the host computing system

60

or separately between the host computer

60

and the back end processor

70

.

Front end graphics processor

64

receives instructions and data from the host computer

60

related to the graphics construct or image in world coordinates. The graphics data are described generally by image primitives which include triangles, textures, lines, etc. As known to those skilled in the art, the world coordinate system is a system in which a scene or object is represented in the computer after being model-transformed from three-dimensional modeling object coordinates. Typically, world coordinates are in floating point and, depending on the graphics program, the world coordinates can be in any unit meaningful to the application such as meters, miles, and angstroms. World coordinates are used near the beginning of the 3-D world to 2-D display screen coordinates transformation. The front end processor

64

then performs a number of transformations, clipping, and lighting instructions to describe the graphics image in screen coordinates.

Back end processor

70

, or deferred scanline converter

70

, then receives these processed triangle data from the front end processor

64

and, through various buffering techniques of updating the current graphics data information with new graphics data information, presents the data in the form of pixels to the output device

80

. In essence, the back end processor

70

performs the following functions: (1) receives incoming triangle data; (2) determines the bounding box for each triangle; (3) uses a coverage mask to provide coverage to the triangles; (4) issues as many triangles as there are coverage masks that are necessary to “cover” the bounding box without overlaps among the coverage masks; (5) determines z-plane equations; (6) writes triangle data to a triangle buffer in accordance with a triangle buffer writing scheme; (7) identifies those triangles that are in competition for a given pixel location; (8) determines the visibility of the triangles for a given pixel location; (9) determines z depth at select locations corresponding to pixel locations by interpolation; (10) compares z depth values of all competing triangles for a given pixel location; and (11) determines the winning triangle among the competing triangles in which the winning triangle represents the triangle (and hence object or portion of an object) that is at the front of all other triangles (and all other objects or portions of objects), assuming no transparencies are involved. Based on the winning triangle, the system generates a pixel from the winning triangle at the given pixel location for delivery to the output system or device

80

.

Still referring to

FIG. 2

, the back end graphics processor

70

includes a triangle buffer logic

71

and a scan-out logic

72

coupled to the triangle buffer logic

71

via bus

91

in accordance with one embodiment of the present invention. The triangle buffer logic

71

generally provides the logic and memory structure (triangle buffer and z buffer) for allocating a fixed amount of storage for each triangle issued by the triangle buffer write logic that can be successfully written to the triangle buffer within the confines of the coverage mask. As explained below, a single triangle received from the front end graphics processor may correspond to a number of issued triangles because of the numerous coverage masks required for full coverage of the received triangle. The triangle buffer in the triangle buffer logic

71

stores triangles that may ultimately win and lose in the scan-out logic

72

down the pipeline for the various pixel locations.

Although each memory location where a triangle may be stored corresponds to a pixel screen space location at the output device, the scan-out logic will not necessarily select the particular triangle stored at triangle buffer location (x, y) as the winning triangle for display at pixel location (x, y). Conversely, just because no triangle is stored at memory location (x, y) does not mean that no triangle will win for this pixel location (x, y). This is because the system of the present invention ensures that a triangle stored at memory location (x, y) will be guaranteed coverage competition for a wide range of pixel locations within the vicinity of pixel location (x, y). This range is a fixed range generally to the right, down, and to the lower left of the location where the triangle is stored. Refer to the coverage mask and sliding mask discussions below for a full explanation. Thus, even if a triangle is stored at memory location (x, y), this triangle may win for coverage at location (x+4, y) instead. By being stored at memory location (x, y), the system guarantees that that triangle can compete against other triangles for coverage at a wide range of pixel locations in addition to pixel location (x, y).

The competition among triangles for the pixel location is necessary because some triangles may be behind, and hence obscured by, other triangles. Z depth comparisons resolve the competition by determining which triangle is the closest to the user. The triangle buffer stores a bounded number of these competing triangles using a coverage mask which limits or bounds the number of triangles that may be competing for a given pixel location. Later in the pipeline, the scan-out logic resolves the competition by determining which of the competing triangles are in front of other triangles to generate one pixel per pixel location. In other instances, no triangles obscure other triangles or only a single triangle is present, in which case the result of the competition is clear.

The triangle buffer is implemented in a double buffered scheme so that one triangle buffer can be used for writing triangle data from the front end graphics processor to the triangle buffer while the other triangle buffer can be used for scan-out purposes to the output device

80

. After every vertical synch pulse, the roles of the two triangle buffers reverse so that the triangle buffer used for scan-out during the previous synch pulse period is now used for storing buffered data and the other triangle buffer which was used for buffering data during the previous synch pulse period is used for scan-out.

The scan-out logic

72

is next. Triangle data processed by the triangle buffer logic

71

and stored in the triangle buffer are provided to scan-out logic

72

via bus

91

. In essence, the scan-out logic

72

includes a triangle cache, a column of coefficient evaluators, an array of z-interpolator processors, and an image composition network which all function to receive 3D triangle data and z-plane equations (for both z depth and color), convert them to 2D spans, z, and dz, and then finally convert them to 1D pixels. In one embodiment, the triangle buffer logic

71

outputs 1 triangle per clock on bus

91

. The scan-out logic

72

outputs 1 pixel per clock on bus

92

. In other embodiments, a triangle cache is not used; rather, appropriate memory address logic is used to process selected data from the triangle buffer to the column of coefficient evaluators.

The scan-out logic

72

uses these subsystem components to enable a “sliding mask” concept, in which a mask of dimensions n×m, where n and m are positive integers and which coincide with twice the coverage mask dimensions in one embodiment, slides across the data stored in the triangle buffer. The sliding mask slides across the triangle buffer from column by column and row by row so that eventually, the sliding mask makes its way from the top left comer of the buffer to the bottom right corner of the buffer one scanline at a time. At each “stop,” the scan-out logic processes the relevant data located within the confines of the sliding window to determine the winning triangle for pixel generation. After the end of a row, the sliding mask wraps around to the next row so that no part of the sliding mask is “hanging” outside the triangle buffer boundary; that is, all parts of the sliding mask are covering the triangle buffer. Of course, the exception is when the sliding mask is at the top edge of the triangle buffer. For example, assume the sliding mask's dimensions are 4 rows by 8 columns. If the bottom row of the sliding mask is on any part of the first three rows of the triangle buffer, some portion of the sliding mask will be “hanging” outside the triangle buffer.

The scan-out logic may include a shader/texture mapper. Alternatively, the shader/texture mapper may be provided separately from the scan-out logic. For shading, the color coordinates R, G, B will suffice. For texture mapping, U, V, and W instead of RGB are stored with the triangle data. The Gouraud shader can output interpolated values for U, V, and W which are fed to two dividers. One divider will divide U by W and the other divider will divide V by W. The outputs of these two dividers are coupled to a texture look-up table to fetch texels. The texels are then fed to a texture tri-lerp unit so that the tri-lerp operation can be performed for the texels. Textures that are not in the texture memory must be drawn into the triangle buffer one pixel at a time.

The output system or device

80

, such as a graphics monitor or printer, receives pixel data from the back-end processor

70

via bus

92

so that the proper pixel associated with a portion of an object surface can be displayed. The particular winning triangle competing for the particular pixel location was determined in the back-end processor

70

. The output device

80

merely displays the winning pixel associated with the object that is not obscured by another object for the particular pixel position. One embodiment of the monitor is a raster scan CRT display device where each horizontal line is scanned on the display sequentially, setting pixel properties (e.g., color, intensity, reflectivity) so that the combination of pixels forms a computer graphics scene.

In sum, the graphics system of the present invention, as shown and described above with respect to

FIGS. 1 and 2

, includes a triangle buffer and a scan-out logic. The graphics system receives triangle data from the front end processor and then writes the processed triangle data to the triangle buffer following a triangle buffer writing scheme in accordance with one embodiment of the present invention. The triangle buffer stores triangle data that may or may not ultimately win for a pixel location, but so long as storage space is available within the defined coverage mask and the triangle data satisfies the requirements of the triangle buffer writing scheme, the triangle data are stored in the triangle buffer for later processing by the'scan-out logic. In one embodiment, two triangle buffers are provided so that one buffer can be written with new incoming triangle data from the front end processor while the other buffer can be used for scan-out and during the next synch pulse period, the roles of the triangle buffers are reversed. For scan-out, the triangle data located within the sliding mask in the triangle buffer are processed. For each pixel location, the triangles that are competing for the given pixel location as determined by the sliding mask are evaluated and compared to each other so that a single triangle wins the competition for representation as the pixel for that given pixel location. These pixels are generated straight out of the triangle buffer and no other buffering mechanism, such as the traditional frame buffer, is utilized. Pixels are generated and provided to the output device without additional buffering.

The basic guiding principle behind this general architecture is that only one opaque triangle is visible at any one pixel location. Thus, if an image has one million pixels, only one million opaque triangles are visible. Preferably, the triangle buffer stores the visible triangle at the location where it is visible. Also, the triangle buffer need not be completely full to render the entire image. Thus, an empty triangle buffer location (x, y) can be associated with a visible triangle and pixel at pixel location (x, y). Furthermore, the system stores triangles in the triangle buffer based on the screen space location of the bounding box. A coverage mask bounds the area or region of the triangle buffer in the vicinity of the triangle data where the system guarantees coverage competition for that triangle. Indeed, as will be discussed later, the sliding mask determines the specific region where the system guarantees coverage competition for each stored triangle in the triangle buffer. By bounding the number of triangles in the coverage mask, the size of the array of z-interpolator processors in the scan-out logic can be bounded to simplify the design. Moreover, for larger triangles (triangles that are larger than the coverage mask), the system provides for multiple entries in the triangle buffer.

Triangle Buffer

The triangle buffer logic receives triangles from the front end processor and implements its triangle buffer write scheme for writing triangle data to a triangle buffer. The triangle buffer is unlike a frame buffer in many respects. Most notably, the triangle buffer stores triangle data whereas a frame buffer stores pixel data.

When the graphics system receives a triangle from the front end processor, it is actually receiving each of the three vertices of that triangle, along with other data that is relevant to the rendering of that triangle as a pixel (e.g., R, G, B). These vertices and other data associated with a given triangle are stored in a triangle buffer, if the system determines through its triangle buffer write scheme that this triangle should be written in a selected triangle buffer location. As will be explained further below, the triangle logic may decide that a particular issued triangle can not be stored anywhere in the triangle buffer because it failed its z comparisons at every triangle buffer location within its coverage mask.

Referring to

FIG. 6

briefly, one embodiment of the triangle buffer is a 640×480 memory array of cells; that is, 640 cells are provided in each row and 480 rows are provided altogether. An example of a 640×480 buffer is a buffer for a VGA system. In other embodiments, the triangle buffer has dimensions of 1280×1024. The specific dimensions may vary depending on the application and user's needs, however, the triangle buffer has dimensions that are consistent with the portion of the output device that displays or outputs pixels. Thus, if a computer graphics monitor can provide for a 1280×1024 screen to display pixels, then the triangle buffer also has dimensions of 1280×1024. Some exemplary cells include cells

147

,

148

, and

149

. In one embodiment, each cell stores 32 bytes of triangle information. A double buffered 640×480 triangle buffer requires a little over 16 Mbytes of memory.

Each triangle is described by vertex coordinates (x, y, z) in 3-D space and some specification of surface properties such as color, texture, intensity, transparency, and reflectivity. Some triangle descriptions also include the normal vectors to the surface at each surface at each vertex. In one embodiment of the present invention, the triangle data stored in each cell of the triangle buffer includes, at a minimum, the xyz spatial coordinates (where z represents depth), and the R, G, and B color coordinates (or texture information) of the three vertices. Additional values as mentioned above may be specified for a given implementation to provide input to more elaborate shading/texture mapping processes.

For example, in

FIG. 3

, cell

149

contains the triangle data having three vertices, where each vertex of the triangle has xyz coordinates, RGB data, and texture data. Based on the vertices, the system can determine the various attributes of the triangle including its color, texture, plane equation, orientation, location on the screen, shape, and size. Consistent with the format of the triangle data, the graphics system stores the three vertices of a triangle in cell

149

. The top vertex has x, y, z, rgb (or alternatively, texture) information associated with the top vertex of the triangle. The middle vertex has x, y, z, rgb (or alternatively, texture) information associated with the middle vertex of the same triangle. The bottom vertex has x, y, z, rgb (or alternatively, texture) information associated with the bottom vertex of the same triangle.

In one embodiment, the 640×480 triangle buffer delivers 32 bytes at 25 MHz, or 32 bytes every 40 ns. Thus, the scan out requirement is 800 Mbytes per second. A chip containing one next generation Rambus interface which utilizes split transaction memory access will be sufficient for scan in and scan out bandwidth.

The triangle buffer of the present invention replaces the traditional frame buffer in more ways than one. The triangle buffer is unlike the frame buffer in its contents and its placement on the pipeline. Typically, conventional frame buffers store information relating to the RGB value of the pixels to be displayed. Using the deferred scan conversion architecture of the present invention, the frame buffer concept has been modified substantially so that each triangle buffer location is capable of storing a complete triangle description, instead of its scanline-converted pixel value. Thus, although a frame buffer stores pixels, the triangle buffer stores triangles. Also, the triangle buffer is one of the first components on the pipeline placed at a point long before any pixels are generated. On the other hand, a frame buffer is normally placed near the end of the pipeline after pixels have been generated. Thus, data comprising triangle descriptions are stored in the cells.

Furthermore, the triangle buffer need not be full for the system to generate pixels for all 1280×1024 pixel location. In accordance with one embodiment of the present invention, a single triangle buffer entry allows that triangle in that entry to compete for coverage in a wide range of pixel locations. Also, just because a particular triangle is stored at a triangle buffer location (x, y) does not necessarily imply that that triangle will be the winning triangle at pixel location (x, y). Finally, just because a particular triangle buffer location is empty does not mean that no pixel will be generated at the corresponding pixel location. One embodiment of the architecture ensures that if a triangle is designated for a particular pixel location on the computer graphics monitor, that triangle will be stored in the triangle buffer at a location somewhere in the vicinity of the corresponding screen space location and available for pixel generation competition.

As explained later in this patent specification in the discussion about the scan-out logic, the triangles stored in the triangle buffer are read out and converted to pixels “on the fly” during every refresh frame period. The “on the fly” generation of pixels is a unique performance enhancing feature of one embodiment of the present invention because it enables scan-out without a frame buffer and eliminates the numerous pixel memory accesses that provided a bottleneck problem due to the low memory-logic interface bandwidth. In sum, the graphics system of the present invention does not utilize a frame buffer at all anywhere along its pipeline and instead, generates pixels based on triangle data in the triangle buffer “on the fly.”

Triangle Buffer Writing Scheme—General Overview

In order to properly generate pixels according to the embodiments of the present invention, the triangles and their respective contents must be placed in the correct memory location in the triangle buffer. The placement of a triangle in a single triangle buffer location ensures that the triangle is guaranteed coverage competition in a range of pixel locations; that is, the triangle need not be placed at every single pixel location where it will be represented as a pixel. The triangle buffer writing scheme in accordance with one embodiment of the present invention ensures such proper placement of the triangles. In other words, the graphics system writes each triangle data to the proper location(s) in the triangle buffer and the contents of that triangle data are as required for proper operation by later processes down the pipeline.

FIG. 3

shows a flow chart of the main triangle buffer writing scheme in accordance with one,embodiment of the present invention. The flow chart begins at “start” step

100

. The flow chart has some preliminary pre-writing operations of properly formatting the received triangle data as shown in steps

110

,

120

,

130

,

140

,

150

,

160

, and an actual writing operation at step

170

. A more detailed discussion of the portion of the triangle buffer write scheme associated with step

170

will be provided later in the context of

FIGS. 7

,

8

,

9

,

12

,

13

,

14

,

15

,

17

,

18

,

19

, and

20

.

Generally, the graphics system receives triangles at step

110

. It then performs some triangle data culling at step

120

to determine if the triangle is smaller than a pixel. If it is larger than a pixel, the system then re-orders the vertices of the triangle at step

130

to its proper top, middle, and bottom locations for storage in the triangle buffer cell. At step

140

, it determines the a bounding box for each triangle so that the coverage mask(s) can be appropriately applied to the triangle later for writing triangle data to the triangle buffer. At step

150

, the system issues a triangle for each coverage mask. Specifically, the system issues a triangle for each target triangle buffer location associated with a coverage mask. At step

160

, the system determines the z-plane and color plane equations so that the system can later use the slope information to obtain an accurate z-depth and color information at a specific point on the triangle. The system also uses the plane equations to generate span information so that the system can determine whether a particular pixel location for which the various scan-out operations are being performed is inside or outside the relevant triangle.

Based on the results of all of these preliminary steps, the actual writing operation to the triangle buffer can be performed at step

170

. For each write operation of a triangle to a triangle buffer location, a coverage mask is used. If a triangle is larger than the coverage mask, multiple coverage masks will be used. Applying the coverage mask(s) to the bounding box of each triangle, the system attempts to write to a designated triangle buffer location within the confines of the coverage mask via series of examining the designated location to determine if it's empty or full, performing z comparisons of new triangle data with existing triangle data, displacing the existing triangle if it loses the comparison, finding a new location for the displaced triangle, finding a new location for the new triangle if it loses the comparison with the existing triangle, and of course, writing the triangle data to selected triangle buffer locations. These operations will be discussed at length below.

The various steps of the flow chart of

FIG. 3

will now be discussed with additional details. Consistent with the system design, the system does not assume any type of ordering of the incoming triangles. Thus, at step

110

, the system receives triangles in any order. However, the system does not wait for all the triangles to be received within a given frame period before processing them; rather, the system processes the triangles as they are received to improve throughput. In other words, the system can receive one triangle while processing a previously received triangle.

To illustrate the triangle ordering, briefly refer to

FIG. 10

where screen

400

represents any screen or monitor that is capable of displaying computer graphics scenes. In some embodiments, the dimensions of the screen

400

are consistent with the dimensions of the triangle buffer; that is, if screen

400

has dimensions 1280×1024 pixels, the triangle buffer has dimensions 1280×1024. During any vertical synchronization period or frame period for this example, triangles

401

to

409

(and other triangles that are not numbered) are received by the system for processing. This frame period coincides with the time between screen refresh.

However, reference to

FIG. 10

in the context of triangle reception does not necessarily mean that the triangles

401

to

409

are stored in the triangle buffer as shown in FIG.

10

. The reader should be mindful that storage of triangles into the triangle buffer is accomplished according to a triangle buffer writing scheme specified herein in the discussion with respect to other figures.

FIG. 10

by itself is not indicative of the triangle buffer writing scheme at all.

The triangles do not necessarily come into the triangle buffer in any particular order. The triangles may come in the order

401

to

409

or some other order. Just because triangle

403

is located above triangle

402

does not necessarily mean that triangle

403

comes into the triangle buffer before triangle

402

. Similarly, just because triangle

401

is located to the left of triangle

404

does not necessarily mean that triangle

401

comes into the triangle buffer before triangle

404

. Furthermore, just because triangle

407

is located farther away from the user than triangle

408

does not necessarily mean that triangle

407

comes into the triangle buffer before triangle

408

. The triangle buffer writing scheme addresses the triangles as they are received without any sorting.

Returning to

FIG. 3

, after a triangle is received at step

110

, the system performs triangle data culling at step

120

. Here, the system compares the surface area of the triangle to that of a pixel. If the triangle data has a surface area that is less than a pixel's surface area, the triangle is discarded. Unnecessarily small triangles are not processed further.

At step

130

, the system re-orders each triangle's orientation to top, middle, and bottom vertices. As explained above, a triangle in a computer graphics system is represented by its three vertices. However, the front end processor delivers the three vertices without any regard to the triangle's orientation. The graphics system of the present invention re-orders the three vertices for proper storage in the triangle buffer cell according to its top, middle, and bottom vertices. The re-ordering step will be discussed further with respect to

FIGS. 4

,

6

, and

11

below.

At step

140

, the graphics system determines the bounding box for each triangle received. The bounding box for one triangle can vary considerably in size and shape from that of another triangle. The bounding box for a given triangle is the smallest box that can be “drawn” around that triangle. If the triangle is relatively large, its bounding box can be larger than a single coverage mask. Accordingly, multiple coverage masks will be needed to “cover” this bounding box. If the triangle is relatively small, its bounding box can be equal to or smaller than a single coverage mask. In this case, only one coverage mask will be needed to “cover” the bounding box. The bounding box is used later with the coverage mask during the actual writing routine. The bounding box determination is discussed further below with respect to

FIGS. 5 and 11

.

At step

150

, the system issues a triangle for each coverage mask needed to “cover” a bounding box. Thus, as discussed above with respect to step

140

, the size of the bounding box determines whether multiple coverage masks will be needed. If multiple coverage masks are needed, the system issues multiple triangles—one for each coverage mask. However, the triangle buffer write scheme as discussed below will ultimately determine whether the system will successfully write any or all of these triangles to the triangle buffer.

At step

160

, the graphics system determines the plane equations for z and color. The system accomplishes this task by using the coordinate information (x, y, z) to determine the z-plane equations and the color information (R, G, B) to determine the color plane equations. The z-plane equation, as known to those ordinarily skilled in the art, allows the system to determine the slope and the actual z depth of selected points on the triangle. The color plane equation, as known to those ordinarily skilled in the art, allows the system to determine the slope and the actual color of selected points on the triangle. In accordance with the present invention, the system determines the z-plane and color plane equations so that the system can also use the slope information to obtain an accurate z-depth and color information at various points on the triangle. The system also uses the plane equations to generate span information so that the system can determine whether a particular pixel location for which the various scan-out operations are being performed is inside or outside the relevant triangle.

At step

170

, the system performs the actual writing operation to the triangle buffer. As will be discussed in great detail later with respect to

FIGS. 7

,

8

,

9

,

12

,

13

,

14

,

15

,

17

,

18

,

19

, and

20

, the writing operation essentially seeks a designated triangle buffer location for a newly received and formatted triangle within the confines of its coverage mask and then compares the new triangle with the triangle already stored therein (if a triangle is already stored therein). Depending on the result of the comparison, the system assigns the winning triangle to that designated location and seeks a new location for the losing triangle within the confines of the coverage mask. If a location cannot be found for the triangle within the confines of the coverage mask, the triangle is discarded. The flow chart ends at step

180

.

As needed, the various operations generally described above with respect to

FIG. 3

will be described in greater detail below. These operations include re-ordering triangle vertices (step

130

), determining the bounding box (step

140

), issuing a triangle for each coverage mask (step

150

), generating plane equations (step

160

), and finally the write operation (step

170

).

Re-ordering of Triangle Vertices

In

FIG. 3

, step

130

requires the re-ordering of triangle vertices. Why does the system re-order the vertices to their top, middle, and bottom vertices? The re-ordering of the vertices allows the system to more effectively implement the scan-out logic portion of the invention. As will be discussed later, the scan-out logic determines and uses the slopes to each of the line segments forming the three sides of the triangle. With the slope information, the system can determine the span of a triangle at a given row. With the span information, the system can determine whether the current pixel location for which the system is generating a pixel is inside or outside the triangle. By re-ordering the vertices to the proper top, middle, and bottom vertices, implementation is facilitated. The system knows that the top vertex information is also associated with the slope of the line segment between the top and middle vertices, the middle vertex information is also associated with the slope of the line segment between the middle and bottom vertices, and the bottom vertex information is also associated with the slope of the line segment between the top and bottom vertices.

To illustrate the issues addressed by this re-ordering operation, refer to FIG.

11

(B). Triangle

420

is represented by three vertices

421

,

422

, and

423

. When the triangle is converted to pixels for the output device, the triangle should be oriented as shown with vertex

421

at the top, vertex

422

at the middle, and vertex

423

at the bottom. However, the front end processor may deliver triangle

420

by sending vertex

422

(middle) first, then vertex

423

(bottom), and then finally vertex

421

(top). At other times, the front end processor may deliver triangle

420

by sending vertex

423

(bottom) first, then vertex

421

(top), and then finally vertex

422

(middle). The graphics system in accordance with one embodiment of the present invention re-orders the vertices so that storage in the triangle buffer is consistent with the proper orientation of the triangle. In other words, the system will store vertex

421

(top) at the top, then vertex

422

(middle) immediately below vertex

421

, and then finally vertex

423

(bottom) immediately below vertex

422

in the same cell of the triangle buffer. Referring briefly to

FIG. 6

, triangle buffer has many storage locations called cells. Three exemplary cells are cell

147

, cell

148

, and cell

149

. In each cell, such as cell

149

, the triangle vertices are stored. In accordance with one embodiment of the present invention, the graphics system stores the three vertices according to their top, middle and bottom orientation. Thus, vertex

1

will store the topmost vertex, vertex

2

will store the middle vertex, and vertex

3

will store the bottommost vertex regardless of the order of the vertices that the front end processor delivers.

At times, the triangle is oriented in such a way that two vertices are at the same level. For example, FIG.

11

(A) shows triangle

410

with vertices

411

,

412

, and

413

. Vertex

411

is clearly the top vertex but vertices

412

and

413

are equally at the bottom. In accordance with one embodiment of the present invention, the graphics system would decide that whichever vertex came in first (

412

v.

413

) would be assigned the middle vertex, and of course, vertex

411

would be assigned as the top vertex for storage in the cell in the triangle buffer. In accordance with another embodiment of the present invention, the graphics system would assign the leftmost vertex as the middle vertex (vertex

412

) and the other as the bottom vertex (vertex

413

). The same scheme applies to ties at the top level such as triangle

440

in FIG.

11

(D) where two vertices

441

and

442

are equally the top vertices. In one embodiment, the graphics system would assign vertex

441

as the top vertex because it was sent before vertex

442

, and in another embodiment, vertex

441

would be the top vertex because it is located to the left of vertex

442

.

The flow chart of

FIG. 4

provides a more detailed view of the re-ordering operation. The flow chart starts at step

131

. At step

132

, the system determines the topmost vertex of the triangle. At step

133

, the system determines the middle vertex of the same triangle. At this point, the system knows that the last remaining vertex is the bottom vertex of the same triangle at step

134

. The system then moves the vertices into their correct top, middle, and bottom positions at step

135

. The flow chart ends at step

136

. During the actual writing step, the triangle will be stored in the triangle buffer cell in this correct position.

Bounding Box

The system determines the bounding box information for each triangle. The bounding box allows the system to determine how many coverage masks will be needed to “cover” the bounding box associated with each triangle. The number of coverage masks needed determines the number of triangles issued. Referring briefly to

FIG. 10

, the triangles

401

to

409

are of varying sizes, shapes, and orientations and come in various order. Typically, the front end graphics processor sends smaller triangles such as those triangles within the dotted line

406

for those portions of the screen where finer granularity is necessary. Larger triangles such as triangles

401

,

402

, and

403

are for those portions of the scene such detail is not needed—usually a uniformly colored and textured surface that takes up a large area of the scene. Regardless of the sizes and shapes of the triangles, the graphics system of the present invention determines a bounding box for each triangle as indicated by step

140

of FIG.

3

. The bounding box is the smallest box that can “fit” around the triangle. However, this “fit” determination is made in two-dimensional space; that is, the z depth information of the vertices is ignored. The box is “standing” upright and all angles of the box are 90 degrees. In other words, the bounding box is a rectangle.

Refer to FIGS.

11

(A) to

11

(E) for a conceptual discussion of the bounding box. The triangles in FIGS.

11

(A) and

11

(D) are similar because the one or more sides of the triangle coincide with one or more sides of the bounding box. Other than the fact the triangle

410

in FIG.

11

(A) is an upright standing equilateral triangle and the triangle

440

in FIG.

11

(D) is a right triangle, these two triangles share a common property—one or two of its sides are oriented either at 0 or 90 degrees (perfectly horizontal or perfectly vertical). The triangles in FIGS.

11

(B),

11

(C), and

11

(E) are similar to each other because none of their respective sides are either perfectly horizontal or perfectly vertical. Observe that in order to draw the smallest box around a triangle, at least one corner of the box must intersect a vertex of the triangle.

In FIG.

11

(A), triangle

410

is an equilateral triangle (all sides are the same length) and is oriented in such a manner that vertices

412

and

413

share the same level below vertex

411

. The smallest box that can “bound” the triangle

410

is bounding box

415

. The top side of bounding box

415

intersects top vertex

411

and the bottom side

414

of bounding box

415

intersects vertices

412

and

413

. The left side intersects vertex

412

and the fight side intersects vertex

413

. Here, the bottom side of triangle

410

(the side between vertices

412

and

413

) coincides with the bottom side

414

of the bounding box

415

. By having the sides of the bounding box intersect the vertices of the triangle, it is the smallest box that can be “drawn” around the triangle.

Analogously, triangle

440

in FIG.

11

(D) has one side that is perfectly horizontal and another side that is perfectly vertical. Thus, these sides also coincide with the sides of the bounding box

446

. Triangle

440

has vertices

441

,

442

, and

443

. Side

444

of bounding box

446

coincides with the horizontal side of the triangle

440

located between vertices