FIELD OF THE INVENTION

The present invention relates to information delivery, and relates more specifically to a cache for information objects that are to be delivered efficiently and at high speed over a network to a client.

BACKGROUND OF THE INVENTION

Several important computer technologies rely, to a great extent, upon rapid delivery of information from a central storage location to remote devices. For example, in the client/server model of computing, one or more servers are used to store information. Client computers or processes are separated from the servers and are connected to the servers using a network. The clients request information from one of the servers by providing a network address of the information. The server locates the information based on the provided network address and transmits it over the network to the client, completing the transaction.

The World Wide Web is a popular application of the client/server computing model.

FIG. 1

is a simplified block diagram of the relationship between elements used in a Web system. One or more web clients

10

a

,

10

b

, each of which is a computer or a software process such as a browser program, are connected to a global information network

20

called the Internet, either directly or through an intermediary such as an Internet Service Provider, or an online information service.

A web server

40

is likewise connected to the Internet

20

by a network link

42

. The web server

40

has one or more internet network addresses and textual host names, associated in an agreed-upon format that is indexed at a central Domain Name Server (DNS). The server contains multimedia information resources, such as documents and images, to be provided to clients upon demand. The server

40

may additionally or alternatively contain software for dynamically generating such resources in response to requests.

The clients

10

a

,

10

b

and server

40

communicate using one or more agreed-upon protocols that specify the format of the information that is communicated. A client

10

a

looks up network address of a particular server using DNS and establishes a connection to the server using a communication protocol called the Hypertext Transfer Protocol (HTTP). A Uniform Resource Locator (URL) uniquely identifies each information object stored on or dynamically generated by the server

40

. A URL is a form of network address that identifies the location of information stored in a network.

A key factor that limits the performance of the World Wide Web is the speed with which the server

40

can supply information to a client via the Internet

20

. Performance is limited by the speed, reliability, and congestion level of the network route through the Internet, by geographical distance delays, and by server load level. Accordingly, client transaction time can be reduced by storing replicas of popular information objects in repositories geographically dispersed from the server. Each local repository for object replicas is generally referred to as a cache. A client may be able to access replicas from a topologically proximate cache faster than possible from the original web server, while at the same time reducing Internet server traffic.

In one arrangement, as shown in

FIG. 1

, the cache is located in a proxy server

30

that is logically interposed between the clients

10

a

,

10

b

and the server

40

. The proxy server provides a “middleman” gateway service, acting as a server to the client, and a client to the server. A proxy server equipped with a cache is called a caching proxy server, or commonly, a “proxy cache”.

The proxy cache

30

intercepts requests for resources that are directed from the clients

10

a

,

10

b

to the server

40

. When the cache in the proxy

30

has a replica of the requested resource that meets certain freshness constraints, the proxy responds to the clients

10

a

,

10

b

and serves the resource directly. In this arrangement, the number and volume of data transfers along the link

42

are greatly reduced. As a result, network resources or objects are provided more rapidly to the clients

10

a

,

10

b.

A key problem in such caching is the efficient storage, location, and retrieval of objects in the cache. This document concerns technology related to the storage, location, and retrieval of multimedia objects within a cache. The object storage facility within a cache is called a “cache object store” or “object store”.

To effectively handle heavy traffic environments, such as the World Wide Web, a cache object store needs to be able to handle tens or hundreds of millions of different objects, while storing, deleting, and fetching the objects simultaneously. Accordingly, cache performance must not degrade significantly with object count. Performance is the driving goal of cache object stores.

Finding an object in the cache is the most common operation and therefore the cache must be extremely fast in carrying out searches. The key factor that limits cache performance is lookup time. It is desirable to have a cache that can determine whether an object is in the cache (a “hit”) or not (a “miss”) as fast as possible. In past approaches, caches capable of storing millions of objects have been stored in traditional file system storage structures. Traditional file systems are poorly suited for multimedia object caches because they are tuned for particular object sizes and require multiple disk head movements to examine file system metadata. Object stores can obtain higher lookup performance by dedicating DRAM memory to the task of object lookup, but because there are tens or hundreds of millions of objects, the memory lookup tables must be very compact.

Once an object is located, it must be transferred to the client efficiently. Modern disk drives offer high performance when reading and writing sequential data, but suffer significant performance delays when incurring disk head movements to other parts of the disk. These disk head movements are called “seeks”. Disk performance is typically constrained by the drive's rated seeks per second. To optimize performance of a cache, it is desirable to minimize disk seeks, by reading and writing contiguous blocks of data.

Eventually, the object store will become full, and particular objects must be expunged to make room for new content. This process is called “garbage collection”. Garbage collection must be efficient enough that it can run continually without providing a significant decrease in system performance, while removing objects that have the least impact on future cache performance.

PAST APPROACHES

In the past, four approaches have been used to structure cache object stores: using the native file system, using a memory-blocked “page” cache, using a database, and using a “cyclone” circular storage structure. Each of these prior approaches has significant disadvantages.

The native file system approach uses the file system of an operating system running on the server to create and manage a cache. File systems are designed for a particular application in mind: storing and retrieving user and system data files. File systems are designed and optimized for file management applications. They are optimized for typical data file sizes and for a relatively small number of files (both total and within one folder/directory). Traditional file systems are not optimized to minimize the number of seeks to open, read/write, and close files. Many file systems incur significant performance penalties to locate and open files when there are large numbers of files present. Typical file systems suffer fragmentation, with small disk blocks scattered around the drive surface, increasing the number of disk seeks required to access data, and wasting storage space. Also, file systems, being designed for user data file management, include facilities irrelevant to cache object stores, and indeed counter-productive to this application. Examples include: support for random access and selective modification, file permissions, support for moving files, support for renaming files, and support for appending to files over time. File systems are also invest significant energy to minimize any data loss, at the expense of performance, both at write time, and to reconstruct the file system after failure. The result is that file systems are relatively poorly for handling the millions of files that can be present in a cache of Web objects. File systems don't efficiently support the large variation in Internet multimedia object size—in particular they typically do not support very small objects or very large objects efficiently. File systems require a large number of disk seeks for metadata traversal and block chaining, poorly support garbage collection, and take time to ensure data integrity and to repair file systems on restart.

The page cache extends file systems with a set of fixed sized memory buffers. Data is staged in and out of these buffers before transmission across the network. This approach wastes significant memory for large objects being sent across slow connections.

The database system approach uses a database system as a cache. Generally, databases are structured to achieve goals that make them inappropriate for use as an object cache. For example, they are structured to optimize transaction processing. To preserve the integrity of each transaction, they use extensive locking. As a result, as a design goal they favor data integrity over performance factors such as speed. In contrast, it is acceptable for an object cache to lose data occasionally, provided that the cache does not corrupt objects, because the data always can be retrieved from the server that is original source of the data. Databases are often optimized for fast write performance, since write speed limits transaction processing speed. However, in an object cache, read speed is equally important. Further, databases are not naturally good at storing a vast variety of object sizes while supporting streaming, pipelined I/O in a virtual memory efficient manner. Databases commonly optimized for fixed record size sizes. Where databases support variable record sizes, they contain support for maintaining object relationships that are redundant, and typically employ slow, virtual memory paging techniques to support streaming, pipelined I/O.

In a cyclonic file system, data is allocated around a circular storage structure. When space becomes full, the oldest data is simply removed. This approach allows for fast allocation of data, but makes it difficult to support large objects without first staging them in memory, suffers problems with fragmentation of data, and typically entails naïve garbage collection that throws out the oldest object, regardless of its popularity. For a modest, active cache with a diverse working set, such first-in-first-out garbage collection can throw objects out before they get to be reused.

The fundamental problem with the above approaches for the design of cache object stores is that the solution isn't optimized for the constraints of the problem. These approaches all represent reapplication of existing technologies to a new application. None of the applications above are ideally suited for the unique constraints of multimedia, streaming, object caches. Not only do the above solutions inherently encumber object caches with inefficiencies due to their imperfect reapplication, but they also are unable to effectively support the more unique requirements of multimedia object caches. These unique requirements include the ability to disambiguate and share redundant content that is identical, but has different names, and the opposite ability to store multiple variants of content with the same name, targeted for particular clients, languages, data types, etc.

Based on the foregoing, there is a clear need to provide an object cache that overcomes the disadvantages of these prior approaches, and is more ideally suited for the unique requirements of multimedia object caches. In particular:

1. there is a need for an object store that can store hundreds of millions of objects of disparate sizes, and a terabyte of content size in a memory efficient manner;

2. there is a need for an object store that can determine if a document is a “hit” or a “miss” quickly, without time-consuming file directory lookups;

3. there is a need for a cache that minimizes the number of disk seeks to read and write objects;

4. there is a need for an object store that permits efficient streaming of data to and from the cache;

5. there is a need for an object store that supports multiple different versions of targeted alternates for the same name;

6. there is a need for an object store that efficiently stores large numbers of objects without content duplication;

7. there is a need for an object store that can be rapidly and efficiently garbage collected in real-time, insightfully selecting the documents to be replaced to improve user response speed, and traffic reduction;

8. there is a need for an object store that that can restart to full operational capacity within seconds after software or hardware failure without data corruption and with minimal data loss.

This document concerns technology directed to accomplishing the foregoing goals. In particular, this document describes methods and structures related to the time-efficient and space-efficient storage, retrieval, and maintenance of objects in a large object store. The technology described herein provides for a cache object store for a high-performance, high-load application having the following general characteristics:

1. High performance, measured in low latency and high throughput for object store operations, and large numbers of concurrent operations;

2. Large cache support, supporting terabyte caches and billions of objects, to handle the Internet's exponential content growth rate;

3. Memory storage space efficiency, so expensive semiconductor memory is used sparingly and effectively;

4. Disk storage space efficiency, so large numbers of Internet object replicas can be stored within the finite disk capacity of the object store;

5. Alias free, so that multiple objects or object variants, with different names, but with the same content identical object content, will have the object content cached only once, shared among the different names;

6. Support for multimedia heterogeneity, efficiently supporting diverse multimedia objects of a multitude of types with size ranging over six orders of magnitude from a few hundred bytes to hundreds of megabytes;

7. Fast, usage-aware garbage collection, so less useful objects can be efficiently removed from the object store to make room for new objects;

8. Data consistency, so programmatic errors and hardware failures do not lead to corrupted data;

9. Fast restartability, so an object cache can begin servicing requests within seconds of restart, without requiring a time-consuming database or file system check operation;

10. Streaming, so large objects can be efficiently pipelined from the object store to slow clients, without staging the entire object into memory;

11. Support for content negotiation, so proxy caches can efficiently and flexibly store variants of objects for the same URL, targeted on client browser, language, or other attribute of the client request; and

12. General-purpose applicability, so that the object store interface is sufficiently flexible to meet the needs of future media types and protocols.

SUMMARY OF THE INVENTION

The foregoing needs and other needs are addressed by the present invention, which provides, in one aspect, a method of delivering an information object to a client from a cache at a server, comprising the steps of (A) establishing a cache table in a memory of the server, the cache table comprising a name key that references a vector of alternates; (B) computing a content key that uniquely identifies the information object by applying a hash function to the information object; and (C) storing the content key in the cache table in one of the alternates. One feature of this aspect is storing, in the cache table, a reference to the location of the information object on a mass storage device, wherein the reference is associated with the content key.

Another feature is computing a content key that uniquely identifies the information object by applying a one-way hash function to the information object. Still another feature is (B) computing a content key that uniquely identifies the information object by applying an MD5 one-way hash function to the information object. Yet another feature is receiving a request to retrieve the information object using the name of the information object; looking up the information object in said cache using the name key; retrieving a vector of alternates associated with the name key; selecting a content key from among the alternates based upon the name key; looking up a location of the information object in the mass storage device based on the content key; and delivering the information object to the client.

According to another feature, the method includes (A1) computing the name key based upon the name of the information object and a context of the request. In yet another feature, step (A1) comprises the steps of computing the name key by applying a one-way hash function to the name of the information object combined with information describing a context of the request.

In still another feature, step (A1) comprises the steps of computing the name key by applying the MD5 one-way hash function to the name of the information object combined with information describing a context of the request. Still another feature involves (D) establishing in the cache table, mappings from name keys to vectors of alternates; and (E) storing the content key referencing the content object in each alternate vector element; and (F) establishing in the cache table, mappings from content keys to object content data.

The invention also encompasses an apparatus, computer system, computer program product, and a computer data signal embodied in a carrier wave configured according to the foregoing aspects, and other aspects.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:

FIG. 1

is a block diagram of a client/server relationship;

FIG. 2

is a block diagram of a traffic server;

FIG. 3A

is a block diagram of transformation of an object into a key;

FIG. 3B

is a block diagram of transformation of an object name into a key;

FIG. 4A

is a block diagram of a cache;

FIG. 4B

is a block diagram of a storage mechanism for Vectors of Alternates;

FIG. 4C

is a block diagram of multi-segment directory table;

FIG. 5

is a block diagram of pointers relating to data fragments;

FIG. 6

is a block diagram of a storage device and its contents;

FIG. 7

is a block diagram showing the structure of a pool;

FIG. 8A

is a flow diagram of a process of garbage collection;

FIG. 8B

is a flow diagram of a process of writing information in a storage device;

FIG. 8C

is a flow diagram of a process of synchronization;

FIG. 8D

is a flow diagram of a “checkout_read” process;

FIG. 8E

is a flow diagram of a “checkout_write” process;

FIG. 8F

is a flow diagram of a “checkout_create” process;

FIG. 9A

is a flow diagram of a cache lookup process;

FIG. 9B

is a flow diagram of a “checkin” process;

FIG. 9C

is a flow diagram of a cache lookup process;

FIG. 9D

is a flow diagram of a cache remove process;

FIG. 9E

is a flow diagram of a cache read process;

FIG. 9F

is a flow diagram of a cache write process;

FIG. 9G

is a flow diagram of a cache update process;

FIG. 10A

is a flow diagram of a process of allocating and writing objects in a storage device;

FIG. 10B

is a flow diagram of a process of scaled counter updating;

FIG. 11

is a block diagram of a computer system that can be used to implement the present invention;

FIG. 12

is a flow diagram of a process of object re-validation.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

A method and apparatus for caching information objects is described. In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.

TRAFFIC SERVER

FIG. 2

is a block diagram of the general structure of certain elements of a proxy

30

. In one embodiment, the proxy

30

is called a traffic server and comprises one or more computer programs or processes that operate on a computer workstation of the type described further below. A client

10

a

directs a request

50

for an object to the proxy

30

via the Internet

20

. In this context, the term “object” means a network resource or any discrete element of information that is delivered from a server. Examples of objects include Web pages or documents, graphic images, files, text documents, and objects created by Web application programs during execution of the programs, or other elements stored on a server that is accessible through the Internet

20

. Alternatively, the client

10

a

is connected to the proxy

30

through a network other than the Internet.

The incoming request

50

arrives at an input/output (I/O) core

60

of the proxy

30

. The I/O core

60

functions to adjust the rate of data received or delivered by the proxy to match the data transmission speed of the link between the client

10

a

and the Internet

20

. In a preferred embodiment, the I/O core

60

is implemented in the form of a circularly arranged set of buckets that are disposed between input buffers and output buffers that are coupled to the proxy

30

and the Internet

20

. Connections among the proxy

30

and one or more clients

10

a

are stored in the buckets. Each bucket in the set is successively examined, and each connection in the bucket is polled. During polling, the amount of information that has accumulated in a buffer associated with the connection since the last poll is determined. Based on the amount, a period value associated with the connection is adjusted. The connection is then stored in a different bucket that is generally identified by the sum of the current bucket number and the period value. Polling continues with the next connection and the next bucket. In this way, the elapsed time between successive polls of a connection automatically adjusts to the actual operating bandwidth or data communication speed of the connection.

The I/O core

60

passes the request

50

to a protocol engine

70

that is coupled to the I/O core

60

and to a cache

80

. The protocol engine

70

functions to parse the request

50

and determine what type of substantive action is embodied in the request

50

. Based on information in the request

50

, the protocol engine

70

provides a command to the cache

80

to carry out a particular operation. In an embodiment, the cache

80

is implemented in one or more computer programs that are accessible to the protocol engine

70

using an application programming interface (API). In this embodiment, the protocol engine decodes the request

50

and performs a function call to the API of the cache

80

. The function call includes, as parameter values, information derived from the request

50

.

The cache

80

is coupled to send and receive information to and from the protocol engine

70

and to interact with one or more non-volatile mass storage devices

90

a

-

90

n

. In an embodiment, the storage devices

90

a

-

90

n

are high-capacity, fast disk drives. The cache

80

also interacts with data tables

82

that are described in more detail herein.

OBJECT CACHE INDEXING

CONTENT INDEXING

In the preferred embodiment, the cache

80

stores objects on the storage devices

90

a

-

90

n

. Popular objects are also replicated into a cache. In the preferred embodiment, the cache has finite size, and is stored in main memory or RAM of the proxy

30

.

Objects on disk are indexed by fixed sized locators, called keys. Keys are used to index into directories that point to the location of objects on disk, and to metadata about the objects. There are two types of keys, called “name keys” and “object keys”. Name keys are used to index metadata about a named object, and object keys are used to index true object content. Name keys are used to convert URLs and other information resource names into a metadata structure that contains object keys for the object data. As will be discussed subsequently, this two-level indexing structure facilitates the ability to associate multiple alternate objects with a single name, while at the same time maintaining a single copy of any object content on disk, shared between multiple different names or alternates.

Unlike other cache systems that use the name or URL of an object as the key by which the object is referenced, embodiments of the invention use a “fingerprint” of the content that makes up the object itself, to locate the object. Keys generated from the content of the indexed object are referred to herein as object keys. Specifically, the object key

56

is a unique fingerprint or compressed representation of the contents of the object

52

. Preferably, a copy of the object

52

is provided as input to a hash function

54

, and its output is the object key

56

. For example, a file or other representation of the object

52

is provided as input to the hash function, which reads each byte of the file and generates a portion of the object key

56

, until the entire file has been read. In this way, an object key

56

is generated based upon the entire contents of the object

52

rather than its name. Since the keys are content-based, and serve as indexes into tables of the cache

80

, the cache is referred to as a content-indexed cache. Given a content fingerprint key, the content can easily be found.

In this embodiment, content indexing enables the cache

80

to detect duplicate objects that have different names but the same content. Such duplicates will be detected because objects having identical content will hash to the same key value even if the objects have different names.

For example, assume that the server

40

is storing, in one subdirectory, a software program comprising an executable file that is 10 megabytes in size, named “IE

4

.exe”. Assume further that the server

40

is storing, in a different subdirectory, a copy of the same file, named “Internet Explorer.exe”. The server

40

is an anonymous FTP server that can deliver copies of the files over an HTTP connection using the FTP protocol. In past approaches, when one or more clients request the two files, the cache stores a copy of each of the files in cache storage, and indexes each of the files under its name in the cache. As a result, the cache must use 20 megabytes of storage for two objects that are identical except for the name.

In embodiments of the invention, as discussed in more detail herein, for each of the objects, the cache creates a name key and an object key. The name keys are created by applying a hash function to the name of the object. The object keys are created by applying a hash function to the content of the object. As a result, for the two exemplary objects described above, two different name keys are created, but the object key is the same. When the first object is stored in the cache, its name key and object key are stored in the cache. When the second object is stored in the cache thereafter, its name key is stored in the cache. However, the cache detects the prior identical object key entry, and does not store a duplicate object key entry; instead, the cache stores a reference to the same object key entry in association with the name key, and deletes the new, redundant object. As a result, only 10 megabytes of object storage is required. Thus, the cache detects duplicate objects that have different names, and stores only one permanent copy of each such object.

FIG. 3A

is a block diagram of mechanisms used to generate an object key

56

for an object

52

. When client

10

a

requests an object

52

, and the object is not found in the cache

80

using the processes described herein, the cache retrieves the object from a server and generates a object key

56

for storing the object in the cache.

Directories are the data structures that map keys to locations on disk. It is advisable to keep all or most of the contents of the directories in memory to provide for fast lookups. This requires directory entries to be small, permitting a large number of entries in a feasible amount of memory. Further, because 50% of the accesses are expected not to be stored in cache, we want to determine cache misses quickly, without expending precious disk seeks. Such fast miss optimizations dedicate scarce disk head movements to real data transfers, not unsuccessful speculative lookups. Finally, to make lookups fast via hashing search techniques, directory entries are fixed size.

Keys are carefully structured to be fixed size and small, for the reasons described earlier. Furthermore, keys are partitioned into subkeys for the purposes of storage efficiency and fast lookups. Misses can be identified quickly by detecting differences in just a small portion of keys. For this reason, instead of searching a full directory table containing complete keys, misses are filtered quickly using a table of small subkeys called a “tag table”. Furthermore, statistical properties of large bit vectors can be exploited to create space-efficient keys that support large numbers of cache objects with small space requirements.

According to one embodiment, the object key

56

comprises a set subkey

58

and a tag subkey

59

. The set subkey

58

and tag subkey

59

comprise a subset of the bits that make up the complete object key

56

. For example, when the complete object key

56

is 128 bits in length, the subkeys

58

,

59

can be 16 bits, 27 bits, or any other portion of the complete key. The subkeys

58

,

59

are used in certain operations, which are described below, in which the subkeys yield results that are nearly as accurate as when the complete key is used. In this context, “accurate” means that use of the subkeys causes a hit in the cache to the correct object as often as when the complete key is used.

This accuracy property is known as “smoothness” and is a characteristic of a certain preferred subset of hash functions. An example of a hash function suitable for use in an embodiment is the MD5 hash function, which is described in detail in B. Schneier, “Applied Cryptography” (New York: John Wiley & Sons, Inc., 2d ed. 1996), at pp. 429-431 and pp. 436-441. The MD5 hash function generates a 128-bit key from an input data stream having an arbitrary length. Generally the MD5 hash function and other one-way hash functions are used in the cryptography field to generate secure keys for messages or documents that are to be transmitted over secure channels. General hashing table construction and search techniques are described in detail in D. Knuth, “The Art of Computer Programming: Vol. 3, Sorting and Searching,” at 506-549 (Reading, Ma.: Addison-Wesley, 1973).

NAME INDEXING

Unfortunately, requests for objects typically do not identify requested objects using the object keys for the objects. Rather, requests typically identify requested objects by name. The format of the name may vary from implementation to implementation based on the environment in which the cache is used. For example, the object name may be a file system name, a network address, or a URL.

According to one aspect of the invention, the object key for a requested object is indexed under a “name key” that is generated based on the object name. Thus, retrieval of an object in response to a request is a two phase process, where a name key is used to locate the object key, and the object key is used to locate the object itself.

FIG. 3B

is a block diagram of mechanisms used to generate a name key

62

based on an object name

53

. According to one embodiment, the same hash function

54

that is used to generate object keys is used to generate name keys. Thus, the name keys will have the same length and smoothness characteristics of the object keys.

Similar to object key

56

, the name key

62

comprises set and tag subkeys

64

,

66

. The subkeys

64

,

66

comprise a subset of the bits that make up the complete name key

62

. For example, when the complete name key

62

is 128 bits in length, the first and second subkeys

64

,

66

can be 16 bits, 27 bits, or any other portion of the complete key.

SEARCHING BY OBJECT OR NAME KEY

Preferably, the cache

80

comprises certain data structures that are stored in the memory of a computer system or in its non-volatile storage devices, such as disks.

FIG. 4

is a block diagram of the general structure of the cache

80

. The cache

80

generally comprises a Tag Table

102

, a Directory Table

110

, an Open Directory table

130

, and a set of pools

200

a

through

200

n

, coupled together using logical references as described further below.

The Tag Table

102

and the Directory Table

110

are organized as set associative hash tables. The Tag Table

102

, the Directory Table

110

, and the Open Directory table

130

correspond to the tables

82

shown in FIG.

2

. For the purposes of explanation, it shall be assumed that an index search is being performed based on object key

56

. However, the Tag Table

102

and Directory Table

110

operate in the same fashion when traversed based on a name key

62

.

The Tag Table

102

is a set-associative array of sets

104

a

,

104

b

, through

104

n

. The tag table is designed to be small enough to fit in main memory. Its purpose is to quickly detect misses, whereby using only a small subset of the bits in the key a determination can be made that the key is not stored in the cache. The designation

104

n

is used to indicate that no particular number of sets is required in the Tag Table

102

. As shown in the case of set

104

n

, each of the sets

104

a

-

104

n

comprises a plurality of blocks

106

.

In the preferred embodiment, the object key

56

is 128 bits in length. The set subkey

58

is used to identify and select one of the sets

104

a

-

104

n

. Preferably, the set subkey

58

is approximately 18 bits in length. The tag subkey

59

is used to reference one of the entries

106

within a selected set. Preferably, the tag subkey

59

is approximately 16 bits in length, but may be as small as zero bits in cases in which there are many sets. In such cases, the tag table would be a bit vector.

The mechanism used to identify or refer to an element may vary from implementation to implementation, and may include associative references, pointers, or a combination thereof. In this context, the term “reference” indicates that one element identifies or refers to another element. A remainder subkey

56

′ consists of the remaining bits of the key

56

. The set subkey, tag subkey, and remainder subkey are sometimes abbreviated s, t, and r, respectively.

The preferred structure of the Tag Table

102

, in which each entry contains a relatively small amount of information enables the Tag Table to be stored in fast, volatile main memory such as RAM. Thus, the structure of the Tag Table

102

facilitates rapid operation of the cache. The blocks in the Directory Table

110

, on the other hand, include much more information as described below, and consequently, portions of the Directory Table may reside on magnetic disk media as opposed to fast DRAM memory at any given time.

The Directory Table

110

comprises a plurality of sets

110

a

-

110

n

. Each of the sets

110

a

-

110

n

has a fixed size, and each comprises a plurality of blocks

112

a

-

112

n

. In the preferred embodiment, there is a predetermined, constant number of sets and a predetermined, constant number of blocks in each set. As shown in the case of block

112

n

, each of the blocks

112

a

-

112

n

stores a third, remainder subkey value

116

, a disk location value

118

, and a size value

120

. In the preferred embodiment, the remainder subkey value

116

is a 27-bit portion of the 128-bit complete object key

56

, and the comprises bits of the complete object key

56

that are disjoint from the bits that comprise the set or tag subkeys

58

,

59

.

In a search, the subkey values stored in the entry

106

of the Tag Table

102

matches or references one of the sets

110

a

-

110

n

, as indicated by the arrow in

FIG. 4

that connects the entry

106

to the set

110

d

. As an example, consider the 12-bit key and four-bit first and second subkeys described above. Assume that the set subkey value 1111 matches set

104

n

of the Tag Table

102

, and the tag subkey value 0000 matches entry

106

of set

104

n

. The match of the tag subkey value 0000 indicates that there is a corresponding entry in set

110

d

of the Directory Table

110

associated with the key prefix 11110000. When one of the sets

110

a

-

110

n

is selected in this manner, the blocks within the selected set are searched linearly to find a block, such as block

112

a

, that contains the remainder subkey value

116

that matches a corresponding portion of the object key

56

. If a match is found, then there is almost always a hit in the cache. There is a small possibility of a miss if the first, second and third subkeys don't comprise the entire key. If there is a hit, the referenced object is then located based on information contained in the block, retrieved from one of the cache storage devices

90

a

-

90

n

, and provided to the client

10

a

, as described further below.

Unlike the Tag Table, whose job is to quickly determine rule out misses with the minimal use of RAM memory, each block within Directory Table

110

includes a full pointer to a disk location. The item referenced by the disk location value

118

varies depending on the source from which the key was produced. If the key was produced based on the content of an object, as described above, then the disk location value

118

indicates the location of a stored object

124

(or a first fragment thereof), as shown in

FIG. 4

in the case of block

112

b

. If the key is a name key, then as shown for block

112

n

, the disk location value

118

indicates the location of one or more Vectors of Alternates

122

, each of which stores one or more object keys for the object whose name was used to generate the name key. A single Tag Table

102

and a single Directory Table

110

are shown in

FIG. 4

merely by way of example. However, additional tables that provide additional levels of storage and indexing may be employed in alternate embodiments.

In the preferred arrangement, when a search of the cache is conducted, a hit or miss will occur in the Tag Table

102

very quickly. If there is a hit in the Tag Table

102

, then there is a very high probability that a corresponding entry will exist in the Directory Table

110

. The high probability results from the fact that a hit in the Tag Table

102

means that the cache holds an object whose full key shares X identical bits to the received key, where X is the number of bits of the concatenation of the set and tag subkeys

58

and

59

. Because misses can be identified quickly, the cache

80

operates rapidly and efficiently, because hits and misses are detected quickly using the Tag Table

102

in memory without requiring the entire Directory Table

110

to reside in main memory.

When the cache is searched based on object key

56

, the set subkey

58

is used to index one of the sets

104

a

-

104

n

in Tag Table

102

. Once the set associated with subkey

58

is identified, a linear search is performed through the elements in the set to identify an entry whose tag matches the tag subkey

59

.

In a search for an object

52

requested from the cache

80

by a client

10

a

, when one of the sets

104

a

-

104

n

is selected using the set subkey

58

, a linear search of all the elements

106

in that set is carried out. The search seeks a match of the tag subkey

59

to one the entries. If a match is found, then there is a hit in the Tag Table

102

for the requested object, and the cache

80

proceeds to seek a hit in the Directory Table

110

.

For purposes of example, assume that the object key is a 12-bit key having a value of 111100001010, the set subkey comprises the first four bits of the object key having a value of 1111, and the tag subkey comprises the next four bits of the object key having a value of 0000. In production use the number of remainder bits would be significantly larger than the set and tag bits to affect memory savings. The cache identifies set 15 (1111) as the set to examine in the Tag Table

102

. The cache searches for an entry within that set that contains a tag 0000. If there is no such entry, then a miss occurs in the Tag Table

102

. If there is such an entry, then the cache proceeds to check the remaining bits in Directory Table

110

for a match.

MULTI-LEVEL DIRECTORY TABLE

In one embodiment, the Directory Table

110

contains multiple sets each composed of a fixed number of elements. Each element contains the remainder tag and a disk pointer. Large caches will contain large numbers of objects, which will require large numbers of elements in the directory table. This can create tables too large to be cost-effectively stored in main memory.

For example, if a cache was configured with 128 million directory table elements, and each element was represented by a modest 8 bytes of storage, 1GByte of memory would be requires to store the directory table, which is more memory than is common on contemporary workstation computers. Because few of these objects will be actively accessed at any time, there is a desire to migrate the underutilized entries onto disk while leaving higher utilized entries in main memory.

FIG. 4C

is a diagram of a multi-level directory mechanism. The directory table

110

is partitioned into segments

111

a

,

111

b

,

111

c

. In the preferred embodiments, there are two or three segments

111

a

-

111

c

, although a larger number of segments may be used. The first segment

111

a

is the smallest, and fits in main memory such as the main memory

1106

of the computer system shown in FIG.

11

and discussed in detail below. The second and third segments

111

b

,

111

c

are progressively larger. The second and third segments

111

b

,

111

c

are coupled through a paging mechanism to a mass storage device

1110

such as a disk. The second and third segments

111

b

,

111

c

dynamically page data in from the disk if requested data is not present in the main memory

1106

.

As directory elements are accessed more often, the directory elements are moved to successively higher segment among the segments

111

a

-

111

c

of the multi-level directory. Thus, frequently accessed directory elements are more likely to be stored in main memory

1106

. The most popular elements appear in the highest and smallest segment

111

a

of the directory, and will all be present in main memory

1106

. Popularity of entries is tracked using a small counter that is several bits in length. This counter is updated as described in the section SCALED COUNTER UPDATING. This multi-level directory approximates the performance of in-memory hash tables, while providing cost-effective aggregate storage capacity for terabyte-sized caches, by placing inactive elements on disk.

DIRECTORY PAGING

As discussed, in a preferred embodiment, the Directory Table

110

is implemented as a multi-level hash table. Portions of the Directory Table may reside out of main memory, on disk. Data for the Directory Table is paged in and out of disk on demand. A preferred embodiment of this mechanism uses direct disk I/O to carefully control the timing of paging to and from disk and the amount of information that is paged.

Another embodiment of this approach exploits a feature of UNIX-type operating systems to map files directly into virtual memory segments. In this approach, the cache maps the Directory Table into virtual memory using the UNIX mmap( ) facility. For example, a mmap request is provided to the operating system, with a pointer to a file or disk location as a parameter. The mmap request operates as a request to map the referenced file or disk location to a memory location. Thereafter, the operating system automatically loads portions of the referenced file or disk location from disk into memory as necessary.

Further, when the memory location is updated or accessed, the memory version of the object is written back to disk as necessary. In this way, native operating system mechanisms are used to manage backup storage of the tables in non-volatile devices. However, at any given time it is typical that only a portion of the Directory Table

110

is located in main memory.

In a typical embodiment, the Directory Table and Open Directory are stored using a “striping” technique. Each set of the tables is stored on a different physical disk drive. For example, set

110

a

of Directory Table

110

is stored on storage device

90

a

, set

110

b

is stored on storage device

110

b

, etc. In this arrangement, the number of seek operations needed for a disk drive head to arrive at a set is reduced, thereby improving speed and efficiency of the cache.

It should be noted when paging data between disk and memory certain safeguards are taken to ensure that the information stored in memory is consistent with the corresponding information stored in a non-volatile storage device. The techniques used to provide efficient consistency in object caches are summarized in the context of garbage collection, in the section named SYNCHRONIZATION AND CONSISTENCY ENFORCEMENT.

VECTOR OF ALTERNATES

As mentioned above, it is possible for a single URL to map to an object that has numerous versions. These versions are called “alternates”. In systems that do not use an object cache, versions are selected as follows. The client

10

a

establishes an HTTP connection to the server

40

through the Internet

20

. The client provides information about itself in an HTTP message that requests an object from the server. For example, an HTTP request for an object contains header information that identifies the Web browser used by the client, the version of the browser, the language preferred by the client, and the type of media content preferred by the client. When the server

40

receives the HTTP request, it extracts the header information, and selects a variant of the object

52

based upon the values of the header information. The selected alternate is returned to the client

10

a

in a response message. This type of variant selection is promoted by the emerging HTTP/1.1 hypertext transfer protocol.

It is important for a cache object store to efficiently maintain copies of alternates for a URL. If a single object is always served from cache in response to any URL requests, a browser may receive content that is different than that obtained directly from a server. For this reason, each name key in the directory table

110

maps to one of the vectors of alternates

122

a

-

122

n

, which enable the cache

80

to select one version of an object from among a plurality of related versions. For example, the object

52

may be a Web page and server

40

can store versions of the object in the English, French, and Japanese languages.

Each Vector of Alternates

122

a

-

122

n

is a structure that stores a plurality of alternate records

123

a

-

123

n

. Each of the alternate records

123

a

-

123

n

is a structure that stores information that describes an alternative version of the requested object

52

. For example the information describes a particular browser version, a human language in which the object has been prepared, etc. The alternate records also each store a full object key that identifies an object that contains the alternative version. In the preferred embodiment, each of the alternate records

123

a

-

123

n

stores request information, response information, and an object key

56

.

Because a single popular object name may map to many alternates, in one embodiment a cache composes explicit or implicit request context with the object name to reduce the number of elements in the vector. For example, the User-Agent header of a Web client request (which indicates the particular browser application) may be concatenated with a web URL to form the name key. By including contextual information directly in the key, the number of alternates in each vector is reduced, at the cost of more entries in the directory table. In practice, the particular headers and implicit context concatenated with the information object name is configurable.

These Vectors of Alternates

122

a

-

122

n

support the correct processing of HTTP/1.1 negotiated content. Request and response information contained in the headers of HTTP/1.1 messages is used to determine which of the alternate records

123

a

-

123

n

can be used to satisfy a particular request. When cache

80

receives requests for objects, the requests typically contain header information in addition to the name (or URL) of the desired object. As explained above, the name is used to locate the appropriate Vector of Alternates. Once the appropriate Vector of Alternates is found, the header information is used to select the appropriate alternate record for the request.

Specifically, in the cache

80

, the header information is received and analyzed. The cache

80

seeks to match values found in the header information with request information of one of the alternate records

123

a

-

123

n

. For example, when the cache

80

is used in the context of the World Wide Web, requests for objects are provided to a server containing the cache in the form of HTTP requests.

The cache

80

examines information in an HTTP request to determine which of the alternate records

123

a

-

123

n

to use. For example, the HTTP request might contain request information indicating that the requesting client

10

a

is running the Netscape Navigator browser program, version 3.0, and prefers German text. Using this information, the cache

80

searches the alternate records

123

a

through

123

n

for response information that matches the browser version and the client's locale from the request information. If a match is found, then the cache retrieves the object key from the matching alternate and uses the object key to retrieve the corresponding object from the cache.

The cache optimizes the object chosen by matching the criteria specified in the client request. The client request may specify minimal acceptance criteria (e.g. the document must be a JPEG image, or the document must be Latin). The client request may also specify comparative weighting criteria for matches (e.g. will accept a GIF image with weight 0.5, but prefer a JPEG image at weight 0.75). The numeric weightings are accumulated across all constraint axes to create a final weighting that is optimized.

The object key is used to retrieve the object in the manner described above. Specifically, a subkey portion of the object key is used to initiate another search of the Tag Table

102

and the Directory Table

110

, seeking a hit for the subkey value. If there is a hit in both the Tag and Directory Tables, then the block in the Directory Table arrived at using the subkey values will always reference a stored object (e.g. stored object

124

). Thus, using the Vector of Alternates

122

, the cache

80

can handle requests for objects having multiple versions and deliver the correct version to the requesting client

10

a.

In

FIG. 4

, only one exemplary Vector of Alternates

122

and one exemplary stored object

124

are shown. However, in practice the cache

80

includes any number of vectors and disk blocks, depending on the number of objects that are indexed and the number of alternative versions associated with the objects.

READ AHEAD

FIG. 4B

is a diagram showing a storage arrangement for exemplary Vectors of Alternates

122

a

-

122

n

. The system attempts to aggregate data object contiguously after the metadata. Because seeks are time-consuming but sequential reads are fast, performance is improved by consolidating data with metadata, and pre-fetching data after the metadata.

In one of the storage devices

90

a

-

90

n

, each of the Vectors of Alternates

122

a

-

122

n

is stored in a location that is contiguous to the stored objects

124

a

-

124

b

that are associated with the alternate records

123

a

-

123

n

represented in the vector. For example, a Vector of Alternates

122

a

stores alternate records

123

a

-

123

c

. The alternate record

123

a

stores request and response information indicating that a stored object

124

a

associated with the alternate record is prepared in the English language. Another alternate record

123

b

stores information indicating that its associated stored object

124

b

is intended for use with the Microsoft Internet Explorer browser. The stored objects

124

a

,

124

b

referenced by the alternate records

123

a

,

123

b

are stored contiguously with the Vectors of alternates

122

a

-

122

n.

The Size value

120

within each alternate record indicates the total size in bytes of one of the associated Vectors of Alternates

122

a

-

122

n

and the stored object

124

. When the cache

80

references a Vector of Alternates

122

a

based on the disk location value

118