专利汇可以提供Method and means of matching documents based on spatial region layout专利检索,专利查询,专利分析的服务。并且,下面是Method and means of matching documents based on spatial region layout专利的具体信息内容。
This invention is related to object matching based on shape similarity and, more particularly, to a method and means by which spatial layout of regions can be captured for purposes of matching documents.
In today's world, increasing numbers of documents are being scanned in large quantities or are being created electronically. To maintain and manage these documents requires new methods that analyze, store and retrieve the documents. Current document management systems can support document database creation from scanned documents and indexing based on text queries. A need for allowing more visual queries has been felt, particularly in retrieving documents when text keywords are unreliably extracted (from scanned documents due to OCR errors), or retrieve too many choices for a user to select from. In such cases the intention of the user is best captured by either allowing more flexible queries making reference to a document genre or type (say, find me a "letter" from "X" regarding "sales" and "support"), or by simply pointing to an icon or example, and asking "find me a document looking similar to it in visual layout." Performing either requires an ability to automatically derive such document genre or type information from similarity in the visual layouts of documents rather than their precise text content, which may be quite different. An example illustrating this can be seen from Figures 1A and 1B which are two similar-looking documents with very different text content.
Matching based on spatial layout similarity is a difficult problem, and has not been well-addressed. The above examples also illustrate the outstanding difficulty. The two documents in Figure 1A and 1B are regarded as similar even though their logically corresponding regions (text segments) shown in Figures 2A and 2B, respectively, differ in size. Furthermore, some of the corresponding regions have moved up while others have moved down and by different amounts.
It is known to extract a symbolic graph-like description of regions and perform computationally intensive subgraph matching to determine similarity, as seen in the work of Watanabe in "Layout Recognition of Multi-Kinds of Table-Form Documents", IEEE Transactions Pattern Analysis and Machine Intelligence. Furthermore, US-A Patent No. 5,642,288 to Leung et al. entitled "Intelligent document recognition and handling" describes a method of document image matching by performing some image processing and forming feature vectors from the pixel distributions within the document.
From Bohnacker U et al: "Matching form lines based on a Heuristic Search" Proceedings of the Fourth International Conference on Document Analysis and Recognition (Cat. No. 97TB100138), proceedings of the fourth international conference on document analysis and recognition, Ulm, Germany, 18 to 20 August, 1997, pages 86 to 90 Vol. 1, XP002165325 1997, Los Alamitos, CA, USA, IEEE Computer. Society, USA. a form identification method based on the A* algorithm is known. From a reference form and a filled form to be compared, two sets of horizontal and vertical lines are built which are searched for pairs of corresponding lines. Spatial metrics are computed and a search tree is built in which corresponding lines of the two forms are stored. After an evaluation of the local certainty of a matched pair of lines it is decided whether the two forms being compared match each other.
Another related document is Niyogi D et al: "An Integrated Approach to Document Decomposition and Structural Analysis" International Journal of Imaging Systems and Technology, US, Wiley and Sons, New York, Vol. 7, No. 4, 21 December 1996 (1996-12-21), pages 330 to 342, XP000637454. The document describes document decomposition and structural analysis, which constitutes one of the major processes involved in document image understanding. Decomposition and structural analysis includes logical grouping and read-order determination on complex multi-articled documents. The system uses bottom-up segmentation techniques to identify the block structure of a document and layout rules to classify and group these blocks into logical units that represent meaningful subdivisions of the document.
The invention as defined in claims 1 and 8 is a method and corresponding system for matching objects, with specific examples of matching documents, based on spatial layout of regions that addresses the above difficulties. It employs a shape similarity model for detecting similarity between 2D objects. The shape similarity model is general enough to encompass the individual region shape variations between members of a shape class, and yet specific enough to avoid mismatches to objects with perceptually different appearance. That is, the shape model models the change in shape of corresponding regions on objects by a set of separate affine deformations, with constraints on the transforms that are intended to capture perceptual shape similarity between objects.
Using the shape model, two objects are taken to match if one of them can be found to belong to the shape class of the other document. Specifically, the "document" matching proceeds in 4 stages, namely, (1) pre-processing, in which logical regions are automatically derived from information in the documents to be matched, (2) region correspondence, in which a correspondence is established between the regions on the documents, and (3) pose computation, in which the individual transforms relating corresponding regions are recovered, and finally (4) verification, in which the extent of spatial similarity is measured by projecting one document onto the other using the computed pose parameters.
The document matching method specifically described herein can be suitably combined with other text-based retrieval methods to enhance the capability of current document management systems. Such a document matching method has several applications. It can be used to describe document genres (such as letters, memos) based on spatial layout. Other uses of the document matching method include the clustering of documents based on similarity for purposes of document database organization.
The "object" matching method includes the following features:
The invention disclosed here is a method of object matching based on a shape model to capture shape layout similarity. In modeling shape similarity, objects are characterized by a collection of regions representing some logical entity of the object, such as say, a logical text region like a paragraph. The methods to obtain such regions are expected to be domain-specific and frequently involve some image pre-processing. Although the term "document" is used throughout this disclosure, it is not meant to limit the application of the invention to documents, but rather this method is intended to apply broadly to "object" matching.
The document matching method described herein generally proceeds in 4 stages after documents containing logical regions to be matched are identified. The steps are: Pre-processing 1, in which logical regions are automatically derived from information in the documents to be matched; Region correspondence 2, in which a correspondence is established between the regions on the documents; Pose computation 3, in which the individual transforms relating corresponding regions are recovered, and finally, Verification 4 is conducted, in which the extent of spatial similarity is measured by projecting one document onto the other using the computed pose parameters. These steps will be discussed in more detail in Section B of this disclosure, below
The document matching method is based on the following shape similarity model that is intended to capture spatial layout similarity of logical regions in 2d objects and, in particular, logical regions of a document image.
The shape similarity model describes the characteristics of the shape class of an object M consisting of a set of m regions Rmi, i = l to m. According to the shape model, the object M is said to be similar to another object I characterized by the set of regions R Ij,j = 1,..n. if enough pairs of corresponding regions can be found such that the shape deformations of the corresponding regions can be specified by a set of affine transforms (Aij,Tij) that obey the following three constraints:
The first constraint specifies that object regions displace along a common direction, called the reference direction. That is, the direction of residual translation of corresponding regions must be the same which is denoted by:
The direction of residual translation can be either manually specified or can be automatically derived by computing the direction of residual motion for all pairs of regions, and recording the commonality in direction using a Hough transform.
The second constraint restricts the amount of displacement each of the regions can undergo to maintain perceptual similarity. The extent of residual translation of all corresponding regions is bounded by δ so that |
Note that included in the constraint on the extent of residual translation, is the case when some regions don't move at all from their original positions while others do along a common direction. When (Aij,Tij) is the same for all regions j for an object i, this reduces to a rigid-body shape model.
The final constraint restricts the displacement of regions such that relative ordering is maintained. That is, the ordering of corresponding regions on objects with respect to the reference direction θ be the same. The ordering of regions is obtained by projecting the centroids of regions onto the reference direction using a direction of projection (orthogonal or oblique). Such a region ordering for an object can be conveniently represented by a sequence R = (Rj1,Rj2,..Rjm). Regions of the same rank appear in this sequence ordered along the direction of projection.
The above constraints have been carefully chosen through studies that observe that such constraints perceptual shape similarity for a wide variety of objects, including faces, MRI scans, etc.
The method of document matching disclosed in this invention involves the following stages:
To use the shape similarity model for document matching, a set of logical regions need to be derived. While the document matching methods admits several methods of obtaining logical regions, we chose to obtain them by a grouping algorithm that uses text segments given by a conventional text segmentation algorithm (we used a version of software in Xerox's TextBridge ™ for extracting text segments). The grouping algorithm performs the following operations:
The above algorithm can be efficiently implemented using a data structure called the union-find data structure as described in a book by Cormen, Leisersen and Rivest entitled "Introduction to algorithms", MIT Press, 1994, to run in time linear in the number of text regions in the document.
The above algorithm has been found particularly useful for grouping consecutive paragraphs of text into single logical regions, as well as for grouping centrally aligned information such as author information in a journal article.
The method of obtaining region correspondence is again meant for general objects, and can be easily adapted to logical regions of documents. The correspondence between logical regions is obtained by using the constraints in the shape similarity model. Thus starting with all pairs of regions on the two objects, all those pairs whose direction of residual translation is not in the specified direction θ are pruned (this is checked within a threshold to allow some robustness against segmentation errors and deviations from the shape similarity model in individual documents). Next, the extent of residual translation constraint is used to further prune the pairs. The distinct regions in the pairs on each object can now be ordered with respect to the reference direction θ. The region orderings can be denoted by the sequences RM and RI respectively. Using the region ordering, and collecting the set of candidate matching regions in object M for each region of object I by Si the result can be denoted by the set sequence SP = (S1,S2,..Sp) where p is the number of regions in object I that found a match to a region in object M. The best region correspondence is taken as the longest subsequence of RM that is also a member sequence of Sp. A member sequence of Sp is a sequence of regions with at most one region taken from the set Si. It can be easily shown that the problem of finding the longest common subsequence (LCS) has optimal substructure property, thus admitting solutions by dynamic programming. In fact, we adapted the dynamic programming algorithm for computing an LCS described in Introduction to algorithms by T.Cormen, Leiserson, and R. Rivest, to give region correspondence by LCS using the following simple procedure:
The above steps give the largest set of corresponding regions between query and prototype that move along the same direction within bounds of the shape model, have a match in individual content, and retain the spatial layout ordering specified. Although the number of possible LCS for general sequences can be exponential in the worst case, for typical spatial layouts of regions in documents, only a few distinct LCS have to be tried to discover shape similarity.
Figures 3A and 3B is used to illustrate the region correspondence for a simple example. The figures are two objects to be matched some of whose regions meet the constraints of the shape similarity model. Their respective region orderings with respect to the reference direction shown in the figure, are given by (ABCD) and (EFGHIJK). The set sequence of candidate matches is given by ({A},{B},{A},{B},{C},{},{D}), where S1 = {A}, S2 = {B}, and so on. There are two LCS that have length with respective region correspondences as {(A,E)(B,H)(C,I)(D,K)} and {(A,G)(B,H)(C,I)(D,K)}. The correctness of these correspondences can be judged in the recognition stage to be described next.
Using the correspondence between logical regions, the individual region transforms can be recovered in a variety of ways including feature matching or direct region matching as mentioned in a paper entitled "Object recognition by region correspondence" in Proceedings Intl. Conference on Computer Vision (ICCV), Boston, 1995 by R. Basri and D. Jacobs. For the domain of documents, since the logical regions are rectangular, the pose parameters of interest are the four elements of the linear transform matrix A and the residual translation T. For a pair of corresponding regions RMi and RIj, these are denoted by
Pose verification involves determining if the two documents register under the shape similarity model. For this the computed residual translation given in the above equation is corrected such that the resulting residual translation is forced to be exactly along the reference direction and within the stated bounds on the extent of such displacement. This is done by perpendicularly projecting the point representing the computed residual translation onto the line of direction θ and taking the point of such projection as the new residual translation Tijnew for each pair of corresponding regions.
Each rectangular region Ri on object M can now be projected onto the object I to give the projected rectangular regionl R'i as follows. The centroid of the region CMi is moved to the position
The above formula accounts for the extent of match as measured by the extent of spatial overlap of corresponding regions, and the extent of mismatch as measured by the areas of regions that do not find the match (included in the denominator term).
Referring to Figure 4, a flow chart representing the four major steps of document recognition employing the invention, after logical regions of documents to be matched are identified. Logical region extraction occurs within the first two blocks where First 1 all region pairs are formed, and then the documents are pruned 2 based on unary constraints discussed in further detail below. Region correspondence is then determined between documents 3. A pose is computed 4 for the documents and match verification is determined based a matching score. Figure 5 is another diagram depicting a more specific scenario where two documents (1 and 2) are scanned, and enter a region segmentation module 7 to establish a correspondence between the regions on the documents. A logical region grouping module 8 is then allowed form region pairs, and unary constraints are then applied to the documents in a layout shape matching module 9 resulting is a matching score between the documents.
Referring back to Figures 1A and 2A, illustrated is the logical region grouping for documents. Figure 1A shows text regions given by a conventional text segmentation algorithm. Figure 2A shows the result of logical grouping on the document image of Figure 1A.
Next, we illustrate region correspondence. Figure 6A depicts a model document by its logical regions. Figures 6B and C are two other documents only one of which is similar (Figure 6B) in spatial layout to the document in Figure 6A. This can be observed from the higher number of corresponding regions (labeled R1 through R5 in Figure 7) obtained when the document of Figure 6B is matched to Figure 6A. Here we assume a vertical reference direction, and the matching regions are indicating by identical colors. As can be seen, the poor correspondence of object in Figure 6C indicates a mismatch. This can also be seen during the verification stage where the pose parameters computed (and corrected) from region correspondences defined by overlap as the document of Figure 6A is overlayed onto the documents of Figures 6B and C. The extent of overlap in such overlay is indicated in Figures 7A and B, respectively, as R1 through R5.
The method of the invention can be carried out in a microprocessor-based system 10 as shown generally in Figure 8. The microprocessor 11 would be programmed to carry out the four main steps of the invention. A memory 12 would be utilized by the microprocessor 11 for storing document templates and tested documents during matching operations. A scanner 13 may be used to scan the test documents into the document matching system; however, as known in the art, documents may be delivered to the system via electronic networks, or the like. Results of testing can be output 14 to the user with indicating means known in the art.
The method of document matching by spatial region layout can be a useful complement to existing methods for document matching based on text keywords or pixel-wise image content. As can be seen from the examples above, the capturing of spatial layout similarity allows the matching of documents that have greater variations in pixel-wise image content. In addition, the matching method is a general formulation that can be applied to other classes of 2D objects besides documents.
标题 | 发布/更新时间 | 阅读量 |
---|---|---|
心磁测量系统 | 2020-05-11 | 375 |
一种紧凑型自带折叠收纳桌的组合柜 | 2020-05-12 | 581 |
一种基于视觉伺服的机械臂自动对接系统 | 2020-05-14 | 258 |
一种超磁致伸缩电静液作动器 | 2020-05-13 | 653 |
一种考虑信号阻隔效应的城市空间5G基站选址优化方法 | 2020-05-12 | 842 |
一种悬臂梁式自升式钻井平台预压载方法 | 2020-05-13 | 377 |
一种模拟空中两机加油对接实验平台 | 2020-05-08 | 163 |
一种清洁取暖方式选择方法 | 2020-05-12 | 473 |
太阳光热保温发酵车间 | 2020-05-11 | 505 |
一种座便器的冲洗控制阀及无水箱座便器 | 2020-05-14 | 105 |
高效检索全球专利专利汇是专利免费检索,专利查询,专利分析-国家发明专利查询检索分析平台,是提供专利分析,专利查询,专利检索等数据服务功能的知识产权数据服务商。
我们的产品包含105个国家的1.26亿组数据,免费查、免费专利分析。
专利汇分析报告产品可以对行业情报数据进行梳理分析,涉及维度包括行业专利基本状况分析、地域分析、技术分析、发明人分析、申请人分析、专利权人分析、失效分析、核心专利分析、法律分析、研发重点分析、企业专利处境分析、技术处境分析、专利寿命分析、企业定位分析、引证分析等超过60个分析角度,系统通过AI智能系统对图表进行解读,只需1分钟,一键生成行业专利分析报告。