专利汇可以提供Methods and systems for managing an application's relationship to its run-time environment专利检索,专利查询,专利分析的服务。并且Disclosed are a framework for managing an application's relationship to its run-time environment and an engine that accepts the framework as input and runs the application within the environment. Aspects of the framework and engine may be changed to suit a changing environment without changing the application itself. By managing details of the environment, the invention leaves developers free to focus on the specific tasks of the application. The framework also allows the engine to provide standardized services such as deadlock and leak detection, progress monitoring, and results logging. As an example, the application may be a software test suite. The invention allows the test suite to be run single- or multi-threaded and with individual tests within the suite running consecutively or concurrently. The invention can be parameterized to accommodate different testing goals, such as basic variation testing, regression testing, and stress testing, without changing the test suite itself.,下面是Methods and systems for managing an application's relationship to its run-time environment专利的具体信息内容。
We claim:
The present invention relates generally to running software applications, and, more particularly, to an application's relationship to its run-time environment.
The relationship of a typical software application to its run-time environment is growing more complex for at least four interrelated reasons. First, environments are themselves growing more complex. The rise of telecommunications often allows an application to take advantage of, or requires an application to be aware of, services and resources provided by remote computing devices located throughout the environment. Each remote device may present its own set of communications peculiarities, such as novel protocols or real-time response constraints. The number of possible interactions grows exponentially with the number of devices and applications involved in one computing task, and all of these interactions need to be managed efficiently. Second, environments may change their characteristics over time. For example, a given application may always need to secure a specific type of resource from a specific type of remote server, but the identities of the servers available to provide that type of resource may change moment by moment. At the same time, new services may be introduced and old services may disappear. Third, the application may need to run in multiple environments. The number of possible environments is proliferating as, for example, hardware and software platforms are optimized for particular uses. Even if an application's relationship to any one environment were to remain unchanged (unlikely for the reasons discussed above), the growth in number and diversity of possible environments leads to a demand for an increase in the flexibility, and thus the complexity, of the application's environmental interactions. Compounding the well known difficulties of developing an application to meet diverse requirements of diverse environments, a new environment may arise whose requirements were unforeseen when the application was developed. Fourth, applications themselves are becoming increasingly complex. For example, an application that formerly ran as one thread on a single, stand-alone computing device may now be reconstituted to run as multiple, interacting threads on multiple devices spread throughout the globe and connected by telecommunications links. One application may need to serve multiple purposes and must be able to present itself accordingly, for instance by providing a variable range of services dependent upon the sophistication of its current users. (In this situation, the requirements of the application's users are considered to be part of the application's environment.) As can be appreciated, these four reasons act in concert, the effects of each contributing, often in unpredictable ways, to the magnitude of the effects of the others.
As an especially pointed example of this situation, consider the case of an application designed to test other applications. (To differentiate between the application doing the testing and the application being tested, the former will be called the “test system” while the latter will be called the “application under test.”) The four reasons given above for increased complexity apply here to a heightened extent because the reasons may apply both to the relationship of the application under test to its run-time environment (its run-time environment including the test system) and to the relationship of the test system to its own run-time environment. As one example, to thoroughly exercise the application under test, the test system should exercise all aspects of the application under test's environmental interactions, through a range of possible environments and as those environments change. For any given environment, the test system may need to call upon different aspects of the application under test for different types of testing, such as basic variation testing, regression testing, and stress testing. This may require that the test system run multiple copies of the application under test at the same time, and each copy may be performing multiple tasks simultaneously. While the test system is coordinating all of these activities of the application under test, the test system still needs to handle its own complex interactions with its own environment, such as reporting test results, trapping execution errors, and detecting other potentially harmful aspects of the application under test's environmental relationship (such as dead locks and memory leaks). It is clear from this example that developing an application in the face of multiple, changing environments can be very challenging, and that the challenges are heightened when developing a test system.
Developers often address complexity in their application's relationships by designing aspects of the application that can be configured to meet changing circumstances. An application's configuration parameters can then be set when the application is run (or when it is compiled). Different sets of configurable parameters are set to reflect different aspects of the application's relationship to its run-time environment. Of course, when building flexibility into an application by means of configurable parameters, the developer predicts the range of needs in the possible set of run-time environments so that the configurable parameters, and the application's response to them, can be set.
Useful as configurable parameters are, it can still be extremely difficult to correctly initialize all of these parameters in a complex application's configuration. Subtle errors may arise from mismatches between one part of an application and other parts of the same application, or between the application and other applications in its environment. Mismatches include unexpected events, unplanned for sequences of events, data values outside the range of normalcy, updated behavior of one application not matched by updates in its peers, etc. Difficult as configuration is for one application, it is exponentially more difficult to configure multiple applications (such as the combination of a test system and an application under test) so that they interact with their environments and with each other in a predictable fashion. Adding to the difficulty of creating correct configurations, a configuration may change with time. Applications may be used in an environment, such as a testing environment, in which their configurations may be changed every time they operate. An application may need to run in an environment whose parameters are beyond the range envisaged by the application's developer and so beyond the range of its configurable parameters. In addition to these considerations, it is not desirable for an application developer to devote too much time to configuration issues: they distract the developer from working on issues at the core of the application and they may require environmental expertise foreign to the developer. Indeed, the expertise needed to correctly develop an application's configurable relationships may exist in no one person.
Developing an application's flexibility so that the application can respond correctly regardless of the environments within which it must run and then managing that flexibility in the face of multiple, changing environments are becoming increasingly burdensome. There is a need to contain the burgeoning complexity of an application's relationship to its run-time environment and to separate relationship issues from core application issues.
In view of the foregoing, the present invention provides a framework for managing an application's relationship to its run-time environment and an engine that accepts the framework as input and runs the application within the environment. Aspects of the framework and of the engine may be changed to suit a changing environment without changing the application itself. By managing details of the application's relationship to its environment, the invention leaves application developers free to focus on the specific tasks of the application. Standard methods for providing services and resources allow some aspects of application development to become standardized. Standard interfaces to, for example, error trapping, progress tracking, and resource use and abuse reporting are developed for use by any application.
As an example, the application may be a software test suite. The invention allows the test suite to be run single- or multi-threaded and with individual tests within the suite running consecutively or concurrently, all without altering the underlying tests. The framework and engine are parameterized to accommodate different testing goals, such as basic variation testing, regression testing, and stress testing, without changing the test suite itself. For any application being tested by the test suite, the engine provides deadlock and leak detection, progress monitoring, and results logging.
While the appended claims set forth the features of the present invention with particularity, the invention, together with its objects and advantages, may be best understood from the following detailed description taken in conjunction with the accompanying drawings of which:
Turning to the drawings, wherein like reference numerals refer to like elements, the invention is illustrated as being implemented in a suitable computing environment. The following description is based on embodiments of the invention and should not be taken as limiting the invention with regard to alternative embodiments that are not explicitly described herein. Section I presents an exemplary computing environment in which the invention may run. Section II describes exemplary embodiments of the invention's framework and engine, showing their structures and operations. To better illustrate the concepts presented in Section II, Section III presents the details of an actual application, the IOHammer Test Suite, developed for use with the invention.
In the description that follows, the invention is described with reference to acts and symbolic representations of operations that are performed by one or more computers, unless indicated otherwise. As such, it will be understood that such acts and operations, which are at times referred to as being computer-executed, include the manipulation by the processing unit of the computer of electrical signals representing data in a structured form. This manipulation transforms the data or maintains them at locations in the memory system of the computer, which reconfigures or otherwise alters the operation of the computer in a manner well understood by those skilled in the art. The data structures where data are maintained are physical locations of the memory that have particular properties defined by the format of the data. However, while the invention is being described in the foregoing context, it is not meant to be limiting as those of skill in the art will appreciate that various of the acts and operations described hereinafter may also be implemented in hardware.
Referring to
In its most basic configuration, a computing device typically includes at least one processing unit 102 and memory 104. The memory 104 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. This most basic configuration is illustrated in
The invention creates data structures and functions to form a consistent framework around an application, connecting the application to the application's run-time environment. Because this framework is created in a consistent manner for every application and with well defined properties, it allows the invention to provide to applications common support and management features that would otherwise have to be built into each application. These features enable an application, written once, to run without modification in different run-time environments, even in environments with characteristics unforeseen at the time the application was written.
The framework data structures 202 consist of two major parts, the application table 208 and the parameter table 210. The application table contains information about each sub-application and includes special functions created by the invention as part of the framework. The parameter table holds variables used by the engine 200 and by the sub-applications 206 when they run. These two tables are described further with reference to
After the module-level checks and initialization, sub-applications 206 in the application 204 are run. Depending upon the application's requirements, these sub-applications may be run consecutively, concurrently, or in some combination of the two. For example, the application may be a test suite targeting a communications application. One sub-application may represent an originator of a communications session, a second sub-application may represent a peer communicating device, while a third sub-application performs the function of a protocol analyzer. These three sub-applications would be run concurrently. In any case, a global initialize function 304 is called to begin the processing of each sub-application. This function works on environmental variables at a level between the generality of the module initialize function 302 and the thread initialize function 306. This division of initialization into separate levels is one method employed by the invention to provide flexible, yet consistent, management of the run-time environment. Of course, if the application consists of only one sub-application, then the global initialize function may be merged into the module initialize function.
Sub-applications 206 may sometimes be run as multiple, concurrent threads. For example, the sub-application may be a variation in a test suite application 204. To regression test the features of an application under test, it may be sufficient to run the variation as a single thread. On the other hand, multiple, concurrent threads can be run to stress test the application under test. To capture the details inherent in running the sub-application as a variable number of threads, the invention provides a thread initialize function 306.
With the run-time environment set up, the invention runs the sub-application 206 itself. As it runs, the sub-application can call on the framework data structures 202 (discussed below with reference to
The end of the process of running an application 204 mirrors the beginning. Each thread ends with a thread terminate function 320. Then each sub-application 206 ends with a global terminate function 322. Finally, the whole application ends with a module terminate function 324 and a module clean up function 326.
Note that the break down into the particular framework functions shown in
Next come references to several of the framework functions discussed with reference to
The parameter table 210 contains one entry 418 for each parameter that can be set and passed when the application 204 is run. Some of these parameters are defined by the application developer, others are standard for all applications. The standard parameters are described below with reference to steps 600, 602, 604, and 606 of
The application table 208 contains an entry 432 for each sub-application 206. After identifying the sub-application by name 434, description 436, and category 438, a flags field 440 specifies run-time characteristics such as whether this sub-application is enabled to be run and whether it should run in a separate thread. The execution factors field 442 contain values that are multiplied by loop counter parameters set by the user to arrive at the actual number of times a loop is performed. For example, if the user sets the sub-application count parameter (discussed below with reference to step 604 of
The following fields refer to framework functions discussed with reference to
wttautogen.exe SPfail -p:2 SpsrvPort CrashTypeFlag -v:2 Crash Restart
generates a framework for an application named SPfail with two parameters SpsrvPort and CrashTypeFlag. The SPfail application contains the two sub-applications Crash and Restart. The command next prompts the user to specify the data types of the parameters. The command then begins to automatically create the framework functions and populate the framework data structures 202.
In step 502, the command creates a parameter table 210 with an entry 418 for SpsrvPort and one for CrashTypeFlag. Because the command knows nothing of these parameters beyond their data types, it populates the parameter table entries with default values. In a similar manner, the command creates in step 504 an application table 208 with an entry 432 for the Crash sub-application and one for Restart.
The command creates a module table 400 that refers to the parameter table 210 and the application table 208 in step 506. It too is populated with default values. The default values for the module table and for the application table include default framework functions. For example, a thread initialize function 306 specific to the Crash sub-application 206 within the SPfail application 204 is put into Crash's application table entry 432.
At this point, a framework has been created, but it reflects very little of the what the application developer had in mind for his application 204. The development work is mostly concentrated into step 508. There, the developer edits the created framework functions and data structures to match his conception of the application. He may, for example, choose to leave some framework functions unaltered, but would certainly flesh out the default sub-application functions 206. The present invention does not remove the development of the application from the developer. Rather, it provides a framework that encapsulates the application and removes from the developer much of the burden of dealing with variability in the application's run-time environment.
In steps 600, 602, 604, and 606, a user asks to run an application 204 and sets parameters for the run. Note that these steps are separated for purposes of this discussion: in some embodiments, all parameters are set in one step. In step 600, the user specifies which sub-applications 206 to run. For example, the command:
SPfail -test:Crash -!test:Restart
runs the Crash sub-application of the SPfail application but prevents the Restart sub-application from running. (The keyword “test” is used to specify a sub-application for historical reasons.)
In step 602, the user sets flags for the sub-applications 206 to be run. The format is:
SPfail -param:<sub-application name or all> <parameter name> <parameter value>
so that the command:
SPfail -param:Crash mint 10
sets the minimum wait time (part of the timers field 444) before calling the Crash sub-application to ten seconds. Other timers and the flags in field 440 can also be set this way.
The user in step 604 sets sub-application control factors. Table 1 is an exemplary list of these factors. The descriptions in the Table are sufficient for most parameters so only a few are mentioned here. The loop counters specified in the command line are multiplied by the execution factors 442 to arrive at the actual number of times the loops are performed. When the application 204 is a test suite, the tests in it are often run over and over again to stress the application under test. This is done by setting the loop counters to high values. As a check on this, parameters can be set that terminate the run of the test suite when a maximum run time is reached or when a maximum number of sub-applications (test variations) have been run. By setting the logging level, the user controls the level of detail in the reported results. An example command setting some of these parameters is:
SPfail -test:Crash -MRT:1200 -MS:50 -log:spfail.log -LL:7 -threads:3
The sub-application Crash of application SPfail is run 50 times or for 1200 seconds (whichever comes first). Results are logged at a “World” level to the file spfail.log. During the run, the number of threads executing is limited to three.
Step 606 allows the user to specify debugging and error control information. Table 2 lists some of the possibilities here. These concepts are familiar in the industry, but putting this functionality into the framework produces the advantage of relieving the developer from having to implement these controls in each application.
The engine in step 608 uses the information gathered in the previous steps and information stored in the framework data structures 202 to create the sub-application data structure 700 for this run. That data structure is discussed with reference to
Step 610 through the End on
A pointer to the sub-application data structure 700 of
The global data class 716 of
The concepts presented above may be more easily grasped in the context of a concrete example. In this section, a straightforward test suite, called “IOHammer,” is presented. IOHammer stresses a computer's hard disk by quickly writing to it. IOHammer is presented in terms of its framework data structures 202 and its framework functions.
Beginning with the framework data structures 202, the following is the module table 400. The module table has NULLs for some of the framework functions: module clean up 326, module check 300, print usage 410, and process arguments 412. In keeping with the flexibility offered by the framework, these functions may be defined if they are useful, or, as in the simple case presented here, left undefined at the application developer's discretion.
The IOHammer test suite is defined to take two parameters. As seen from the following parameter table 210, one parameter is the size of a write buffer and the other is the disk drive that IOHammer will exercise. If no disk drive is set, then IOHammer will exercise all accessible disk drives.
The IOHammer test suite has only one sub-application, so its application table 208 has only one entry 432. As is often the case, the execution factors 442 are all set to one. As in the module table 400, some functions are not defined: global initialize 304, global terminate 322, sub-application test 310, sub-application post 312, and sub-application post test 316.
As seen from
For each thread that will be run, the IOHammerThreadInitialize function 802 sets up a write file on each disk drive that will be exercised.
The IOHammer test suite as presented here has only one sub-application. The IOHammerVariationWrite sub-application 804 writes to the files set up in the IOHammerThreadInitialize function 802, using the write buffers allocated in the IOHammerInitialize function 800.
The IOHammerThreadTerminate function 810 cleans up by closing and deleting the files set up by the IOHammerThreadInitialize function 802.
The IOHammerTerminate function 812 incorporates the functionality of the global terminate 322, module terminate 324, and module clean up 326 functions. It frees the write buffers allocated by the IOHammerInitialize function 800.
It can be appreciated from this example that for applications much more complicated than IOHammer, the framework itself does not become more complicated. The framework may incorporate more parameters, and more of the framework functions may be defined, but the framework substantially separates issues of the application's run-time environment from issues of the application's core functionality.
In view of the many possible embodiments to which the principles of this invention may be applied, it should be recognized that the embodiments described herein with respect to the drawing figures are meant to be illustrative only and should not be taken as limiting the scope of invention. Therefore, the invention as described herein contemplates all such embodiments as may come within the scope of the following claims and equivalents thereof.
标题 | 发布/更新时间 | 阅读量 |
---|---|---|
软件程序的测试方法和装置、存储介质和电子设备 | 2020-05-11 | 38 |
回归测试用例确定方法、装置、计算机设备及存储介质 | 2020-05-17 | 957 |
一种基于节点重要度评估的回归测试用例优先级排序方法 | 2020-05-08 | 644 |
一种回归测试驱动的软件版本管理方法、系统及存储介质 | 2020-05-21 | 804 |
测试数据帧生成方法及装置、软件测试方法及装置 | 2020-05-25 | 318 |
回归测试的方法、装置、计算机设备和存储介质 | 2020-05-12 | 138 |
一种机构内部的众测管理方法及系统 | 2020-05-20 | 716 |
一种基于电网系统的自动化测试方法 | 2020-05-26 | 637 |
面向航天软件的在轨升级方法 | 2020-05-14 | 934 |
数据源配置测试方法、系统、计算机设备和存储介质 | 2020-05-24 | 255 |
高效检索全球专利专利汇是专利免费检索,专利查询,专利分析-国家发明专利查询检索分析平台,是提供专利分析,专利查询,专利检索等数据服务功能的知识产权数据服务商。
我们的产品包含105个国家的1.26亿组数据,免费查、免费专利分析。
专利汇分析报告产品可以对行业情报数据进行梳理分析,涉及维度包括行业专利基本状况分析、地域分析、技术分析、发明人分析、申请人分析、专利权人分析、失效分析、核心专利分析、法律分析、研发重点分析、企业专利处境分析、技术处境分析、专利寿命分析、企业定位分析、引证分析等超过60个分析角度,系统通过AI智能系统对图表进行解读,只需1分钟,一键生成行业专利分析报告。