Technical Field

This invention relates to methods and apparatus for estimating unknown quantities. More particularly, the present invention relates to methods and apparatus for estimating unknown quantities under a rule which cannot be clearly expressed by numerical formulae or a rule which can be expressed by numerical formulae, and which requires much labor for actually calculating the numerical formulae.

Background Art

From past years, a so called artificial neural network is known in which plural neuron devices are classified into multiple layers, each neuron device belonging to each layer is connected to all neuron devices belonging to other layers, and by supplying an input pattern to neuron devices belonging to an input layer, a predetermined output pattern is output from neuron devices belonging to an output layer, the predetermined output pattern being determined based upon coefficient of coupling and threshold value of each neuron device. The artificial neural network is called as a hierarchical perceptron because the neuron devices constitute a hierarchical arrangement in the artificial neural network.

In the hierarchical perceptron, a so called learning is repeated. The learning is performed by supplying a known input pattern to a group of neuron devices of the input layer and supplying a known output pattern corresponding to the known input pattern as a teacher pattern, by calculating a difference between the teacher pattern and an output pattern output from the neuron devices belonging to the output layer based upon the input pattern, and by varying the coefficient of coupling and the threshold value of each neuron device so as to decrease the calculated difference. After carrying out sufficient learning, when an arbitrary input pattern is supplied to the neuron devices belonging to the input layer, an output pattern which should correspond to the input pattern is output from the neuron devices belonging to the output layer. That is, when at least a part of a plurality of input data of an input pattern is varied, corresponding unknown quantities are obtained as an output pattern.

The hierarchical perceptron is determined its kinds and number of an data of input pattern, and when output pattern is known which should be output corresponding to a known input pattern, after carrying out the learning by a necessary number of times using these known data, an output pattern corresponding to an arbitrary input pattern is obtained. Therefore, the learning should be carried out again when the kind and number of data of the input pattern are varied, a disadvantage arises that varying in applicable extent cannot be easily dealt with.

Further, when the number of data of the input pattern increases, and the number of neuron devices belonging to each layer or the number of layers is not increased, accuracy of the output pattern is lowered, therefore the number of neuron devices constituting the hierarchical perceptron is great, and a disadvantage arises that a required time for learning is lengthened caused by the number of the neuron devices being great.

Furthermore, when a part of the input pattern and data which should be a true output pattern are obtained by a conventionally known measurement method and the like, a disadvantage arises that obtaining the rest of the input pattern is impossible caused by the arrangement of the hierarchical perceptron.

The present invention was made to solve the above-mentioned problems. It is an object of the present invention to supply novel methods and apparatus for estimating unknown quantities, the methods and apparatus enable estimating unknown quantities under a rule which cannot be clearly expressed by numerical formulae or a rule which can be expressed by numerical formulae, and which requires much labor for actually calculating the numerical formulae.

Disclosure of The Invention

To perform the object above-mentioned, a method for estimating unknown quantities according to claim 1 comprises the steps of;

   supplying arbitrary quantities corresponding to unknown inputs and common known inputs as input values and a predetermined measurement value as a teacher pattern to plural artificial neural networks each of which has multiple layers arrangement and has finished learning of a predetermined function,

   correcting the arbitrary quantities corresponding to the unknown inputs in each artificial neural network so as to decrease a difference between a cumulative addition value and the teacher pattern based upon the input values, the difference between the teacher pattern and the cumulative addition value of outputs from the plural artficial neural networks, and a partial differential values corresponding to the corresponding artificial neural network, and

   obtaining quantities which are corrected so as to make the difference to the least value as estimated results of unknown inputs.

As to the method for estimating unknown quantities according to claim 1, plural sets of artificial neural networks are provided, each artificial neural network being performed learning so as to obtain a predetermined output corresponding to rather simple multiple data of input pattern. Arbitrary quantities corresponding to unknown inputs and known inputs are supplied as input values and a predetermined measurement value is supplied as a teacher pattern to the plural sets of artificial neural networks, correction of an input layer of each artificial neural network is carried out so as to decrease the difference between the cumulative addition value and the measurement yalue. And, correction results of the arbitrary quantities corresponding to the unknown inputs of each artificial neural network is obtained as unknown quantities estimation results when the difference becomes sufficiently small.

Therefore, scale of each artificial neural network should not be enlarged too great so that a required time is extremely shortened which is required for learning of a function at a first stage for each artificial neural network.

A method for estimating unknown quantities according to claim 2 comprises the steps of;

   obtaining plural function value output by supplying known inputs to plural artificial neural networks each of which has multiple layers arrangement and has finished learning of a predetermined function,

   obtaining a difference between a cumulative addition value of each obtained function value output and a measurement value previously obtained in correspondence to the known inputs, and a difference between partial differential values corresponding to each function value output and partial differential values previously obtained in correspondence to the known inputs,

   learning a partial differntial function in each artificial neural network based upon the calculated difference between the partial differential values, the known inputs, and the difference between the predetermined measurement value previously obtained in correspondence to the known inputs and the cumulative addition value,

   repetitively carrying out the learning of the partial differential function till the difference between the predetermined measurement value and the cumulative addition value and the difference between the partial differential values become sufficiently small,

   supplying arbitrary quantities corresponding to unknown inputs and common known inputs as input values and a predetermined measurement value as a teacher pattern to all artificial neural networks,

   correcting the arbitrary quantities corresponding to the unknown inputs in each artificial neural network so as to decrease a difference between a cumulatively addition value and the teacher pattern based upon the input values, the difference between the teacher pattern and the cumulative addition value of outputs from the plural artficial neural networks, and a partial differential value corresponding to the corresponding artificial neural network, and

   obtaining quantities which are corrected so as to make the difference to the least value as estimated results of unknown inputs.

As to the method for estimating unknown quantities according to claim 2, plural sets of artificial neural networks are provided, each artificial neural network being performed learning so as to obtain a predetermined output corresponding to rather simple multiple data of input pattern. Known quantities are supplied to each artificial neural network instead of the unknown quantities which are objected for estimation. The difference between the measurement value previously obtained in correspondence to the known quantities and the cumulative addition value of the function value outputs from each artificial neural network, and the difference between the partial differential values each of which corresponds to the function value output from each artificial neural network and the partial differential values (partial differential value teacher pattern) each of which is previously obtained by numerical differential method and the like are obtained. By carrying out learning of the partial differential function in each artificial neural network based upon the difference between the teacher pattern previously obtained in correspondence to the known inputs and the cumulative addition value, and the difference between the teacher pattern and the partial differential values, pre-processing of an unknown quantities estimating system in its entirety which includes the plural sets of artificial neural networks, is finished. Thereafter, arbitrary quantities corresponding to unknown inputs and known inputs, and a measurement value are supplied to the plural sets of artificial neural networks, correction of an input layer of each artificial neural network is carried out so as to decrease the difference between the cumulative addition value and the measurement value. And, correction results of the arbitrary quantities corresponding to the unknown inputs of each artificial neural network is obtained as unknown quantities estimation results when the difference becomes sufficiently small.

Therefore, scale of each artificial neural network should not be enlarged too great so that a required time is extremely shortened which is required for learning of a function at a first stage for each artificial neural network. And, a required time for learning the partial differential function is also shortened because learning of the partial differential function in each artificial neural network which has finished learning of the function in such manner is carried out based upon the difference between the partial differential value corresponding to the function value output and the partial differential value (partial differential value teacher pattern) which has previously obtained, and the difference between the measurement value which is known information and the cumulative addition value of the estimated values of each artificial neural network.

The known inputs, for example measurement condition and the like, and arbitrary quantities corresponding to unknown inputs are supplied as input values and the measurement value is supplied as a teacher pattern to the plural sets of artificial neural networks which have finished the pre-processings (the learning of the function and the learning of the partial differential function), and in this condition, correction of an input layer for inputting unknown quantities of each artificial neural network is carried out so as to decrease the difference between the cumulative addition value of the function value outputs from all artificial neural networks and the teacher pattern. Therefore, estimating of unknown quantities which was not possible at all in past years, is easily performed.

An apparatus for estimating unknown quantities according to claim 3 comprises;

   plural artificial neural networks each of which has multiple layers arrangement and has finished learning of a predetermined function,

   partial differential value calculating means for calculating a partial differential value corresponding to a function value output of each artificial neural network,

   cumulative addition means for cumulatively adding the function value outputs each of which is output from each artificial neural network,

   first difference calculating means for calculating a difference between a predetermined teacher pattern which has previously obtained and the cumulative addition value,

   second difference calculating means for calculating a difference between a partial differential value corresponding to the function value output and a partial differential value which has previously obtained, for every artificial neural network,

   partial differential function learning means for carrying out learnng of a partial differential function based upon the difference between the partial differential values, calculated by the second difference calculating means, and the difference between the teacher pattern and the cumulative addition value, calculated by the first difference calculating means,

   input unknown quantities correcting means for correcting arbitrary quantities corresponding to unknown inputs and for obtaining the corrected quantities which make the differences to the least values, as estimated result of unknown quantities, in response to all artificial neural networks having been performed learning of the partial differential function by the partial differential function learning means so as to lessen the difference calculated by the first difference calculating means and the difference calculated by the second difference calculating means than a predetermined value, wherein all artificial neural networks are supplied arbitrary quantities corresponding to the unknown inputs and the common known inputs as input values so as to obtain the function value outputs and the cumulative addition value of the function value outputs, each artificial neural network corrects the arbitrary quantities so as to decrease the difference between the cumulative addition value and a teacher pattern based upon the difference between the cumulative addition value and the teacher pattern, and a partial differential value corresponding to the function value output of the corresponding artificial neural network, and

   information taking out means for taking out the quantities which are corrected by the unknown quantities correcting means.

As to the apparatus for estimating unknown quantities according to claim 3, by supplying the known inputs and the known teacher pattern to each artificial neural network which has multiple layers arrangement and has finished the learning of the predetermined function, the difference between the cumulative addition value of the function value outputs and the teacher pattern is obtained by the first difference calculating means, and the difference between the partial differential value which is calculated by the partial differential value calculating means and the partial differential value (partial differential value teacher pattern) which is previously obtained by numerical differential method and the like is obtained by the second difference calculating means. Learning of the partial differential function is carried out in each artificial neural network by the partial differential function learning means based upon the difference between the obtained cumulative addition value and the teacher pattern, and the difference between the partial differential values. And, after the learning of the partial differential function is carried out by the partial differential function learning means till the difference between the cumulative addition value and the teacher pattern and the difference between the partial differential values become sufficiently small, arbitrary quantities corresponding to the unknown inputs are corrected by the unknown quantities correcting means so as to decrease the difference between the cumulative addition value and the teacher pattern based upon the partial differential value corresponding to the function value output of the corresponding artificial neural network and the difference between the teacher pattern and the cumulative addition value of the function value outputs which are obtained by supplying the arbitrary quantities corresponding to the unknown inputs and the common known inputs as the input values to all artificial neural networks, and the quantities which decrease the differences to the least values are obtained as the estimated result by the unknown quantities correcting means. Then, the quantities corrected by the unknown quantities correcting means are taken out by the information taking out means. The taken out quantities are the unknown quantities which are objected for estimation.

More particularly, by carrying out the predetermined learning in each artificial neural network, a condition is obtained in which each artificial neural network performs a processing which is equivalent to a numerical formulae regulating a relationship between plural inputs and an output. And, learning of the partial differential function is carried out in each obtained artificial neural network based upon the difference between the cumulative addition value of the function value outputs and the teacher pattern, and the difference between the partial differential value corresponding to the function value output and the partial differential value (partial differential value teacher pattern) which is previously obtained, by supplying known inputs which should essentially be unknown quantities and the teacher pattern to the artificial neural network. Consequently, plural processings the number of which is determined in correspondence to kinds of the unknown quantities, which processings are equivalent to numerical formulae (processings and the like equivalent to plural numerical formulae which have different constants to one another, respectively) and processings equivalent to partial differential functions of the numerical formulae are possible to be carried out in corresponding artificial neural network. Thereafter, the output is obtained by carrying out corresponding processing in each artificial neural network, then the estimated value corresponding to the measurement value by cumulatively adding the outputs of all artificial neural networks. Then, the difference between the estimated value and the measurement value is calculated. By correcting the unknown inputs of each artificial neural network based upon the calculated difference, the arbitrary quantities corresponding to the unknown inputs are varied so as to decrease the difference. When the varying processing of the unknown inputs is repetitively carried out, and the difference becomes sufficiently small, the corrected quantities are taken out as unknown quantities estimated result.

An apparatus for estimating unknown quantities according to claim 4 comprises;

   plural artificial neural networks each of which has multiple layers arrangement and has finished learning of a predetermined function,

   partial differential value calculating means for obtaining variation quantity of function value output as a partial differential value by varying a known quantity which should correspond to an unknown input of each artificial neural network by a very small amount,

   cumulative addition means for cumulatlvely adding the function value outputs each of which is output from each artificial neural network,

   first difference calculating means for calculating a difference between a predetermined teacher pattern which has previously obtained and the cumulative addition value,

   second difference calculating means for calculating a difference between a partial differential value corresponding to the function value output and a partial differential value which has previously obtained, for every artificial neural network,

   partial differential function learning means for carrying out learning of a partial differential function based upon the difference between the partial differential values, calculated by the second difference calculating means, and the difference between the teacher pattern and the cumulative addition value, calculated by the first difference calculating means,

   input unknown quantities correcting means for correcting arbitrary quantities corresponding to unknown inputs and for obtaining the corrected quantities which make the differences to the least values, as estimated result of unknown quantities, wherein all artificial neural networks are supplied arbitrary quantities corresponding to the unknown inputs and the common known inputs as input values so as to obtain the cumulative addition value of the function value outputs, each artificial neural network corrects the arbitrary quantities so as to decrease the difference between the cumulative addition value and a teacher pattern based upon the difference between the cumulative addition value and the teacher pattern, and a partial differential value corresponding to the function value output of the corresponding artificial neural network, and

   information taking out means for taking out the quantities which are corrected by the unknown quantities correcting means.

As to the apparatus for estimating unknown quantities according to claim 4, by supplying the known inputs and the known teacher pattern to each artificial neural network which has multiple layers arrangement and has finished the learning of the predetermined function, the difference between the cumulative addition value of the function value outputs and the teacher pattern is obtained by the first difference calculating means, and the output variation value is obtained by the partial differential value calculating means as the partial differential value by varying the known quantities by a very small amount which quantities should correspond to unknown input information. Then, arbitrary quantities corresponding to the unknown inputs are corrected by the unknown quantities correcting means so as to decrease the difference between the cumulative addition value and the teacher pattern based upon the partial differential value corresponding to the function value output of the corresponding artificial neural network and the difference between the teacher pattern and the cumulative addition value of the function value outputs which are obtained by supplying the arbitrary quantities corresponding to the unknown inputs and the common known inputs as the input values to all artificial neural networks, and the quantities which decrease the differences to the least values are obtained as the estimated result by the unknown quantities correcting means. Then, the quantities corrected by the unknown quantities correcting means are taken out by the information taking out means. The taken out quantities are the unknown quantities which are objected for estimation.

More particularly, by carrying out the predetermined learning in each artificial neural network, a condition is obtained in which each artificial neural network performs a processing which is equivalent to a numerical formulae regulating a relationship between plural inputs and one output. Consequently, plural processings the number of which is determined in correspondence to kinds of the unknown quantities, which processings are equivalent to numerical formulae (processings and the like equivalent to plural numerical formulae which have different constants to one another, respectively) and processings equivalent to partial differential functions of the numerical formulae are possible to be carried out in corresponding artificial neural network. Thereafter, the output is obtained by carrying out corresponding processing in each artificial neural network, then the estimated value corresponding to to the measurement value is obtained by cumulatively adding the outputs of all artificial neural networks. Then, the difference between the estimated value and the measurement value is calculated. By correcting the unknown inputs of each artificial neural network based upon the calculated difference, the arbitrary quantities corresponding to the unknown inputs are varied so as to decrease the difference. When the varying processing of the unknown inputs is repetitively carried out, and the difference becomes sufficiently small, the corrected quantities are taken out as unknown quantities estimated result.

Brief Description of the Drawings

• Figure 1 is a block diagram illustrating an apparatus for estimating unknown quantities according to an embodiment of the present invention;
• Figure 2 is a schematic diagram illustrating a portion corresponding to one hierarchical perceptron in detail;
• Figure 3 is a diagram illustrating an example of an estimated error evaluating function; and
• Figure 4 is a schematic diagram illustrating a portion corresponding to one hierarchical perceptron of an apparatus for estimating unknown quantities according to another embodiment of the present invention in detail.

Best Modes for Executing The Invention

Referring to the attached drawings, we explain the present invention in detail.

Figure 1 is a block diagram illustrating an apparatus for estimating unknown quantities according to an embodiment of the present invention, while Figure 2 is a schematic diagram illustrating a portion corresponding to one hierarchical perceptron in detail.

The apparatus for estimating unknown quantities includes plural hierarchical perceptrons 1i (i=1, 2, ..., m), a sigma unit 2 for cumulatively adding function value outputs gij from the hierarchical perceptrons 1i, a function value error operating section 3 for receiving a cumulative addition result Oj(t) output from the sigma unit 2 and a measurment value Sj(t) as a teacher pattern, and for calculating a difference between the both, partial differential value calculating sections 1ik (k=1, 2, ...n) for calculating partial differential values corresponding to the function value outputs gij from the hierarchical perceptrons 1i, partial differential value error operating sections 3ik for receiving outputs from the partial differential value calculating sections 1ik of each hierarchical perceptron 1i and partial differential values (partial differential value teacher pattern) which are previously calculated by numerical differential method and the like, and for calculating a difference between the both, partial differential function learning sections 6i for carrying out learning of partial differential functions in corresponding hierarchical perceptron 1i based upon the difference calculated by the function value error operating section 3 and the differences calculated by the partial differential value error operating sections 3ik, correcting sections 12i for correcting inputs at an input layer of the hierarchical perceptron 1i based upon the difference between the cumulative addition result Oj(t) which is calculated by the function value error operating section 3 and the measurement value Sj(t) as the teacher pattern, and the partial differential values calculated by the partial differential value calculating sections 1ik, a control section 4 for selecting the correcting section 12i and the partial differential function learning section 6i and for repetitively operating the selected correcting section 12i and the psrtial differential function learning section 6i by a predetermined number of times (a number of times which decreases the difference to be sufficiently small), and a collecting unit 5 for outputting the inputs corresponding to unknown quantities of each hierarchical perceptron 1i under a condition that the correcting processing is repeated by a predetermined number of times by the correcting sections 12i.

Each hierarchical perceptron 1i is a hierarchical perceptron in which learning of the function is sufficiently carried out by supplying known input pattern and corresponding teacher pattern. The hierarchical perceptron 1i performs processing corresponding to the numerical formula which is determined based upon the input pattern and the teacher pattern. The numerical formula may not actually be expressed by a numerical formula, may be a numerical formula which requires much labor for expressing with a numerical formula. Of course, the numerical formula may be a numerical formula which is already expressed with a numerical formula. A number m of the hierarchical perceptron 1i is determined in correspondence to a number of unknown quantities which are objected for estimation. Further, the hierarchical perceptron 1i may be controlled to operate in synchronism, respectively. The hierarchical perceptron 1i may be controlled to operate in asynchronism.

The partial differential function learning section 6i alternately repeats learning (for example, back propagation learning) of the hierarchical perceptron 1i based upon the difference calculated by the function value error operating section 3 and learning of the hierarchical perceptron 1i based upon the difference calculated by the partial differential value error operating sections 3ik. The partial differential function learning section 6i performs learning of the partial differential function with slightly missing the learning result of the function.

Further, the correcting section 12i includes correcting sections 12ik the number of which is equal to the number of unknown quantities of corresponding artificial neural network 1i. The correcting section 12ik receives the difference calculated by the function value error operating section 3 and the difference calculated by the partial differential value operating section 1ik and corrects corresponding unknown quantity.

Operation of the apparatus for estimating unknown quantities having the above-mentioned arrangement is as follows.

Back propagation learning is performed in each artificial neural network 1i by supplying the known inputs such as measurement condition and the known inputs corresponding to unknown quantities to the input layer of each artificial neural network 1i and by supplying the known measurement value determined based upon these known inputs to the artificial neural network 1i as the teacher pattern, so that weighting factors and threshold values of each neuron device which constitutes the artificial neural network 1i are determined. When the back propagation learning is sufficiently performed, processing corresponding to a numerical formula which is not actually expressed with a numerical formula is determined in each artificial neural network 1i. That is, the output pattern gij of the artificial neural network 1i is expressed with the following equation.

$\text{gij = gi(t, ai1, ai2, · · ·, aiL) (1)}$

In the equation, t represents time, and ai1, ai2, ..., aiL represent unknown quantities.

After the back propagation learning (learning of the numerical formula) has been finished in each artificial neural network 1i in the above-mentioned manner, one of the known inputs corresponding to the unknown quantities among the input pattern is varied by a very small amount so as to obtain an output pattern and to obtain coresponding partial differential value by the partial differential value calculating section 1ik. Further, a partial differential value in a case that the known input corresponding to the unknown quantity is varied by a very small amount, is previously calculated by the numerical differential method and the like and is supplied as the teacher pattern. In this condition, the partial differential function learning section 6i is selected by the control section 4, and the selected partial differential function learning section 6i is repetitively operated by a predetermined number of times (number of times which decreases the difference to a sufficiently small amount) so that the back propagation learning is performed again. Thereby, the weighting factors and the threshold values of each neuron device which constitutes each artificial neural network 1i are determined so as to perform the processing which is equivalent to the operation of the equation (1) and an operation of a partial differential function of the equation (1).

Further, learning of each artificial neural network 1i may be performed individually. When plural artificial neural networks 1i perform processings equivalent to functions and partial differential functions which differ from one another only in their constants, the weighting factors and threshold values obtained by any artificial neural network as the learning result may be employed as they are in other artificial neural networks as their weighting factors and threshold values. Thereby, a required time for learning is extremely shortened. And, when the obtained weighting factors and threshold values are employed as they are, the learning of the function and the learning of the partial differential function are no need for the other artificial neural networks, thereby a required time for learning is extremely shortened.

After the necessary learnings (the learning of the function and the learning of the partial differential function) have been finished in the above-mentioned manner, each unknown quantity is estimated with high accuracy by selecting the correcting sections 12i instead of the partial differential function learning sections 6 and by operating the collecting section 5 by the control section 4.

When time t and known information such as measurement condition and the like are supplied to each artificial neural network 1i, each artificial neural network 1i outputs the predetermined output pattern gij(t). In this case, the cumulative addition value Oj(t) output from the sigma unit 2 is represented by the following equation.

   In the initial stage, though unknown quantities are arbitrarily determined, the obtained cumulative addition value Oj(t) is different from the actual measurement value Sj(t). Therefore, the difference between the measurement value Sj(t) and the cumulative addition value Oj(t) is calculated by the function value error operating section 3 as an estimated error di(t), and partial differential values of the cumulative addition values Oj(t) are calculated based upon an equation (3) by the partial differential value calculating sections 3ik.

   And, information for the unknown quantities are estimated with high accuracy by correcting the information for the unknown quantities other than the known inputs of each artificial neural network 1i based upon an equation (4) by the correcting sections 12i. Further, εk represents a learning gain (correcting gain) of the unknown quantity ai.

$\text{aik = aik + εk{Sj(t) - Oj(t)}[∂{gi(t,}$

$\text{ai1, ai2, ai3, · · · aiL)}/∂aik] (4)}$

   That is, when the correcing processings by the correcting sections 12i are repeated, the estimated errors di(t) decrease, and finally the estimated errors di(t) reach nearly 0. Therefore, an analizing result for the unknown quantities are obtained by collecting and outputting information for unknown quantities of each artificial neural network 1i at the timing by the collecting section 5.

Description is made in more detail in a point that estimation of unknown quantities is performed by repeating the processing in the equation (4).

When an estimated error evaluating function Ej(t) is defined by the following equation, an equation (5) is obtained.

$\text{Ej(t) = (1/2){Sj(t) - Oj(t)}²}$

$\text{∂Ej(t)/∂Oj(t) = - {Sj(t) - Oj(t)} (5)}$

When it is supposed that correcting of unknown quantities in each physical formula operating unit is performed based upon a maximum slope decreasing method, estimation of unknowns for minimizing a value of the estimated error evaluating function can be performed based upon the equation (6).

$\text{aik = aik - εk {∂Ej (t)/∂aik}}$

$\text{= aik - εk {(∂Ej (t)/∂Oj (t)} {∂Oj (t)/}$

$\text{∂aik} = aik + εk {Sj (t) - Oj (t)} {∂Oj (t)}$

$\text{/∂aik} (6)}$

   Figure 3 is a diagram illustrating an example of an estimated error evaluating function.

Estimating operation of unknowns which minimize a value of the estimated error evaluating function is described, together with Fig. 3 and a table 1 which indicates conditions of the estimated error evaluating function. In the chart 1, Δaik is a correction value according to the unknown quantity aik.

As being mentioned in the foregoing, it is sufficient that the estimating of unknown quantities is performed to decrease the value of the estimated error evaluating function. Therefore, the correction value Δaik is determined to be negative when a slope of the estimated error evaluating function is positive, and is determined to be positive when a slope of the estimated error evaluating function is negative, by taking a sign of the slope of the estimated error evaluating function Ej(t). It may be thought that an unknown quantity aik corresponding to a local minimum is obtained when the above-mentioned estimation of unknown quantities is performed, because Fig. 3 includes a local minimum point. The estimation processing is not performed for only one unknown quantity, but is performed for all unknown quantities in a synchronous manner, thereby the estimated error evaluating function itself is varied following repetition of the eatimation processing. As a result, unknown quantities which minimize the value of the estimated error evaluating function are obtained finally. Thereafter, estimation of unknown quantities can be performed by collecting and outputting the finally obtained unknown quantities by the collecting unit 5.

As is apparent from the foregoing description, when it is known that a constant relationship between cause and effect exist between an input pattern and an output pattern, and the relationship is not expressed with a numerical formula, the apparatus for estimating unknown quantities according to the embodiment obtains a condition which is equivalent to a condition that the relationship is expressed with a numerical formula by constructing the relationship between cause and effect which is not expressed by a numerical formula, in each artificial neural network 1i. Then, the apparatus for estimating unknown quantities performs estimating processing based upon the obtained condition. Therefore, unknown quantities are estimated with high accuracy in a condition that only the known measurement condition and the like and the measurement value are given, for example.

Further, when it is known that the relationship between cause and effect existing between the input pattern and the output pattern can be expressed with a formula, and that operating load becomes extremely great because the formula includes integration operation, convolution operation and the like, operation load is extremely decreased because a condition which is equivalent to a condition that the relationship is expressed by a formula, is obtained, and estimating processing is carried out based upon the obtained condition in each artificial neural network. Thereby, unknown quantities are estimated with high accuracy under a condition that only known measurement condition and the like and measurement value are given, for example. Consequently, directivity of a sensor and the like are estimated with high accuracy using the apparatus for estimating unknown quantities according to the embodiment, for example.

Second Embodiment

Figure 4 is a block diagram illustrating an apparatus for estimating unknown quantities according to another embodiment of the present invention.

The embodiment differs from the above-mentioned embodiment in that input varying sections 15ik for varying only one input by a very small amount, output holding sections 16ik for holding outputs before and after the varying of the one input, and difference calculating sections 17ik for calculating a difference between the both outputs are provided instead of the partial differential value calculating sections 1ik which are provided in each artificial neural network in the above-mentioned embodiment.

In this embodiment, variation in output corresponding to the variation of the one input by a very small amount is calculated without performing partial differential operation. Though the variation in output is equivalent to the partial differential value, function which is similar to that of the above-mentioned embodiment is performed. In this embodiment, a required time for learning is shortened in comparison with the above-mentioned embodiment, because learning of partial differential function is no need in this embodiment.

Possibility of Industrial Utilization

As is apparent from the foregoing, methods and apparatus for estimating unknown quantities according to the present invention, can calculate unknown quantities such as physical quantities and the like of a physical source with high accuracy and with extremely short time period, in comparison with conventional analyzing apparatus which use a supercomputer and the like, based upon measured values of a physical quantity, which values being obtained at plural points apart from the physical source such as a magnetic field source and others, and observation condition at the measurement timing. Further, the methods and apparatus extremely shorten a required time for learning in artificial neural networks.