Shusaku Tsumoto and Shoji Hirano
Department of Medical Informatics, Shimane University, School of Medicine, Japan

One of the most important problems in rule induction methods is how to estimate which method is the best to use in an applied domain. While some methods are useful in some domains, they are not useful in other domains. Therefore it is very difficult to choose one of these methods. For this purpose, we introduce multiple testing based on recursive iteration of resampling methodsfor rule-induction (MULT-RECITE-R). This method consistsoffourprocedures, which includesthe innerloop and the outer loop procedures. First, training samples(S₀) are randomly split into new training samples(S1)and test samples(T₁) using a resampling scheme. Second, S₁ are again split into training sample(S₂) and training samples(T₂) using the same resampling scheme. Rule induction methods are applied and predefined metrics are calculated. This second procedure, as the inner loop, is repeated for finite times estimated from inner precision preset by users. Then, third, rule induction methods are applied to S₁,and the metrics calculatedby T₁ are compared with those by T₂. If the metrics derived by T₂ predicts those by T₁, then we count it as a success. The second and third procedures, as the outer loop, are iterated for finite times estimated from the outer precision preset by users. Finally, fourth, the overall results are interpreted, and the best method is selected if the resampling scheme performs well.We apply this MULT-RECITER method to three newly collected medical databases and seven UCI databases. The results show that this method gives the best selection of estimation methods in almost the all cases.