What METAL-MLEE Does

The main purpose of METAL-MLEE is to obtain meta-data, i.e. data about the performance of learning algorithms on different databases (these databases that are used to gather performance data for the learning algorithms are also called base-databases to distinguish them from the databases of meta-data obtained in the process). The term performance of an algorithm includes such measurable properties as estimated error on unseen data, CPU time needed for the training and the evaluation phase, and complexity of learned model.

METAL-MLEE can be used to:

• Check if the format of a base-database conforms to the standard database format (see Section 4).
• Carry out an error estimation experiment to obtain error estimates for one or more learning algorithms for a base-database. Error estimation strategies include crossvalidation, holdout estimates and leave-one-out.
• Obtain data characteristics for a base-database
• Obtain additional database measurements for a base-database.
• Calculate error estimates and other statistical measures to describe the performance of the learning algorithm and statistically compare different learning algorithms.
• Document the details of an experimentation run for future reference.
• Extract a meta database from the output files created for each experiment.
• Manage and make comparable experiments that were carried out on different machines.
• Normalize CPU time measurements for measurements obtained on different machines.

For use with the data mining advisor ((cite: advisor stuff)), METAL-MLEE is used in a standardized way. This is described in Section 7.