How to calculate the performance (MAE, RMSE, NMAE) of Decision tree

shannoncates_te

Hi guysI'm a newbie here and I really need your help for my thesis. Our team is using a hybrid kind algorithm (Decision tree + item k-nn) that produces a recommender system.

So this is our process, it worked actually but the main problem is the result it only shows the ranking we want to know the performance result (MAE, RMSE, and NMAE) of the recommender system. Hope you can help me! thanks

lionelderkrikor

Hi@shannoncates_te

I will try to give you some response elements :

1. First to be sure : , the scoring metrics MAE, RMSE,NMAE are associated to the performance of a regression model (the predicted value is continuous).

In your cases, it seems that you are on a classification task (recommendation system) and you are using classification algorithm (k-NN / decision tree) and recommendation system. So to measure the performance of your models, you need to calculate the accuracy (ratio right predictions / total predictions), the recall, the precision and other one which are proposed by Rapidminer.

If you are on a regression task don't consider this paragraph 1.

2.To measure the performance of your model, you can perform a cross validation with thecross validationoperator associated to theperformance (regression)operator : In the parameters of this last operator, you have to check the score metrics you want to calculate.

Here you can find a simply process with a decision tree model inside a cross validation operator (to adapt to your own models/process):

<运营商激活= " true " class = "过程”兼容ibility="8.0.001" expanded="true" name="Process">
















































< portSpacing端口= " sink_result 3”温泉cing="0"/>

NB : you have to replaceperformance (classification)operator by theperformance (regression)operator in theright side

of thecross-validationsubprocess (testing part).

NB2 : In your first screenshot, i don't see any "prediction column".(results ofapply modeloperator)

I hope this wil be helpful,

Regards,

Lionel