I am having a very hard time in analysis of my features importance. What I am doing is I have 4 features and I incrementally add the features to see the effect of each feature on the final result. When I start with feature 1, for instance 300 out of 600 wanted data will be discovered. Then when I add feature 2, 100 out of 600 will be discovered and so on. However when I start with f2 then instead of f1 then 270 out of 600 will be discovered and this number for f1 will be for instance 200 out 600. So depending on which feature I incrementally add the importance of each feature varies and does not make sense. can any one suggest a beer way how I can analyze the imporance of feature in this case?
This is a question that should probably be posted in crossvalidated. Anyway, if you are familiar with R there are at least two approaches using random forest or variations of it (e.g. gradient boosting machine: this will give you a plot of variables importance just using summary on the model produced) are available to do variable importance.