How to combine two machine learning algorithm outputs?

Question

If I have two very different datasets and two very different classification techniques, is there a good way to combine the two outputs? I understand an average may work but is there a more relevant way to do this? I've heard of several concepts like boosting and ensemble learning, would these be applicable?

Answer 1

There are two general ways to go about this problem. The first, called boosting, uses weighted voting to decide on the prediction. The main idea is to combine advantages of both classifiers.

The second approach, called stacking, uses the outputs of the two classifiers as features into another classifier (possibly with other features, e.g. the original ones), and use the output of the final classifier for the prediction.

In the absence of further details, this is the best answer I can give.

See Bagging, boosting and stacking in machine learning on Stats.SE for more.