The diversity of opinions is both cure and curse for the effective use of crowd-sourced intelligence. To unify crowd-sourced intelligence for a sensible decision, we develop an algorithmic approach for decision aggregation by properly quantifying reliability of information from multiple sources. The key idea is to model reliability propagation of decisions on an ensemble of relevance graphs, where the optimizations of reliability propagation and graph ensemble are mutually reinforced. The propagated reliability can be used to aggregate intelligence from multiple sources and facilitate decision-making by leveraging various types of inter-correlations of information sources as well as information subjects. Meanwhile, the optimized graph ensemble can retain the relevance structures with respect to the crowd-sourced intelligence. We show that, when aggregating analysts' recommendations in stock markets, our approach is effective and superior compared to alternative methods.
