When Design Rules Break: Benchmark Composition Determines Whether Label Informativeness Predicts GNN Aggregator Choice

A new study finds that widely used design rules for graph neural networks fail to generalize when tested across different benchmark families, with the composition of datasets — not numerical insufficiency — determining whether performance patterns hold [1][2]. The research examines aggregator selection — choosing between sum, mean, and max operations — across 24 node-classification datasets spanning citation networks, heterophilic graphs, co-purchase, co-authorship, and the Facebook-100 collection of dense friendship networks [1][2]. Edge homophily, a common measure of how often connected nodes share labels, proved only weakly predictive of the performance gap between GIN-Sum and GIN-Mean models [2]. Label informativeness, which measures how much a node's features reveal about its label, predicted the aggregator performance gap well on legacy benchmarks. However, that relationship degraded substantially when Facebook-100 graphs were included in the analysis [2]. In the Facebook-100 graphs, near-zero label informativeness coexisted with a strong preference for sum aggregation, yielding performance gains of 7-10% and up to 13% under extended training [2]. The effect was localized to 1-hop neighborhoods and replicated across different GNN architectures [2]. Stochastic block model ablations, including degree-corrected variants that matched Facebook-100 degree scales, failed to reproduce this behavior, indicating that mean degree alone does not explain the phenomenon [2]. Among several label-independent graph statistics examined, the spectral gap uniquely distinguished the Facebook-100 graphs from other low-informativeness datasets [2]. The researchers also identified training regimes that interact with aggregator choice and demonstrated that PNA, a principal neighborhood aggregation architecture, can underperform the best single-aggregator GIN on standard citation benchmarks [2]. The findings suggest that benchmark composition determines whether design rules appear to generalize, and the Facebook-100 regime provides a concrete target for future adaptive aggregation methods [2].