Contextual methods are increasing in popularity among methods for remote sensing image classification. In particular, Markov random fields (MRF) have the ability to incorporate spatial, temporal, and other contexts into the classification procedure, providing increases in accuracy and the visual coherence of classification results. However, performance of both the base classifier and its associated spectral energy estimation technique greatly affects classification results. This paper, therefore, compares the effects of some parametric and non-parametric methods of spectral energy estimation for a spatial MRF model in order to understand the importance of selecting high performing techniques. Effects are evaluated on a range of datasets, including hyperspectral, multispectral, and high resolution data, considering variation in training set size. By examining both the results and assumptions of these techniques with respect to different types of remote sensing imagery, recommendations can be made for the operational use of a class of MRF models.