This paper studies the problem of learning a full range of pairwise affinities gained by integrating local grouping cues for spectral segmentation. The overall quality of the spectral segmentation depends mainly on the pairwise pixel affinities. By employing a semi-supervised learning technique, optimal affinities are learnt from the test image without iteration. We first construct a multi-layer graph with pixels and regions, generated by the mean shift algorithm, as nodes. By applying the semi-supervised learning strategy to this graph, we can estimate the intra- and inter-layer affinities between all pairs of nodes together. These pairwise affinities are then used to simultaneously cluster all pixel and region nodes into visually coherent groups across all layers in a single multi-layer framework of Normalized Cuts. Our algorithm provides high-quality segmentations with object details by directly incorporating the full range connections in the spectral framework. Since the full affinity matrix is defined by the inverse of a sparse matrix, its eigendecomposition is efficiently computed. The experimental results on Berkeley and MSRC image databases demonstrate the relevance and accuracy of our algorithm as compared to existing popular methods.