We propose necessary conditions and show that, for sparsely but strongly connected heterogeneous networks, in the presence of global external inputs, there is a small active subnetwork (active core) inherently embedded within it. Surprisingly, although the whole network is heterogeneous and unbalanced, the active core possesses a balanced state and its connectivity structure is close to a homogeneous Erdös-Rényi network. The small size of the active core embedded in a large network also suggests an emergent network mechanism for sparse coding in neuronal systems.