The stationary discontinuity induced by the singular source and its coupling with convection of fluid present challenges to numerical computation. We theoretically show that a splitting scheme is always not well-balanced and leads to incorrect results. For an unsplitting scheme, we present a consistency condition of the numerical fluxes for the singular source, which ensures the numerical scheme to be well-balanced. However, it can be shown that the well-balanced property of a scheme cannot guarantee the correct numerical solutions in extreme cases. To fix such difficulties, we propose a solution-structure based approximate Riemann solver. The proposed solver can be applied to construct the numerical flux in a general finite volume method, which can lead to an advanced well-balanced scheme. Numerical tests show that the discontinuous Galerkin method based on the present approximate Riemann solver has the ability to capture each wave accurately.