In this talk, we revisit the diffusion approximation of Stochastic Gradient Descent, which gives us some insight of the behaviors of SGD. Usually, diffusion approximation provides weak approximation only in a finite time horizon. Motivated by the backward error analysis of numerical SDE, we explore the long time weak approximation of SGD in the framework of diffusion approximation for strongly convex objective functions. Our analysis builds upon a truncated formal power expansion of the solution of a Kolmogorov equation arising from diffusion approximation, where the main technical ingredient is uniform-in-time bounds controlling the long-term behavior of the expansion coefficient functions near the global minimum. We hope this work will motivate more research covering deeper aspects of stochastic optimization algorithms.