Traditional machine learning and statistical modeling methodologies are rooted in a fundamental assumption: that both training and test data originate from the same underlying distribution. However, the practical reality often presents a challenge, as training and test distributions frequently manifest discrepancies or biases. In this work, we study covariate shift, a type of distribution mismatches, in the context of deep nonparametric regression. We thus formulate a two-stage pre-training reweighted framework relying on deep ReLU neural networks. Specifically, in the first pre-training stage, unlabeled data from both the source and target distributions is utilized to estimate the density ratio through deep logistic regression. Following this, a density ratio reweighting strategy is seamlessly integrated into deep nonparametric regression, incorporating the previously estimated density ratio. We rigorously establish convergence rates for the unweighted, reweighted, and pre-training reweighted estimators, illuminating the pivotal role played by the density-ratio reweighting strategy. Additionally, our analysis illustrates the efficacy of pre-training and provides valuable insights for practitioners, offering guidance for the judicious selection of the number of pre-training samples.