Let $X_1,...,X_n\stackrel{iid}{\sim} \mu$ where has a density with respect to the Lebesgue measure on $\mathbb{R}$: $\mu(dx)=\rho(x)dx$. For every $x$ show$$\lim_{n\to\infty}\sum_{i=1}^n \exp\left(\frac{-|x-X_i|^2}{2(\sigma/n)^2}\right)=\rho(x)\sigma\sqrt{2\pi}.$$
I am very confident the result is true as it has been empirically verified. I arrived at this conjecture because of the following formal argument:\begin{align}\sum_{i=1}^n \exp\left(\frac{-|x-X_i|^2}{2(\sigma/n)^2}\right)&\approx n \int\exp\left(\frac{-|x-y|^2}{2(\sigma/n)^2}\right)\rho(y)dy\\&=\sigma\sqrt{2\pi}\left[\int\frac{n}{\sigma\sqrt{2\pi}}\exp\left(\frac{-|y|^2}{2(\sigma/n)^2}\right)\rho(x+y)dy\right]\\&\xrightarrow{n\to\infty}\sigma\sqrt{2\pi}\int\delta(y)\rho(x+y)dy=\sigma\sqrt{2\pi}\rho(x)\end{align}where I have used the law of large numbers (this is really the part where I am not being precise), the weak convergence of the gaussian to the Dirac delta, and finally the definition of the Dirac delta.
