Proving that the set of limit points of a set is closed

From Rudin's Principles of Mathematical Analysis (Chapter 2, Exercise 6)

Let $E'$ be the set of all limit points of a set $E$. Prove that $E'$ is closed.

I think I got it but my argument is a bit hand wavy:

If $x$ is a limit point of $E'$, then every neighborhood of $x$ contains some $y\in E'$, and every neighborhood of $y$ contains some $z\in E$. Therefore every neighborhood of $x$ contains some $z\in E$, and so $x$ is a limit point of $E$. Then $x\in E'$, so $E'$ is closed.

The thing that's bugging me is the leap from one neighborhood to another. Is this formally correct?