Definition:$\lim_\limits{\large x\ \to\ x_0 \atop \large x\ \in\ E}f(x) = L$ iff for every $\epsilon > 0$, there exists a $\delta > 0$ such that $\vert f(x) - L \vert \leq \epsilon$ for all $x \in E$ such that $\vert x - x_0 \vert < \delta$
Does the part "$x \in E$" under "$\lim$" mean that any $x \in E$ can be made to approach $x_0$ and the associated limit will be defined. Hear me out, my confusion stems from $\delta$ being able to constrain the scope in such a way that for all the $\{x \in E: |x - x_0| < \delta\}$, limit will be defined, even if some $x_0$'s in E's would produce undefined limits.