I am a little confused about how a cyclic group can be infinite. To provide an example, look at $\langle 1\rangle$ under the binary operation of addition. You can never make any negative numbers with
sequences and series - What is the sum of an infinite resistor ladder ...
This resolves your problem because it shows that $\frac {1} {\epsilon}$ will be positive infinity or infinite infinity depending on the sign of the original infinitesimal, while division by zero is still undefined. This viewpoint helps account for all indeterminate forms as well, such as $\frac {0} {0}$.
There are the following textbooks to learn about infinite-dimensional manifolds: "The Convenient Setting of Global Analysis" by Andreas Kriegl and Peter W. Michor
functional analysis - What is a good textbook to learn about infinite ...
The dual space of an infinite-dimensional vector space is always strictly larger than the original space, so no to both questions. This was discussed on MO but I can't find the thread.
linear algebra - What can be said about the dual space of an infinite ...
So my question is how a basis can even exist for the infinite-dimensional case when the definition of a linear combination only makes sense for finitely many vectors and the basis in this case has an infinite number of elements by definition.
I would like to have some examples of infinite dimensional vector spaces that help me to break my habit of thinking of $\mathbb{R}^n$ when thinking about vector spaces.