This is a basic ignition system, which is how yours is arranged, but ignore the wire colors:
https://i0.wp.com/www.ihcubcadet.com/forum/messages/106/29778.jpg?resize=640,361
As you can see, the condenser is connected to the points and coil, and is grounded, either through a wire to the frame/engine, or directly to the frame through its case when bolted down. When the key is on, the coil should have battery Voltage at both low-Voltage terminals, as will the condenser and points, assuming the points are open. When the points are closed, there should be 0V on the coil (-) terminal, condenser, and points; this allows current from the battery to flow through the coil, creating a magnetic field. When the points open, that current stops, and the magnetic field collapses. This change in field tries to keep the current flowing, but it can't, so creates a high Voltage in the secondary winding that is great enough to make a spark; it also creates a high(er) Voltage at the points, which spark too. The condenser reduces the strength of the spark at the points, so they last longer; it also slows the rise of the Voltage at the secondary, making for a longer duration spark through the plug.
You never answered if the 4V is with the points open or closed. Also, where did you measure that Voltage when you disconnected the condenser, at the coil or points?