Correct to a point.
The wire from the coil to the points is the other end of the primary winding.
So if the points are open, and your meter is very high impedance, then there will be very little current flow through the meter to ground or coil windings and the voltage measured will be close to your supplied voltage.
When the points close, the coil's rather low DC resistance will swamp the meter's very high DC resistance and you'll see a measurable voltage drop.
If it's not much drop at all there's a small chance the coil, the connections or actual wire is going bad.
Because under a very very light load and no current, there is very little voltage drop.
A bad wire, bad connection or primary wire might have high resistance and limit that current.
But if it is a large drop, the primary winding might be shorted across it's turns.
These are very basic trouble shooting guides as a coil gets really flaky when going bad vs its temperature!!