Take your nearly finished garment to your favorite button store, and carefully slide a button (still attached to the card), into the buttonhole. If the button pops out, it's too small. The buttonhole should stretch slightly when you slide the button through it, and the button should completely cover the buttonhole's edges.
You can see that the top button (above) is a wee bit small for the hole, but the bottom one's just right.
Check your button for fit before finishing the yarn ends.
If the buttonhole is too large for your buttons, don't tear out all your work, just start at the end and sew across the hole to tighten it up. (We used contrasting yarn, but you'll want to use matching yarn).