There are several possible reasons for this. If the soil is compacted, if the soil hasn't enough organic matter for nutrients, if the bulbs aren't mature enough or drought during the fall. You could topdress with compost or scratch in some bulb food and then topdress. Leave the foilage to yellow before removing so the leaves can feed the bulbs for next years bloom. If you can remember where they are located in the fall, you can scratch in more fertilizer then.
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.