You will need to decrease the watering a bit and let the leaves turn yellow before removing them. The leaves are what feed the bulbs for the next seasons blooms. Most folks just dispose of forced bulbs, but you can plant them in the garden and see if they sprout next year. They may never bloom again or could take a couple of years before you see any flowers.
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.