Peonies and ants have a symbiotic relationship. The ants that protect them generally eat sweets. To be sure they arean't attracted to your camper, don't leave out sweets or any food for them to be attracted to. Make sure all crumbs are cleaned up. Also, don't disturb the ant colony by trying to kill them. When that happens they feel threatened and often start several colonies nearby.
Enjoy your peonies and don't forget to add lots of compost to the planting bed. They'll love you for it.
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.