Wow, that's a pretty good looking shrub for one that was neglected. It does look a little lopsided.* They are decicuous and the leaves turn nice colors in the fall if they get enough sun.* They bloom on old wood (previous year) so if you prune it for shape you may not have blooms on those branches next year.* I'd wait until the leaves drop to prune.* You can remove spent blooms at any time.* Here's some info on them.
Prune them using method 1 here.
Btw, it's best to rejuvinate them by pruning 1/3 of the oldest branches to the ground each year for 3 years.* That way you'll still have flowers.* That's the same way to rejuvinate viburnums and lilacs.
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.