Having lived where you are, I would suggest you make some type of cold frame to set your pots in until all threat of frost has passed. Here's a map that gives the approximate last frost dates.
Here's another site as I find the colors in that one hard to read in the lighter blues. Just click on your state.
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.