Hi Ill Bosstino,
How deep to dig is difficult to say.* My suggestion would be to dig down until you no longer see any debris.* Then dig out another 12 inches of earth.* All this will need to be properly disposed of as it will most likely contain heavy metals and other toxic chemicals.* Contact your local landfill and ask about how to dispose of the soil and debris.* Then contact your local extension service if you are in the US and have a soil analysis done for toxic chemicals and heavy metals.
Another option would be to dig down as I have mentioned and build a raised bed for your veggies and fruits.* Add enough soil so the new soil is 3 feet deep.* I would suggest a mix of 60% screened topsoil and 40% aged compost.* Your veggies will love you for it.
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.