Hi Somna,
Congratulations on your new home!* You might want to use a pruning saw to cut the roots as close to the concrete as you can and then pull them out.* Cutting the roots in sections might work, even if you can't cut all the way through.* That will help them to rot or make it easier to pull out.*
Another option would be to see if you can rent one of these.
http://www.weedwrench.com/index.htm
http://www.misterhoneysuckle.com/
If you find you need to purchase one and can't rent it, and won't have any further need for it, consider donating it to a local native plant society or garden group.
Newt
When weeding, the best way to make sure you are removing a weed and not a valuable plant is to pull on it. If it comes out of the ground easily, it is a valuable plant.