Just "scrubbing" the driftwood isn't enough to prepare it for the aquarium.
I've got driftwood in mine, and it isn't turning my water brown because I prepped it first. Here's what I did.
I put the driftwood in a pot of water and boiled it for half an hour. Poured the water down the drain, put fresh water in, boiled again. I did this three or four times.
After boiling it, I soaked it in dechlorinated water for I think a week, changing the water completely every 2 days.
If you JUST soak it, it will take longer. You could alternatively just boil it over and over until the water you pour out of the pot is clear, but I'm not sure how many times boiling that would take.
After the driftwood stops leeching color into the water, then it's safe to put into your tank.