I have had my 3 african dwarf frogs for a couple of years or so, now. They live in a small unheated tank. I had them with a betta for most of that time, but the betta was old and finally died earlier this year. Right now there is a very small black moor in there with them and they seem to be just as happy with it, as they were with the betta.
I've had frogs in heated and unheated tanks over the last few years and have had great success with each. These ones started in a heated community tank, but I moved them to my small tank as I thought they'd have a better chance of eating properly without the competition.
Dwarf frogs don't eat as fast as fish - so make sure that you put in a sinking food that sinks quickly - goldfish are piggy and they snap up a lot of it before it can get to the frogs. That, I would say, is the only drawback to having them with the goldfish.