Late spring is usually best as it's starting to warm up. I wouldn't do it now it's cooling down , as if they've been kept indoors in a room temperature tank, even with floating them in a bag in the pond the cold water could render then sluggish and less likely to eat. If the pond is in some way heated to stave off cold weather and to prevent ice forming, ( you can get heaters ) then moving them to such a pond would be no problem.
Optimum temp for goldfish is around 15-18 degrees C, so stick a thermometer in the pond, and if the temp is significantly lower than 15 degrees then keep them indoors till spring and just keep up with large regular water changes on the tank they are in.
I'd say no smaller than 3 inches if there are already large goldfish or Koi in the pond already, as they may get eaten or picked on, but if the pond is new or has no large fish then young goldfish can go in at 1 inch or so.
