I turn mine off. And, just as a reminder, along with the heater! if the heater is placed in a way that some of it is out of the water during the water change.
For one thing, about half the time I'm also rinsing off a filter cartridge at the same time, and removing cartridges while the filter is running makes an unnecessary mess in the tank.
For another, if you leave the filter running, the gunk you stir up will clog your filter cartridge a bit more.
And third, with a HOB filter, like your Penguin, you're going to have a pretty substantial waterfall if you leave it on while doing a, say, 25% water change. That may not be a problem in a tank with no live plants, couldn't say.
When I do water changes, the filter is usually off for 10-30 minutes (occasionally a bit longer), and I've never had any measurable "mini-cycles" after a water change.