Crypts(especially) and Swords dislike having their roots disturbed. Vals will also often lose leaves if they are bruised. Java Ferns, Anubias, Java Moss prefer to be attached to wood or rocks and so don't want their roots in the substrate anyway and are fine being moved. So it varies by species. If you decide you don't want to disturb them, that's fine and you can jiggle the shallow gravel near them and or even just wave the cylinder near them and get most of the debris. You should still deep clean any open gravel areas that you can.
As more of your tank becomes covered with plants you don't want to disturb you can make up for some (not all) of the lost gravel-cleaning by making your water changing a bit more frequent if possible. You can also sometimes disturb substrate closer to plants by using a thin stick prior to staring your gravel cleaning process.
~~waterdrop~~