Dirt is not a problem, but pool filter sand is. What it feels like in your hand is not a reliable guide, I fell into this trap some years ago with a plant substrate (also not good with substrate fish, BTW). Here is a post (January 4, 2020) from Mark Wolek, a member of
CorydorasWorld. that covers this.
Let's talk sand for a moment. I see so many people recommend PFS for Corys which doesn't seem ideal to me.
Reef keepers with sand sifting fish won't use PFS because it is sharp edged and, while it may not kill the fish, it is likely uncomfortable for sharp edged sand to pass through soft gills. Reefers have argranite sand like Southdown available to them, which is very fine, powdery and soft, I have a whole trash can of the stuff from my reef, but it will buffer water, so it's out for Corys.
PFS works by having sharp edges which are used to catch dirt. Over time all of these sharp edges fill with collected material, wear down, and the sand needs to be replaced.
I know a LOT of people use PFS for their Corys, but is it really ideal? I don't think it is... A fish isn't going to stop sifting sand because of discomfort, it's engrained in their biology to do this.
Black Diamond "Sand" is basically coal slag and is also very sharp. The recommended size, 2040, is too large to sift. The finer 3060 sand may be siftable but looked pretty sharp to me, and also has a green tint I am not fond of.
In the wild, from the videos I've seen, I suspect they are actually sifting through mulm, very fine sand and mostly soil, which is probably incredibly soft (ever step in the bottom of a lake or non-rock bed river with bare feet? SO soft!)
So what is really the best sand for sifting fish, that won't affect pH or hardness? I am 100% not sold on PFS yet.
The most commonly used pool filter sand is #20 silica sand. It comes from ground quartz, which creates sharp silica grains with jagged edges excellent for trapping particulates from passing water."
https/www.swimuniversity.com/change-pool-filter-sand/
'Many years of forced water wears away the sharp edges of pool filter sand. Over time, the sand grains become more circular, losing their sharp edges, and their ability to trap small particles.'
https/www.poolcenter.com/sandPoolFilter