The ammonia would be from the dirt.
In a sense, yes. The issue with dirt is primarily the organics. As they decompose, which they do very rapidly, CO2 and ammonia is produced. The ammonia is so great the plants cannot handle it, and it will kill the fish. This is why many "dirt" substrate experts recommend a dry start and no fish for up to six months. By that time, the dirt has already lost over half its value. So most of us ask, why bother?
You mentioned in another post the amazing plant growth from dirted tanks. This is a bit misleading. The one and only benefit to soil/dirt is the initial level of CO2 occurring from the decomposition. In new tanks, CO2 is usually limited, and as the organics accumulate over the first few months in the substrate, the decomposition increases and CO2 becomes more prevalent. Even the soil substrate experts like Diana Walstad will admit than after one year, all else being equal, a tank with dirt and a tank with inert sand will both be equal in their plant growth capability.
So, putting all the above together, if you use soil/dirt you have no fish for six months, you cannot have digging fish like many catfish, and by the time the tank is established...the same tank with sand would have been running for six-plus months and continue running with the same plant growth.
It's your tank to do with as you like, but we are simply pointing out that there really is no "benefit" with soil when you consider the negatives, and in the end nothing is any better anyway.