You likely have wear in one or more of your compensator feed circuit components, which since it seems to be isolated to the 3-2 downshift, would be limited to that circuit...That said, there's still more than enough potential causes...
The specific data points you'd want to monitor are TCC Slip Speed, TCC Duty Cycle and TCC command (on/off) as well as shift speed and gear selection (ie what gear you are in per the system vs reality) for all gear transitions (up and down shifts)...A fast adapt may help but I'm guessing, it will only be temporary as whatever's worn (can be multiple things) will overcome the TEHCM's adaptation efforts and your symptoms will return.
I'm going to be doing a video on the compensator feed system in those transmissions and publish to my Youtube channel in early May. There's almost no information about this particular system, how it works and the role it plays in the function of the transmission, which is somewhat surprising since it's so critical and important to the overall health of the unit.
Anyways, fixes include everything from servicing the TEHCM (sometimes clogged pressure switch diaphragms or worn seals cause clunky shifts) to rebuilding the entire transmission - all depends on what's worn and the extent of the wear/damage.