Sounds like a good guess, the cleaner the aerodynamic "break" at the tail of the car - the better the Cd (coefficient of drag).
Downside you've found, more crud that finds it's way into the 'burble" gets deposited on the tail since there are fewer random air currents to blow it away.
Unless there's a cloud of smoke following you, sounds like the "White Car Blues" of all the dirt showing - Just like my "Black Car" version where every spec shows up.
I never understood why some folks say that white doesn't show the dirt as much. I can believe the right shade of tan, but white??? (grin)