I had this occurring and it appeared random. Then one time it lasted long enough for me to get out of the car and listen, which I heard sucking air. The intake tube was rubbing on the PCV valve hose (on top of the valve cover driver's side) and wore a hole in it (vacuum leak). When I plugged it with my finger rough idle went away, unplugged it and back it came. So you can try looking for vac leaks...
Edit: I should read the whole post, you've already checked for vac leaks. Donnyd21's answer sounds pretty plausible in that case.