Yah, too much slop in the shifter & it won't move the cables as needed. No synchro for reverse, so that can't be an issue...
Sounds like the boot is off so you can see the mechanism. You'll notice that front to back movement of the lever moves one cable, and there are mechanisms to move the other with side to side movement. Cable moves one way when pushed left, then another way when pushed right, off of the center position. Sounds like you're not getting that second cable (L&R) to move as needed because of the broken pieces...
"Rubber Side Down"