FFXI Freezing And Unfreezing

Hi anyone know how I can fix this issue where the game only freeze and unfreeze when I'm attacking mob? This happens after being in any sort of congested zone like say Dyna-D or if I'm in Odyssey. I originally thought maybe it was frame drops but that does not seems to be the issue. This freeze only occur on FFXI it seems. Only way I have found to fix this issue is to restart the game or PC. Which is very annoying. Anyone have any idea I'm all ears.

Disable the Controller setting if u dont use controller.