IIRC Hawk had the same problem; also Ichiro has reported this before.
I couldn't work out what the problem was, and concluded that the best thing to do would be to log timing info of various DirectX functions that gfx_directx calls and other debug info to try to narrow it down.
Haven't touched gfx_directx in a long time, but I need to do some updates and bugfixing on it soon anyway.
Edit: Or I should figure out whether it's possible to distribute something like
PIX instead.