I'm confused as to how it's currently held on, pictures please!

Are you just sliding the boss over the gearbox shaft thread, and then using the reverse thread screw to hold a washer against it?

If you want a quick fix I'd drill down through the boss and shaft (when in situ) as big as you can, and then drop through a bolt (same size as the hole). That way it's pinned in place laterally and radially. (It's not going to fall off or turn)

If you have the time/tools, I'd use a high tensile bolt, thread lock on the other end of the nut and tack weld points.

If you can't do that, take lots of bolts with you!