6-32 tap and set screw work well, too. Drill first, with the correct drill bit - #35 or #36 work equally well in this role. Larger diameter, larger allen (not brittle, like the tinyass allen that fits a 4-40, I think it's 0.050"). More meat, coarser threads, easier to work with. I thread all mine with 6-32, unless they were already threaded for 4-40.
EDIT - 6-32 set screws are a hell of alot easier to find, too. <thumbsup>