Use Application.MacroOptions
This example adds a user-defined macro called TestMacro to a custom category named My Custom Category.
Function TestMacro()
MsgBox ActiveWorkbook.Name
End Function
Sub AddUDFToCustomCategory()
Application.MacroOptions Macro:="TestMacro", Category:="My Custom Category"
End Sub