Pyblish tries to do it for you, but prioritizes being clear on the order of processing the plug-ins.
They are actually combined (visually under a header) only when "processed one after the other". Since Pyblish' order of processing for plug-ins is dependent of the order they are added in the context.
In your scenario you have two collectors collecting instances of family rig that run with the same order as other plug-ins. The same order means it's somewhat undefined which must run first or last and as such no specific order is assumed. This means your two rigging collectors might not run right after each other, another collector could be running in-between. That's exactly what's happening.
- set1 (rig)
- MyInstance (default)
- Maya_Instance (animate)
- RigInstance (rig)
And this is stored in the context (as a list).
So you would have (somewhat simplified example here):
context = ['set1', 'MyInstance', 'Maya_Instance', 'RigInstance']
In this order even though
RigInstance are both of family
rig they cannot be grouped together because they don't run one after the other. So @marcus solution is to "sort" the instances by family so they can always be grouped together.
Basically it is you taking control and telling it the order you prefer.
Theoretically you can sort it in any way you like and take full control over the order of the plug-ins if you need to. Again it's something that Pyblish doesn't try to make too much assumptions about.
Also see @tokejepsen link:
I think this only happens with Plug-ins that conflict by the name of the class.