Plugin data converters

[hannes]

proposal: converters

pros:

• more plugin compatibility helps with shared plugins and a plugin marketplace

cons:

• would require adding/changing the pyblish base architecture.
• unsure if easy from a technical POV
  it seems easy on first sight, but afraid i overlooked something and it would snowball

it’d be easier to only allow plugins of 1 type
but sharing plugins between a cmds and pymel user would be a pain.
though the enduser can always make their own collector plugin that converts the instances…

proposal:

register converters, which handle the type translation.

1. register a converter
2. register 2 incompatible plugins.
3. before a validator plugin runs on an instance colelcted by the first plugin,

• it checks if it’s compatible.
• if not, it checks if any suitable converters are registered.
• if not, it fails

example

a collector collects instances with meshname
a validator requires a pymel mesh, and would error it tries to do instance.verts

before we run def process(self, instance):
pyblish would check for compatibility between plugins.
notice they are not compatible. and see if any suitable converters are registered.

the converter would take all data in the instance, and translate it. (and cache it for future validator plugins)

goal

the goal is to make collectors and validators written in different languages compatible.
pymel, openmaya, cmds cmdx …

converters can be shared on the “marketplace”, just like plugins

[hannes]

taking the advice of our previous discussion: minimise pyblish core code changes.

this can be a custom plugin instead.
ideally pyblish supports dependencies for plugins.

this is low priority for me so will get back to this when the other features are more fleshed out.

[marcus]

this can be a custom plugin instead.

From what I gather, this seems like an appropriate solution to this problem.

[hannes]

continuing convo here from Pyblish workflow manifest

Pyblish workflow manifest

I’m cautiously optimisting about the idea of “converters”, it does seem like a steep hill to climb. Is node paths the only thing of interest? What about PyMEL attributes or data types like matrices etc? I’ve even heard of some using the MASH Python library, which is apparently a thing.

Pyblish workflow manifest

that’s why for now converters are plugins that can be made be the user.
want to support a new datatype? create your own converter
we can have a few basic ones for most common scenarios. now we support 90% of all plugins. the user can make the remaining ones themselves

easy ones for maya

• meshname -> pymel mesh
• transform -> shape
• meshname -> dagpath
• short -> long meshname

if there are any interesting checks on github that are worth creating plugins for let me know. would be funto test weird things such as MASH. but think the boring basics take priority for now.

most stuff i find is either meshname(cmds) openmaya and pymel which is easy to convert to each other.

[hannes]

option 1

support events / callbacks before a plugin/action is about to process on an instance.
then we can subscribe to this, and somehow capture the instance and convert it.
something similar to the filter-plugins setup

• a decorator around the process function allows you to do whatever
  convert the instance to your fav type
• but how do you manage each individual plugins convertors

option 2

after you ran your collectors, run a convertor plugin that collects the instance as a new instance

CONS:

• creates a lot of instances and results in a messy workflow
• doesn’t support collecting instances afterwards, relies too much on order

option 3

every plugin has a input-type, and output-type if it creates instances.
pyblish-base lets you register convertors, just like plugins, hosts, gui
if a plugin runs, it gets the instance, then before it runs, it asks for the type of the isntance.
if this does not match the input type, we check our registered convertors.
if there is a matching one, it can be converted, else error like usual.

CONS:

• needs extra functionality added to pyblish-base
• seems like the most work, but can’t see why this wouldn’t work

[hannes]

found a solution that works as a collector / plugin based approach.
it creates a single instance instead of 1 instance for each instance of another type
then that instance collects all other instances when it’s iter function is accessed.
it seems to work fine

• test code registering a collector and validator with 2 different families.
• convertor plugin sample string to int

class ConvertorInstance(pyblish.plugin.Instance):

    def __init__(self, converter_function, input_families, *args, **kwargs):
        """
        input_families: the family to find instances that need converting
        ex. pymesh

        family/families: the family used by validators to find the converted instances
        ex. meshname

        :param converter_function: function to convert the instance, can be a generator or anything iterable.
        """
        self.converter_function = converter_function
        self.input_families = input_families
        super(ConvertorInstance, self).__init__(*args, **kwargs)

    def __iter__(self):
        context = self._parent  # does not yet support instances parented to other instances, only parented to context
        mock_plugin = pyblish.api.Plugin
        mock_plugin.families = self.input_families
        original_instances = pyblish.logic.instances_by_plugin(context, mock_plugin)

        self.converted_instances = self.converter_function(original_instances)

        for instance in self.converted_instances:
            yield instance

current issues

when testing with the GUI it is unclear which instance failed the validation for now

same with clarity on which instance was successfull.

i think we can get around this if we register convertors in pyblish, just like how we register paths, plugins etc.
and dont treat convertors as standard plugins.