Plugin Ordering/Automation Framework

Originally posted by Marcus in Google Groups.

A thought occurred to me about the ordering in which plugins are currently being run.

Selection -> Validation -> Extraction -> Conform

Currently (mod: remember, this is an old post), this order is hard-coded. It is documented, it works and makes sense and there’s really no reason to change it. However we could make this programmable.

class SelectObjectSet(Selector):
    order = 0

class ValidateNamingConvention(Validator):
    order = 1

class ExtractAsMa(Extractor):
    order = 2

class ConformWithAsana(Conform):
    order = 3

This time, the behaviour is coming from the order attribute located on each plugin. To restore our currently hard-coded behaviour, all we have to do is query this number upon running a publish.

plugins = discover()
plugins_in_order = sorted(plugins, key=lambda p: p.order)

And viola, running each plugin in turn will have returned the ordering to how it was, without hard-code.

for plugin in plugins_in_order:
     plugin().process(context)

An automation framework

So this got me thinking. If we can control the order of how plugins are run, we’re opening up some new and exciting doors for Pyblish. In short, it would go from being a simple validation framework to an automation framework. Here’s how.

Consider shot 15 or Project X. In shot 15, there are 18 characters, 37 vehicles and a number of buildings and props taking part of the scenery. We know this, because it is in the shot recipe.

shot15_recipe.yaml

shot15:
    - John01: characters/John
    - Peter01: characters/Peter
    - Bush01: prop/Bush1
    - Bush02: prop/Bush1
    - MilitaryTank01: vehicle/MilitaryTank
    - ...

Alongside this recipe is metadata for each instance.

shot15_metadata.yaml

shot15:
    - John01:
        - position: [256, 0, 154]
        - rotation: [0, 0, 90]
        - scale: [1, 1, 1]
    - Peter01:
        - position: [240, 0, 141]
        - rotation: [0, 0, 90]
        - scale: [1, 1, 1]
    - ...
}

With this information, it isn’t too far of a stretch to imagine how this shot may look once fully set up. But can you imagine how much of a hassle it would be for the intern(s) assigned to the task of setting this up?

Well, with Pyblish this doesn’t have to be complicated at all.

import pipeline
import host

from pyblish import Builder

class BuildShot(Builder):
    order = 0

    def process_context(self, context):
        shot = pipeline.get_current_shot()
        recipe = pipeline.get_recipe(shot)
        metadata = pipeline.get_metadata(shot)

        for instance, asset in recipe.iteritems():
            # Reference each asset into the scene
            asset_path = pipeline.get_asset_path(asset)
            host.create_reference(asset_path, name=instance)

            for attribute, values in metadata.iteritems():
                # Position, rotate and scale each asset, according to its metadata
                host.set_attribute(instance, attribute, values)

            # Append instance to the context
            context.create_instance(name=instance)

There. This builder imports a limitless supply of assets into a scene per specification from its recipe and associated metadata. Once complete, we’ve got a couple of options.

Run validations upon the results, to make sure the build produced the expected results.
Conform, to log that this shot has been built by whom and when, for the record
Conform, if there were errors, report these to the nearest TD capable of doing something about it
Once the artist has modified the scene:

Validate, to make sure no one else has modified the same things since he last built

  • Extract new recipes
  • Extract updated metadata
  • Extract animation or pointcaches
  • Validate extracted animation

Disadvantages

This ordering is clearly less flexible than our proposed node-workflow could be.

There is no feedback on where a particular ordering-number fits in
There is no ordering of plugins with the same number such as having multiple validators all use order=1
However it is a step forward from the current fixed order we have today, and this is without changing the way it works currently. Each number could become part of the superclasses and thus only be overridden if needed.

class Selector(Plugin):
    order = 0