Pyblish Order Issue

Support
#1

Hi

This ones a long shot. I’m having an Issue where my plugins are not executing in the correct order.
I’ve have two extraction plugins (in separate modules,) one of their orders is a bit higher than the other.

for a in plugins: print a.name, a.order
#plugins.sort(key = lambda x: x.order)
pyblish.util.extract(context=context, plugins=plugins)

the print out from the loop before extraction:

Collect Renders 0
Nuke Render Exists 1
Copy Renders 2.1
Publish Renders 3.15
Submit Deadline Job 3.2
Cleanup Renders 3.4
Collect Script 0
Write Node Exists 1
Nuke Script Exists 1
Non "Lemonade" Write Nodes 1
Save Script File 1.9
Publish Script File 3
Cleanup Script File 3.4

The problem is, that it is running the “Copy Renders” before the “Save Script File”. Interestingly if I sort the plugins list by order (see commented out code) it runs them in the correct order.

Its really odd, so I went and created just a simple bit of test code and couldn’t replicate it.
The test code excluding plugins (following code bellow), is similar in its approach to how I use it in my main code:

pyblish.api.register_plugin_path(pluginfolder)

plugins = pyblish.api.discover()

import pyblish.util
context = pyblish.util.collect(plugins=plugins)
context = pyblish.util.validate(context=context,plugins=plugins)
context = pyblish.util.extract(context=context,plugins=plugins)
context = pyblish.util.integrate(context=context,plugins=plugins)

As I said this is a long shot as I haven’t provided all the code, but if any one has even the slightest idea about what might potentially be going wrong it might be very helpful. In the mean time I shall just try and break down my code bit by bit.

thanks

Phil

#2

A snippet of the plugins, for the record they are sitting in separate modules.

class ExtractRenderFile(pyblish.api.InstancePlugin):
    order = pyblish.api.ExtractorOrder + 0.1
    families = ["renders"]
    name = "Copy Renders"

    def process(self, instance):
        self.log.info("Copying Renders")
        pyblish.api.emit("pluginStarted", instance=instance, plugin=self)
        pc.skip_plugin(self.log,instance)
        # now do the saving bit ...

class ExtractScriptFile(pyblish.api.InstancePlugin):
    order = pyblish.api.ExtractorOrder
    families = ["script"]
    name = "Save Script File"

    def process(self, instance):
        self.log.info("Saving script file")
        pyblish.api.emit("pluginStarted", instance=instance, plugin=self)

        pc.skip_plugin(self.log, instance)
#3

Can’t tell exactly what’s going on, but here’s a debugging tip.

Give this a go with your plug-ins, and see if you’re getting the expected results.

from pyblish import api
plugins = api.discover()
print(plugins)

That should you the order used in GUIs and util.py.

With a series of plug-ins, you can put them through the sort function used by discover().

import pyblish.plugins

class ExtractRenderFile(...):
  pass

class ExtractScriptFile(...):
  pass

plugins = [ExtractRenderFile, ExtractScriptFile]
pyblish.plugins.sort(plugins)

print(plugins)

This will also give you the plug-ins in question with the order they should be having when publishing from anywhere.

#4

Ahh I think I’ve got to the bottom of, thank you very much.

I assumed that calling the functions publish, extract or the like, would sort the plugins? I didn’t realise that when you passed a list of plugins the order of the list was important, and It was up to me to sort.

In my code I was using discover to find the plugins, but I was also doing some filtering of the plugins myself before using them, and I wasn’t maintaining the correct order whilst doing so.

If I am correct in saying that, when you pass plugins to any of the CVEI convenience functions, the function doesn’t take care of the ordering and assumes you have passed it in the correct order, would it be possible to make them resort, or is there a good reason why they shouldn’t?

#5

That is correct; discover() provides a order, governed by the MyPlugin.order attribute, but this attribute is merely a hint. All implementations use discover(), so the order is consistent so long as none of them goes rouge.

The idea is to enable users such as yourself to take control of this order for whatever reason, and GUIs to provide a visual way of rearranging plug-ins, based on whichever constraints the implementor put in place. For example, it might not be a good idea to allow a user to shuffle extractors in front of collectors. But it might make sense to move a quick validator in front of a long-running one, if the artist knows this is where things typically go south and don’t want to wait around for the long-running one to finish each time.

Perhaps it’s worth considering whether the util.collect() and c/o should forcefully reorder, as maybe they won’t work at all when it isn’t exactly right, and since they are top-level convenience functions they might not need to be as flexible.

Let me know your thoughts.

#6

OK Fair enough, that makes sense.

I think from my point of view, it was just a lack of knowledge. I just assumed it would sort it, as a precaution, because I didn’t think there would ever be a case where you would want it to run in an order that wasn’t dictated by the plugins order.

Allowing the user to reorder certainly would make things more difficult to account for, and the user might not know what orders might be invalid\likely to error if there is dependency on context instance data changes. So it would be up to me to implement it in a way in which he could safely reorder.

That said just because I don’t want that functionality doesn’t mean that others wont. I would prefer if the CVEI functions did sort and I didn’t have to, but perhaps the solution is just making this behaviour more visible in the documentation, and perhaps it is but I didn’t realise the importance of it when I was reading it. :slight_smile:

thanks
Phil

#7

Sure, let’s keep it in mind for now and if in the future anyone else struggles with this same assumption, then we can choose to move forward.

Thanks for bringing this up, Philip.