Good to know about Pyblish

marcus · August 19, 2015, 11:00pm

Got a few questions and insight about Pyblish recently and thought I’d share them along with answers.

Q: Can I use Pyblish for just validation?

Yes. But…

Pyblish couples validation with writing to disk for a very important reason; to ensure that everything written is valid.

This also means that nothing can get written to disk even when you really want to, such as during that tight deadline when things break last minute.

This is intentional.

The philosophy is fail fast. In other words, that it’s important to point out faults as soon as possible, as they are unlikely to get quicker or less costly to fix in time.

That being said, yes, you can choose to just utilise the validation portion of Pyblish, which includes an intuitive GUI with visualisation and error reports to your users, and simply stick to using native save functionality or your own custom tools.

There has also been some ideas about being able to “sudo” the execution of plug-ins, under the guise of someone capable of taking responsibility, such as a supervisor. But so far it hasn’t been needed.

https://github.com/pyblish/pyblish/issues/93

Q: How does the Dependency Injection mechanism work?

Edit: DI has since this writing been deprecated in favour ContextPlugin and InstancePlugin.

Is it really DI?

DI in Pyblish is inspired by how it has been implemented and is used in AngularJS.

The reason DI got implemented in the first place was to lower the initial learning curve for beginners, even though it meant making the project internals slightly more complex. It also opened up doors for some low hanging fruit, i.e. “services” which was added alongside it.

Technically

In short, it features a module-level Provider object to which Service objects are registered and later retrieved through your plug-ins when asked for.

For example.

import pyblish.api

def my_service():
  return "my_value"

pyblish.api.register_service("my_service", my_service)

class MyPlugin(pyblish.api.Plugin):
  def process(self, my_service):
    print my_service()

# my_value

The way it works under the covers is that it introspects the method process and retrieves any services matching it’s arguments.

In the above case, my_service is an available service and my_service is asked for by process and as a result it gets it.

You can try this process out for yourself by using the Provider object.

import pyblish.plugin

def my_service():
    return "Hello World!"

p = pyblish.plugin.Provider()
p.inject("my_service", my_service)

def process(my_service):
    print my_service()

p.invoke(process)

# Without the middle-man, here's the equivalent.
process(my_service)

Let’s have a look at how it knows to do this.

plugin.py:Provider

def invoke(self, func):
    args = self.args(func)
    unavailable = [a for a in args if a not in self.services]

    if unavailable:
        raise KeyError("Unavailable service requested: %s" % unavailable)

    inject = dict((k, v) for k, v in self.services.items()
                  if k in args)

    return func(**inject)

Here we can see the function being passed to invoke and how it compares args against services, args being the arguments of the function passed in.

The interesting part here is of course how these arguments are found. The inspect module to the rescue!

@classmethod
def args(cls, func):
    return [a for a in inspect.getargspec(func)[0]
            if a not in ("self", "cls")]

From here we could start to talk about how this works across IPC, and if you’re interested, have a look at how Pyblish RPC mirrors the behaviour in order to maintain this functionality across separate processes or even across the wire.

Q: How does create_asset() work?

Edit: create_asset() has since this writing been renamed create_instance()

How come data is entered via .create_asset() but retrieved through .data()?

The ability to provide data through Context.create_asset is a new feature and not well explained through the examples. It’s a convenience factory method which produces an object of type Asset.

Here is the equivalent.

import pyblish.api

context = pyblish.api.Context()

asset = context.create_asset("MyAsset")
asset = pyblish.api.Asset("MyAsset", parent=context)

To reduce the chance of misuse, Context.create_asset was added to automatically parent the newly created Asset to the Context, but both are equally valid in the eyes of Pyblish.

As assets typically have a family and other data, Context.create_asset was augmented with the ability to pass on any data during initialisation.

import pyblish.api

# Before
context = pyblish.api.Context()
asset = pyblish.api.Asset("MyAsset", parent=context)
asset.set_data("family", "MyFamily")

# After
context = pyblish.api.Context()
asset = context.create_asset("MyAsset", family="MyFamily")

But at the end of the day, it’s an optional convenience factory function.

Q: How does the custom test work?

Granted, writing tests is complex and you typically never have to write one.

The reason I wanted to illustrate it in the examples was to fully explain the reason for CVEI and why it works the way that it works today. The very next example is about explaining how this test has a default setting such that you won’t have to think about it. I felt it was still important for you to know it was there, to resolve ambiguity, and that it can be customised should you need to.

Alas, this is mostly an excuse and it’s possible both the implementation and examples needs work.

Q: How does Plug-in Order work?

Why not make orders global?
How about linking plug-ins to particular groups?

As you develop plug-ins, you later register them with Pyblish at run-time.

When publishing, Pyblish goes ahead and “discovers” all exposed plug-ins and sorts them by their corresponding order. It is this order in which plug-ins are processed.

# Plugin3          Plugin1
# Plugin1   --->   Plugin2
# Plugin2          Plugin3

CVEI plug-ins - Collection, Validation, Extraction and Integration - maintain a default order which Pyblish builds most of it’s logic upon, primarily when to stop processing, that order is 0-3 respectively.

It is expected that you use CVEI when developing, unless you have a reason not to. If you do develop your own custom plug-in, you can assign it the order from any of the CVEI plug-ins and it will be treated accordingly.

import pyblish.api

class MyCustomPlugin(pyblish.api.Plugin):
  order = pyblish.api.Collector.order

I suppose these variables could be individually provided, such as…

import pyblish.api

class MyCustomPlugin(pyblish.api.Plugin):
  order = pyblish.api.COLLECTOR_ORDER

The former was born rather than chosen, but in retrospect I might have implement it this way again if given the chance due to…

Cosmetics; it looks better
Encapsulation; less global variables
Hackable; you know where to look for hacks

At the end of the day, order itself is a simple solution to a complex problem; dependency management, and for that there are more interesting solutions in the talks.

Read more

Q: Why isn’t data a plain dictionary?

Edit: Since this writing, data in both the Context and instances is now a plain dictionary.

Creating and accessing data is currently done through getters and setters.

import pyblish.api
context = pyblish.api.Context()
context.set_data("key", "value")

Admittedly, this isn’t very Pythonic and you might think a dictionary should have worked just fine.

import pyblish.api
context = pyblish.api.Context()
context.data["key"] = "value"

And you would be right.

It has been discussed and will eventually make it into the API, it just hasn’t been a priority due to not adding any actual value besides cosmetics; though it’s clear it is much more pretty, Pythonic and probably easier to pick up and learn.

Of course, pull-requests are welcome.

Read more