Background Publishing

Feature Proposals
#1

I came to think of a possible implementation and use of the Background Publishing issue.

What if, whenever you hit Publish:

  1. Your scene is saved (into a temporary directory that you cannot see nor touch).
  2. Pyblish launches standalone, either automatically or by hand.
  3. Your plug-ins then operate solely on the saved file, with no interaction with your currently opened file.

Benefits

It would mean support for any host, even those without scripting capabilities, without integration, other than running an external app or in worst case, asking the user to open it themselves once the scene has been saved into the designated location. For example, an artist could save out JPEGs from Photoshop or sculptures from ZBrush and validate those before conforming them to an asset library.

It would also mean publishing without interfering with the scene. A user can publish, it can take however long it takes, and move on with his work undisturbed.

It would also mean publishing can be destructive. That publishing could make permanent changes to the scene before performing any serialisation. For example, it may be desired to bake all keys so as to not bring along any constraints or expressions. As baking alters the scene, this can be difficult and error-prone in the currently opened scene.

Disadvantages

You would be unable to interact with your scene through Pyblish, such as performing repairs or custom actions.

In some cases, publishing can take longer. Although in others, it can go faster.

It may take longer because the scene must be saved prior to publishing and also in cases where deep introspection is needed, such as querying attributes or relationships between nodes within Maya (in which case you need mayapy). It can go faster, because publishing can be done using multiple instances of a host for introspection. For example, it could launch mayapy once per plug-in. E.g. extracting a pointcache and quicktime could each run concurrently.

Compromise

As both the current (foreground publishing) and background publishing have their own pros/cons, it may be ideal to try and get the best of both.

For example, there could be a toggle/setting specifying whether to perform a background or foreground publish. It could be specified at any point prior to publishing, such as per-studio, per-project, per-user or per-publish.

Related

#2

Sounds interesting. You’d have to make sure, that the newly spawned instances have identical environment to the original. Which should be automatically taken care of if the new process is being spawned as a new process from the original, but I have a feeling there might be some unexpected troubles from that.

I’d also be worried about the extra loading times per plugin. On certain networks (our currently for instance), this might be extremely slow.

Fully background publishing might not be the best choice though. I’ve prefer artist to have at least some kind of feedback about what’s going on.

Overall though it’s a very interesting concept worth exploring.

#3

Oo, that is a good point, did not consider that. You’re right in that at least the environment variables should remain identical, or at least can be made identical fairly easily. But in case of having launched the software from a wrapper of sorts, there may be other things happening than simply starting the process; like registering it with some sort of central service discovery. And not to mention taking up another license (?) though I’m not sure how that works when both are being used on the same workstation, which possibly varies depending on the software (?).

Not as straightforward as I would have hoped.

#4

Certainly dependent on the software. Can’t remember from top of my head, but I’ve certainly run into softwares over the past years that took a licence per instance, rather than machine.

#5

Sounds like it limits our options then.

Do you know of any software that allows for a headless copy that doesn’t occupy a license?

#6

Houdini and Nuke allow this for sure. Can’t remember about maya.

#7

That is very good news.

#8

I think the important bit here is that it becomes possible to run things in the background and not necessarily be the only solution. Personally I’m with @mkolar that it’s fine for artists (maybe even preferrable) to directly see what’s going on and immediately get their validations running in the current open scene.

On the other hand there’s batching a lot of publishes. I think it would be great if there was a possibility to just drag and drop files onto a Pyblish tool. Or batch publish a list of file paths with Pyblish in Python.

import pyblish
pyblish.publish_files(files)

I remember batch processing was already being discussed somewhere, but I can’t find it right now. :wink:

#9

I could see this being useful when extracting data that takes a long time. Although I do find when artists wants to publish it is to get feedback or think they are done. In both cases they wouldn’t go back to the work file, so if the extraction failed they would have reopen the file.