Currently I feel there are couple ways to implement pyblish inside a Shotgun environment
Shotgun API: Wrap Pyblish around Shotgun API. Can be slow without some form of caching.
tk-multi-publish: Wrap Pyblish functionalities around tk-multi-publish app. One convenient way is to overwrite publish hooks with Pyblish internals, matching the existing steps (scan_scene, primary_pre_publish, primary_publish, secondary_pre_publish, secondary_publish, etc.) to Pyblish plugins.
Shotgun Toolkit API: A thin wrapper around Shotgun Toolkit API. Just using Shotgun Toolkit to provide data for context and instance and register publish and the rest are just Pyblish.
This is an interesting thought. I wonder whether it wouldn’t involve too much “hand-holding” of Pyblish and whether it would really enable you to get the most of out of it. For example, Pyblish has the notion of a “validation” step that I’m not sure tk-multi-publish could represent in it’s current stage.
The idea is for no publish to be made possible, unless the content is up to par with pipeline standards. But if that is of less importance, then you could potentially skip that part in which case the mapping might be almost 1-1.
Perhaps the most straightforward way forward, with the least amount of dependency on Shotgun, would be step (1) and (3), to use Shotgun in order to collect information about content, and let Pyblish validate and extract it. Once done with that, it could again communicate with Shotgun for final integration and notifications with its database.
If you are familiar, or can make the mental conversion, with ftrack, then perhaps this repo could help shape your idea of how it could work. It’s developed by two engineers at different studios using it in production at the moment.
I would go with option 2 or 3, but personally lean towards 3. Option 2 might be more familiar to existing Shotgun users, but as @marcus says there might be too much bending of Shotgun workflow to Pyblish workflow, and you might have problems with maintenance when Shotgun changes anything on the UI side.
Option 3 is very similar to how we build pyblish-ftrack. I think one of the benefits of option 3 is the utilization of Shotguns path templating, but if you can make that available for Pyblish I you’ll have a winner:)
currently the validation is done mostly in primary_pre_publish hook, for example in the :
class PrimaryPrePublishHook(Hook):
# initialization code ...
# check if pre_publish is in Maya, if Maya call self._do_maya_pre_publish() method
def _do_maya_pre_publish(self, task, work_template, progress_cb):
import maya.cmds as cmds
progress_cb(0.0, "Validating current scene", task)
# get the current scene file:
scene_file = cmds.file(query=True, sn=True)
if scene_file:
scene_file = os.path.abspath(scene_file)
# validate it:
scene_errors = self._validate_work_file(scene_file, work_template, task["output"], progress_cb)
progress_cb(100)
return scene_errors
If len(scene_errors), we will issue some form of Exception or error message to user and interrupt the publish process.
If you are familiar, or can make the mental conversion, with ftrack, then perhaps this repo could help shape your idea of how it could work. It’s developed by two engineers at different studios using it in production at the moment.
I am not currently familiar with ftrack, but I do wish to coordinate the effort on their side so we can try to keep as consistent a interface as possible. Also to reduce dependency and coupling between the extension and production tracking system as much as possible.
Production Tracking is currently enjoy some form of boom with several old and upcoming competitors (Alien Brain, Nim Lab, Tactics, and also the open source Stalker), so it might be good idea to keep that in mind when developing extensions.
Main advantage of 2 is that the user doesn’t need to learn new UI. Even though I think the main Pyblish UI is also intuitive enough that the learning curve isn’t steep. Also the Hook system in Shotgun is de-couple enough that we do not need to change the main application code. It is also easy to switch between the old code and the Pyblish code by change the configuration YAML file.
The only disadvantage you will currently have, will be not having access to Action in Pyblish but apart from that I think option 2 is looking pretty good.
Maybe go for this at first, take it as far as possible and if you find yourself struggling, a transition to the Pyblish GUI should be a matter of artists hitting another button; all of your code can remain the same.
It is probably two separate repo. One is more coupled version (depends on Shotgun’s Hook system) and the other more conform to the Pyblish standard. If Pyblish itself is modular enough, sharing code between the two should be too big of a problem.
If we figure out a proper archiecture for Pyblish extension we may even share Plugin code between different extensions.
I think the main point is to give the option of using Shotguns publishing UI and Pyblishs UI. As for similarities in the interface we might be able to do something, but it may not be worth it. Think the main point is giving developer access to version control.
All extension are python packages, so it might be a good idea to keep all functional code in pyblish-shotgun, so you can do something similar to this in the primary_pre_publish:
import pyblish_shotgun
class PrimaryPrePublishHook(Hook):
# initialization code ...
# check if pre_publish is in Maya, if Maya call self._do_maya_pre_publish() method
def _do_maya_pre_publish(self, task, work_template, progress_cb):
progress_cb(0.0, "Validating current scene", task)
if scene_file:
scene_file = os.path.abspath(scene_file)
# validate it:
scene_errors = pyblish_shotgun.validate_all()
progress_cb(100)
return scene_errors
That’s exactly what I had in mind. So the user doesn’t even know (or need to know) we are using different internals
Shotgun do have a framework model for shared code. However it is quiet specific to Shotgun so code sharing between different Production Tracking systems is not possible.
So I guess another way to approach it is to use pyblish-shotgun as a library that can be used either by Shotgun Publish hooks or a very thin Shotgun App that really is just a launcher that launch a instance of Pyblish GUI.
This directly launched Pyblish from Ftrack in the browser. Is this similar to what you mean @dshlai? Guessing it’ll be launched from Shotgun desktop.
And you were thinking that you’ll have another entry called Pyblish UI... underneath the Shotgun filemenu?
Right now I think its just cosmetics:) You can decide where to put the access to the Pyblish UI. Main point is to have the library of pyblish related code in pyblish-shotgun.
One thing to look out for would be that the pyblish-maya extension, automatic creates a Publish entry underneath File in Maya. This may confuse the users when they are used to go to Shotgun/Publish...
Yeah, that’s where I want to put it (in the Shotgun menu).
I haven’t install pyblish-maya extension yet. In this case I will probably opt not to install that at the same time. I will like to exam that repo as well to see it work under the hood.
If you aren’t going to be using Pyblish QML, then the integration isn’t going to do much for you.
The only thing of interest that I can think of are the plug-ins; the idea is for plug-ins of every host to have access to some basic facilities, such as getting the path to the currently open file through context.data["currentFile"]. That way you can bridge plug-ins by not relying on host-only libraries for that information. The more of that we have, the more flexible plug-ins can get in terms of where they run.
If you have a look at the plug-ins available for Maya (there aren’t many) and either copy those or implement your own, then your plug-ins should align perfectly with other integration packages.
