Adoption questions

Hello-

I’m new to pyblish and evaluating for adoption. I have a few questions prior to the next steps. I am hoping someone can answer them.

  1. After the asset is verified and goes off to be publish/saved/used by production, where does it go? Does pyblish have a service that it uses that maintains the location and for other service to retrieve the assets?

  2. Can pyblish be used from multiple sites. In other words, is it possible to keep two separate sites/independent installations in-sync?

  3. Related to question 2, can a pyblish service be replicated to another location?

If pyblish utilizes other backend tools that are closer to the file system, I’d like to know more about these.

If you need more clarification please let me know.

Thank you.

Hello @Rita_Garcia, welcome to the forums!

To answer your questions broadly, you can think of Pyblish as just for-loop, where each element is a snippet of Python code you write.

def snippet1():
  print("Hello ")

def snippet2():
  print("World")

def publish():
  for snippet in [snippet1, snippet2]:
    snippet()

publish()
# Hello World

In Pyblish, these snippets are referred to as “plug-ins”, each plug-in is written by you and Pyblish does only what is within these plug-ins.

For example, if you somehow would like to interact with files, then it is within a plug-in that you decide what to do with these files. Pyblish has no knowledge of what is actually in a plug-in, and you can use the Python standard library as you normally would.

import shutil
from pyblish import api

class MyPlugin(api.ContextPlugin):
  def process(self, context):
    shutil.copytree("/src/dir", "/dst/dir")

And the same goes for validation. If you should want to ensure one of your conventions, then it’s within a plug-in you could do this.

from pyblish import api

class MyValidator(api.ContextPlugin):
  order = api.ValidatorOrder

  def process(self, context):
    assert context.data["user"] == "marcus", "Wrong user"

You can give Pyblish special knowledge about your plug-in via the order attribute. In this case, you are letting Pyblish know that this is a “validator” in which case throwing an exception will cause publishing to stop before moving onto the next phase, extraction.

There are 4 pre-defined orders in Pyblish that are run one after the other - collection, validation, extraction and integration.

To get some hands on experience with these concepts, I would recommend you spent a few minutes running through the first few examples in Learn Pyblish by Example, especially:

  1. Introduction
  2. Hello World
  3. Quickstart
  4. Files

Yes, as a standalone Python package, you could keep it in sync like any normal Python package. For example via a shared directory, or by keeping track of the version you are currently running.

$ pip install pyblish-base
$ python -c "import pyblish;print(pyblish.version)"
1.4.2

Would it be possible to elaborate on this? I’m not sure what it means.

Pyblish runs entirely within a single Python process and makes no attempts to connect to any external program.

Let me know if this answers your questions, or if you have got more, and good luck! :slight_smile:

Hey @Rita_Garcia :slight_smile:

Welcome to the Pyblish forums!

  1. This is something you as a developer will decide. Pyblish is just a framework to work with. Saying that there are a couple of packages of plugins that try to give you a more complete pipeline, although they can also be used as inspiration for your own; pyblish-magenta, pyblish-napoleon and pyblish-starter.
    You can also browse various production packages, that different studios have open-sourced; pyblish-bumpybox and pyblish-kredenc.

  2. Pyblish by it self does not have any feature like this, but as its a framework for plugins you could write a syncing plugin. You could also utilize any production tracker like Ftrack or Shotgun, and have Pyblish work with those.

Thanks for answering my questions and providing more material for further reading. Apologies for my initial vague questions, let me try to elaborate.

I believe @tokejepsen answered, in that, Pyblish does not deal with the assets once they are successfully saved to the file system.

It looks like Pyblish is a good gate keeper for asset validation and we can create a plugin, or use an existing BumpyBox’s toolkit to save the asset to the file system. Ditto for the asset retrieval from the file system.

My concern is the ability to quickly, reliably replicate the contents on the file system to another network.

If I am way off base with the scope, apologies. If you do know of other systems, like @tokejepsen suggested FTrack, Shotgun, then please let me know. However, I have used Shotgun previously and studios still need a service similar to what I am describing.

Thanks in advance.

Do you mean the data of the artist files, like Maya/Nuke/Houdini scenes?
Or the pipeline files themselves, like Pyblish?

Ok, you might need to elaborate on this one. When do you expect replication to happen, who does it and how is it done currently?

In a nutshell, publishing as far as Pyblish goes is about outgoing data from software such as Maya. The data can be put anywhere, with various rules and context sensitivity. But once put there, it does no more. It also does not concern data that has already been output, such as existing assets on disk.

To answer both your questions, the files I was referring to would come from maya/nuke et al.

Replication is done not that often. Perhaps once to twice a day.

Thanks for the clarification Rita.

I think the reason a solution to this is taking so long is because I don’t think we yet fully understand what you are asking.

Would it be possible to share a small example if how replication happens currently?

Replication is done every time a job starts on the farm and finishes. The replication is done on the assets using rsync however, we can be flexible with the replication method. Likewise, if you are aware of other utilities to help with this problem, we are open to evaluating those.

What render farm management software are you using?

I’d have to check on the software. Is this a consideration since our replication is performed by another service?

It mostly because I know Deadline has event plugins and dependency scripts. If you wanted to replicate the source files across different locations, I would look into these.

@Rita_Garcia, I think there might be some confusion about what Pyblish is.

I would refer you back to my first response in this thread, all of what Pyblish can do is there. It is not an ftrack or Shotgun replacement, it does not involve replication and is not a SaaS of any kind.

To really get a sense of what it does, the Learn Pyblish by Example is a tutorial-style introduction to how it works and what it does; spend 10 minutes there and you’ll know whether it is what you are looking for.

On top of that, there is the Webinar which is a 10 minute walk-through of what it is, 10 minutes of production examples followed by 10 minutes of Q&A.

If possible, it would be really great to hear your impression of what you expected Pyblish to be, and what else you are comparing it to, so that I can work on improving the message here on the forums and website for when the next person comes looking in this direction.

I appreciate all the links you and @tokejepsen has provided. I will go over these links and contact if I have other questions.

@tokejepsen, do you have a link for Deadline? The name makes it hard to seek out the product. Thanks!

@tokejepsen, do you have a link for Deadline?

Sure😀 http://deadline.thinkboxsoftware.com