User Fields in Pyblish QML

#1

Goal

An idea from @mkolar (here) about providing the ability for developers to add custom fields for users to fill in during publishing. Most prominently, a comment field.

Implementation

Each filled in field is added to the context after Selection, but before Validation and are meant to get accessed either during Extraction of Validation.

I haven’t encountered an implementation before, but it seems rather common in web applications, apparently FTrack does something similar and possibly Shotgun too.

Ideally, I would like to see these fields added on a per-plugin basis. When a plug-in capable of making use of a comment should there be a commenting field, and if a plug-in is capable of making extensive use of fields, such as specifying start- and end-frames for e.g. a playblast or cache, I’d like fields to remain specific to this plug-in.

Perhaps this is a separate concern better handled separately?

Another question is, where do users fill in these fields?

Option A - Registration

User can add a file some place global with wanted fields, and register it at run-time.

pyblish.register_user_fields("//server/userfields.json")

fields.json

[
  {
    "label": "Comment",
    "type": "text",
    "name": "comment",
    "value": "Write a comment about this publish",
  }
]

This could work, though it doesn’t leave much room for adding fields to a specific plug-in, unless they are specified up-front.

{
	"ValidateInstances": [{
	    "label": "Comment",
	    "type": "text",
	    "name": "comment",
	    "value": "Write a comment about this publish",
	}],
	"ExtractPlayblast": [{
		"label": "Start frame",
		"type": "number",
		"range": [0, 1000]
	}]
}

Alternatively, the data can be registered directly.

pyblish.register_user_fields([
  {
    "label": "Comment",
    "type": "text",
    "name": "comment",
    "value": "Write a comment about this publish",
  }
])

Option B - Embedded

Fields could be embedded into a plug-in.

As each plug-in is loaded up-front, the GUI have access to all of its properties before publishing.

class ValidateInstances(...):
  fields = [
    {
      "label": "Comment",
      "type": "text",
      "name": "comment",
      "value": "Write a comment about this publish",
    }
  ]

The issue here is, what happens when two or more plug-ins define the same fields? As order is most often undefined, we can’t rely on a “last one wins” strategy.

Option C - Automatic

The above approach overlaps a previous issue, that we might be able to merge.

The thinking here is that, a developer would add arguments to his/her process_* method and they would automagically appear as fields in the GUI.

def process_instance(self, instance, comment):
  # comment is a user-field and gets added to the GUI

To know which widget to provide for, a type and description can be provided via the docstring; given it conforms with Google Napoleon or similar.

def process_instance(self, instance, comment):
  """Process this instance

    Arguments:
      comment (str): Provide a comment
  """

Another thing to consider is that these fields are specific to a GUI and have little to do with the plug-in itself, so adding them to the plug-in might not be the best approach for future-proofing and keeping responsibilities separate - e.g. we wouldn’t want these fields when using the plug-ins through another application, like a web application incapable of using them, or that needs similar fields for different purposes. Our plug-ins would become convoluted very quickly.

How else can we handle this? How does FTrack/Shotgun do it, is that a good/better way?

Options for plugins
Pyblish 1.2 Released
#2

I’d certainly be voting for option B or C. A is a bit too generic and clumsy when one wants to link these fields to plugins.

B seem the most elegant and flexible to me. It doesn’t add much bulk to the plugins unless user specifies tons of extra fields, which really shouldn’t happen. Very few of these should be used per plugin to avoid clutter but that I think should be up to the td integrating pyblish into studio. If they want to flood artist with options and make plugins massive list of fields, then be it. With great power comes great responsibility.

C is very nice, but maybe doesn’t provide quite as much flexibility as B.

The issue here is, what happens when two or more plug-ins define the same fields? As order is most often undefined, we can’t rely on a “last one wins” strategy.

How about keeping the custom fields close to it’s plugin in the UI? That way even if 2 plugins ask for comment, each would just get it’s own comment field.

custom_attrs.jpg933×798 109 KB

Don’t know how they do it under the hood, but from user point of view, It’s practically our option B. Specify a list of fields in this exact format in an ‘action plugin’ and expect the same fields back so you can work with them. When they introduced it, it was super easy to understand and start using. Hence the reason I like probably.

Well if we make sure the they always must have a default value, then the data would exist even without the UI generated so it wouldn’t break any functionality. I’d see these as kind of user overrides on values you’d be setting in plugin probably anyway. If you’re building a plugin that should work the same with UI and without, then you just don’t ask for any custom fields and it’s the same as it is right now.

#3

That looks great, @mkolar. Definitely something to consider as a next design. It’s a bit cluttered, but maybe some of the detail could potentially be hidden until hovered?

Untitled.gif881×701 86.9 KB

For reference, here’s how the original train of thought were.

The disadvantage of having fields per-plugin, visually, is like you say, when multiple plug-ins ask for a similar fields, such as comments.

I consider per-plugin comments to be highly specific. The act of simply describing your publish in the first place is tricky enough.

Context Fields

We could possibly have a look at separating the concerns. Some fields are for plug-ins whereas others are for the world/file, such as a comment.

context.set_data("fields", ["comment"])

Context fields could reside somewhere global in the GUI, whereas plug-in fields reside like in your illustration.

Defaults

I thought some more about where to store the data and how to couple it with it’s corresponding fields. Here’s how I’m thinking.

class ExtractReview(...):
  fields = ["format", "quality", "outputPath"]
  1. All fields added to a plug-in is visualised in the GUI.
  2. If any field receives a value from the user, the corresponding key and value is added to the Context.
  3. If the corresponding key already exists in the Context, it is used as a default value.
  4. If a field has no definition (like above), type defaults to “string” and label a title-cased version of the name.

Syntax

Fields are more or less a schema so how about aligning ourselves with something like JSON Schema? With the exception of storing fields in a list as opposed to a dictionary so as to have control over order.

E.g.

[
  {
    "name": "comment",
    "label": "Comment",
    "description": "Age in years",
    "type": "string",
    "placeholder": "Enter comment here.."
  }
]

Definition:

name: Name accessible via code, e.g. context.data("__fields__")["comment"]
label: Visual name, e.g. drawn next to field
description: Fields documentation, e.g. visible on hover
type: Determines which widget to draw, e.g. textfield for string, checkbox for boolean
placeholder: Optional, unselectable hint about what to enter

Primitive types can be found here:

Access

Fields can be added to the context for further processing.

class ValidateInstances(...):
  fields = [
    {
      "label": "Comment",
      "type": "text",
      "name": "comment",
      "placeholder": "Write a comment about this publish",
    }
  ]

  def process_context(self, context):
    comment = context.data("__fields__")["comment"]
    comment["placeholder"] = "Write something nice here"
Simplifying the GUI
#4

Absolutely. I haven’t touched the ‘instances’ side, but the less clutter the better. I generally think that the UI could use some more breathing space. It’s tiny right now. Nice airy UI is always nicer to use, than crammed one.

Agreed. It might very nicely go with the plans to show context as instance. I can imagine the top or bottom of the UI (slightly separate from the instances and extractors) have are which displays important info from current context and let’s user add to it things like comments and such which get passed into plugins that ask for them.

Agreed

Totally. what you have there is clear, simple to use and has contains all we need.

#5

Ah yes, that is a really good idea!

The Context could always appear in the same spot of the UI, e.g. either top or bottom, so common fields would always remain in a fixed and common location.

edit: I replied before reading the rest of your post, and noticed you said exactly this. :blush:

#6

I was wondering if User Fields are still on the road map and what the directions will be going forward? Or how are things like commit/publish messages and increment version currently being handled?

#7

It is and it fills a unique gap in the feature set, but there are a lot of things on the roadmap and I’m a single-threaded, synchronous being.

About commit messages in particular, I had an alternative idea. User fields are still uniquely useful for arbitrary data so this is an addition, rather than an or.

As a new instance, it could get a unique family, with it’s own plug-ins operating on it. Such as comment.

Thoughts?

#8

Interesting but a little unintuitive for end users I think. We have to keep the dumbest possible animator in mind with the UI (no offence meant, purely a figure of speech). So comment which is arguably the most commonly used extra piece of information for publishes should be blatantly obvious and editable in the ui. I can see tons of weird comments going through if user types it once and doesn’t see it afterwards

Apart from that, having is as an instance doesn’t sit well with me as it is most likely a comment describing the state of work at this stage in general and probably applies to all the instances being created. At least indirectly. It’s not a representation of something in the scene, so I’d keep it separate from it.

The way I see it, it’s more like a data in the context, or data in an instance if studio needed per instance comments for any reason. (That would be an overkill, but I can think of a few situations where it might be useful)

Once again I would point to the shotgun publish UI as a very loose reference because it has very intuitive placements of these elements, that everyone gets at glance.

1 Like
#9

Thanks @mkolar.

That’s interesting, I actually consider it exactly like that - something equally part of your work, yet separate in terms of what it is, as any other instance.

Like an animated rig, a dedicated but optional playblast, and a comment. Three instances, processed by separate plug-ins, yet part of the same piece of work.

I also think comments especially should be more obvious than this though, the thing I liked about this alternative is that it opens up for other things a user might want to add as an instance, where you could develop a series of plug-ins that only trigger if the user-defined instance (how’s that for a term?) is present.