Of course a choice could be provided. Do you have an example use case of what kind of “layers” are usually set up per object as opposed to being scene-wide in a lighting scene?
If you have a very specific example we can keep it into account.
A bit off-topic. But have you seen this?
Have some use cases, but it mostly very generic that we often use renderlayers per object. Right now we are using renderlayers to get out some paint effects with maya software along with some shaders from mental ray, so the renderlayer helps us switch renderer.
I wouldn’t worry too much, I was mostly asking whether you guys had a solution in place that I could look at.
Yeah, looks good. Probably something for the future.
Interesting. This also directly means it should be able to “switch renderer”. We have had this need in the past as well.
Yeah, have renderlayer overrides on the “Render Using” in the render settings to change the renderer per layer.
Holly molly, that looks incredibly precise.
Ok, it seems we might be picking up some pace here shortly!
Currently we’ve been trying to cover a lot of ground everywhere which has greatly shown in the amount of content this topic currently houses. And it’s time to grab the overall context and set some milestones and directions.
Here are some key points:
model family he should be allowed to do so. We’re not saying a free-for-all in production, but removing the restrictions we currently had in place where allowing a model to be published was dependent on whether you working in a modeling context.Outputs are versioned per family per instance. eg. Dog family: model and Dog family: playblast are both versioned separately.
It’s currently still in discussion what decides when a version is upgraded. At the moment we were always incremeting upon a publish. Yet I remember @marcus recently being a bit in doubt about that specific decision and moving into the direction of @mkolar where the versioning could be dependent on the Artist’s work file, and as such be in control of output versions.
We introduce the concept of Packages that define what set of published instances belong (or just pair nicely) together.
Especially the introduction of packages could potentially solve a lot of “management” issues referring to “what bulk of data do I need”?
Feel free to jump in with any thoughts, questions or ideas! Are you using any similar techniques like packages? (Do you even have such terminology?)
Apologies for the long unformatted post. I felt like dumping some ideas but only have iPad with me for a few days.
I like the concept of packages (Much better name btw than ‘group’ even though I’d personally go for bundlebut that really has no relevance to the discussion 
).Something of that kind is certainly needed. In a sense we are actually already using this concept in a very scattered and loose way when needed, but it’s a case of artist hand picking components to add to an asset in the database which together make up its output. So for example firefx v05 for a shot in ftrack might have components firefastV001, fireslowV004 and sparksV003. Somehow naturally I tend to call them bundles.
I did go through the magenta 0.6 Notes that @marcus posted in another thread and most of it makes good sense. Here are my thought on some things that are less clear.
The definition of a subset is a little bit puzzling in the context of an asset build. In the shot it roughly corresponds to a partial output of the scene. Probably something that came in a single instance (a character) comes out as a subset of current shot scene (animated cache of a character). This kind of makes sense, even though I find it hard to grasp the difference between the instance and a subset.
However when working on a asset build like a character. I can’t seem to find its usefulness and it appear to be only adding one level of complexity unnecessarily.
I assume that Packages are technically just json files, which contain a list of ‘links’ pointing to specific versions of components, plus a reviewable (movie,gif,image). If that’s the case, wouldn’t it make sense to separate the reviewable out of it and just treat it as another component of type ‘review’? That way the package would literally become just a simple file containing information about its needed components and a link to its corresponding review components.
Naming
You introduced a term ‘components’ in the asset map. This might be a good time to consider whether it makes sense to use it more widely. In my eyes it’s a perfect term for what we call ‘instance’ now. It applies very well to both asset build and shot part of the pipeline I think.
E.g.
From ben/modeling I feel that I’m publishing ‘components’ modelProxy, modelDefault and reviewTurntable, rather than instances. These together naturally form some kind of modelpackage’ that contains their respective versions. Sometimes it’s very useful to tag what proxy corresponds to what full res model. For example if hi res needs to be skin wrapped to low poly or if uv needs to match between the two.
When importing such model to scene artist then simply chooses which component he wants to import (or all of course)
From Sh001/animation it is the same story. I publish components Ben, table and ReviewMovie that, out together form animPkg.v001 for given shot. The principle stay the same across the pipeline with some exceptions of course such as camera and render elements, which are always a bit more complicated.
The main reason for calling it component is actually the fact that it come much more naturally to people in the 2 studios where I’m currently deploying pipeline. Instance is very generic a technical terms, that in top of it has quite specific connotations in the CG world. (Object instance the in the scene). This is find more confusing everyday.
Thanks for chiming in @mkolar! For completeness, here are those notes.
About package vs. bundle and instance vs. component, what I’ll say is, it’s all good. We know what they mean by now and everything else I think is attempting to merge personal preference. Which isn’t always easy, but perhaps not necessary either. Picking something and sticking with it on the other hand is.
For a description and rationale as to what Instance means, I’ll refer you to the Instance API notes.
We’ve struggled with comprehending this concept as well. The more we experiment with it the more fundamental it becomes, and I think in the latest and current version that Roy is working with at the moment, subsets are the only thing published from anywhere so it’s important we find a clear way of communicating this.
It is quite difficult to break down. Perhaps we need graphics or, better yet, examples. I’m hoping Roy will have something in this arena within the coming weeks.
That’s right, just a series of links to connected versions of subsets, like v013 of the character Dave01 together with v056 of Rider03.
example.pkg
{
"ben01": "v004",
"ben02": "v004",
"camera01": "v001",
"sofa01": "v001"
}
The way I’d like to look at packages is like just another subset. It’s a piece of data extracted from something like Maya, and as such I figure it must be capable of being extracted in different forms. One of which is the .json, for computer consumption, and another, perhaps .mov for human consumption.
I figure that whenever a character is published in isolation from surrounding characters, a package is updated, and it’s perhaps equally important to visualise the whole package in a way that doesn’t require loading the entire thing into memory.
But it could also be the case that the latter part is better left to a dedicated process in which an artist produces “dailies” for review. I would go for both, one automatic and one manual, but it would be up each studio to decide, which I think is also ok.
Just to hop in here for a bit it’s good to see the “confusion” around subset being raised again. It’s definitely one of the harder things to grasp.
To be completely honest I think @mkolar makes a good point about the components. To me an instance has always been a bit vague, but @marcus has clarified things to me by bringing up that an instance is a “inverse of a file on disk”. As such extracting an instance should result in a specific set of data that should act as a single file. (It could be multiple files, but only when they belong together!). Nevertheless the same data could be represented in different ways (e.g. as an Alembic or a Maya Binary). If i’m not mistaking that would be two different instances.
Though I do have to say I’ve also seen different formats for the same data being referred to “Representations”. Is an instance always a single representation? Are they exactly the same? Or what’s different?
Then there’s the need for subset (or component) to differentiate different kinds of data output: modelDefault and modelProxy. Of course these can all come with their own instances (Alembics, etc.)
So to me what @mkolar mentions as component is actually a subset. component == subset
Exactly, here’s some examples to help narrow it down.
The hierarchy works like this (with a minor simplification, see below about versions).
That is, subsets contain instances, and an instance is a single file or file-sequence. (Redacted, see below)
- modelDefault
- modelDefault.png
- modelDefault.ma
- modelDefault.mov
- modelLowres
- animRig
- simRig
- clothRig
- ...
Those are all from the asset department. Here are some from shots.
- Ben01
- Ben01.abc
- Ben01.geoCache
- Ben01.gif
- Sofa03
- masterLayer
- bgLayer
- bgLayer.png
- bgLayer.0001.png
- bgLayer.0002.png
- bgLayer.0003.png
- ...
Inbetween the Subset and the Instance are versions.
For example.
- Ben
- modelDefault
- v001
- Ben
- modelDefault
- v001
- modelDefault_v001.png
- modelDefault_v001.ma
What we really need are some practical examples, because the reasoning for these things are so deeply nested in production circumstance and is difficult to hold on to. I might even have missed out on a few steps here myself. Roy, was this how we left things?
Looks we just found the problem of the subset, component, instance, representation talk.V
In the context of Pyblish itself and the API notes that @marcus pointed to, instance is not one specific file per se. When publishing you can run multiple extractors on it and create multiple different representations of this instance. So Ben has instance modelDefault which has .png, .ma and .mov representations.
Whereas in magenta As explained by @marcus in the previous post. Instances seem to be a single file or a file sequence so modelDefault is a subset and instances are .png, .ma and .mov
This inconsistency between Pyblish, Magenta makes the concepts discussed here a little ambiguous.
Seems right to me, however as mentioned earlier this is quite different to what instance is described as in the API docs.
Instance as an inverse of a file makes very good sense when importing, however when publishing it doesn’t apply the same way in the current state of Pyblish core and UI.
@mkolar, are you referring to the Workflow portion of the API docs? To be fair, it does say that these are guidelines, and not definitions. The definition is above.
But you make a good point, those guidelines should not be there. Those should be part of an extension, such as Magenta. I’ve updated that page, see here for revision history if you need to double check.
With that done, I think that’s as slim as the definition for Instance can get, and even though it may be possible that we’ll develop another extension that produces another definition of the word, I think that is acceptable. Extensions define what is what, they define the “pipeline” if you will, the software does not. The software is only there to facilitate these definitions, to enable you to think in ways that makes sense for your workflow.
@mkolar I’m having a closer look at this now and I think I’ve gotten confused… sigh…
Typically we would have broken down the problem into smaller chunks by now that we could discuss in isolation, but this hasn’t happened. Like we’re biting off more than we (or at least I) can chew.
Any ideas on how we can divide this, and focus on one portion at a time? I’m working on some more graphics to help communicate things. Maybe an animation would help?
It would be fair to think that, if it is this complicated, we must be doing something wrong. But I don’t think this is the case. I think this is a naturally complex subject with many interacting parts. I also think there is a better way to break it down, we just haven’t found it yet.
Ok, I’ve updated the notes to reflect how we left things during out last spring at Colorbleed.
Here’s the gist of it (pun).
@BigRoy, it looks like these guys are doing what you are doing with shaders and alembic, maybe their source is open for peeking?
Auto shader hookup on Alembic import
I remember me and @tokejepsen had a solution to this a few years back. It was a bit hardcoded for the project at the time and fairly simple but it might be worth looking at it. @tokejepsen do you remember where it was stored.
If I remember correctly we saved shaders as maya scene with the mesh and all the assignments. Then when we brought in the alembic file we have transferred all the connections from the original mesh to the animated one. I believe we also tried the way of connection alembic output to a non-animated shaded mesh. But can’t remember what were the troubles there. Maybe @tokejepsen could chime in if he has better memory.
Anyway, I’ll be needing it again too so I’ll dig through some old code and see if we can compare solutions.
Can’t remember exactly, but I think we transferred the animation from the cache to the shaded mesh by connecting attributes together.
Ah, yeah @BigRoy had a solid, albeit ad-hoc, solution to this as well. Seeing as you’re all using it, maybe make it a meal and start a GitHub repo about it?
Could be the start of pyblish-alembic?
I would make it a standalone package altogether; doesn’t seem like it needs anything from Pyblish in order to work. Then we could have a look at integrating it with Pyblish. Kind of like capture.py; useful on it’s own, and useful with Pyblish.
