[solved] Slow concurrent publishes

Finding that when publishing multiple times, Pyblish gets increasingly slower and noticed this in the task manager;

Is conhost.exe part of Pyblish?

Hmm, no, that’s a Windows process, not related to Pyblish.

Looks like it’s related to outputting to console windows, are you seeing heavy output in your Nuke console by any chance?

Do keep an eye on it when publishing though, I don’t think it should be related, and I haven’t noticed any slowdowns on my end but I typically don’t publish many times during testing. How many is many by the way, 10, 100?

If you could find a way to replicate it I could have a closer look.

Edit: “concurrent” means “simultaneously” by the way, you don’t mean when publishing from multiple hosts at the same time do you?

Edit: “concurrent” means “simultaneously” by the way, you don’t mean when publishing from multiple hosts at the same time do you?

No, I meant one publish after another in the same host session.

How many is many by the way, 10, 100?

10s

So I might have found something. I figured “python.exe” is Pyblish, and I kept an eye on it while opening the UI and validating, and here the results;

UI Launching

  1. 34.352K - Initialisation
  2. 72.228K
  3. 81.948K
  4. 94.860K
  5. 100.732K
  6. 103.276K

UI Launch and Validate

  1. 34.124K - Initialisation
  2. 84.304K
  3. 93.788K
  4. 101.760K
  5. 107.924K
  6. 114.184K

On both sessions the opening of Pyblish and execution of the plugins got increasingly slower.

Is this memory consumption of the python.exe process?

Is this memory consumption of the python.exe process?

yes

Ok, those values look normal. When you say slower, are we talking from seconds to minutes, or minutes to hours?

I’m looking to pin down whether the performance degradation is something worth investing time into solving. It’s quite possible there are things that can be optimised, but optimising means locking in features and will make updates take longer, maintenance to become more difficult and bugs more easily appearing.

Ok, those values look normal. When you say slower, are we talking from seconds to minutes, or minutes to hours?

Its seconds to minutes.

Oh, that’s really bad.

Any particular plug-ins that take longer, or is it all of them?

It seems to be all of them.

Ok, it might be the terminal getting choked up on messages.

I’ll have a look.

For performance, you can publish via scripting. This slowdown is most likely GUI related only.

I can wait, but artists can get increasingly annoyed with publishing, so I thought I would flag this now.

Ok, as a workaround for artists, you can ask them to close the GUI with shift held down. It will quit the process and launch a new. It’ll take a second or two to boot up, but should make publishing go as fast as the first time around.

1 Like

This has been solved in the latest pyblish-win.

Can’t remember the specific bug fixed to speed this up, @marcus can you?

1 Like

It might have been this one.

https://github.com/pyblish/pyblish/issues/201

1 Like