Installing Pyblish on a network


#1

Hello men

Goal
Implementing Pyblish in a studio that does not have any publishing system.

Issues

  • I need to install it on server then artists run it from server
  • Plugins will be called from server
  • Assets will be extract on server (in the project directory)

points

  • Server is Windows Serevr 2008
  • We need to setup pyblish on PCs of artists remotely and set PYTHONPATH or PYBLISHPLUGNPATH

i need some good helps for them as you have experiences on such hings or more, and i like to continue to this topic.
i will share my achivements here.

i can tell you more if you need more information.

Thanks.


#2

This shouldn’t be any problem at all as far as all the clients have at least read permissions to the folder where you host it. It’s a very standard setup.

This is the only thing you really need to deal with carefully. I’ve set this up in multiple locations by simply using wrappers for apps that set these up for each software. There are tons of ways to do this. The simplest and the least flexible might be to just use group policy on the domain to set the variable globally in the studio.

Better way of course is using something like be or ecosystem.


#3

Hey @mkolar
Thanks for reply.

So i just should install pyblish on Server and set permission to user for read?
And what should we do on client PC? just set the env variables?

And about PATHs i will research on be about that.


#4

Don’t use the standard pyblish installer, but instead go with manual installation. https://github.com/pyblish/pyblish-win/wiki/Manual-Installation

When that is done and client machines can access the folder where it’s installed, you just set env variables as you said.

For testing you can just set them manually to see if pyblish launches within maya for example. If that is working, you can move to finding solution for setting the variables automatically on all the clients.


#5

Very good tips, yes, i will use manual installation.
thanks.
i will put results here and ask my questions


#6

Yes. That sounds about right. They just need access to it in one form or another. Then all that’s left is to “tell” the computer itself where to find it.

Preferrably this is a matter of setting a PYTHONPATH environment variable. Though I wouldn’t recommend setting the system’s variables but look into the direction of “wrapper scripts”.

It might sound complex but it’s actually really simple. Basically the idea is that you run a script (for example a .bat file on windows) that sets the environment and then runs the application.

E.g. for windows making a .bat file with:

set PYTHONPATH=Z:/pythonpath/

Will override the environment variable PYTHONPATH within that running command prompt that started from the batch script. Of course the application closes since it reaches the end does nothing useful other than that.

Now if we just append a line that runs an application it will run that software with that particular environment variable.

Of course a “wrapper script” can do anything you want. All it means it’s running something to trigger something else. Actually it’s just some lines of code, a script. We call it a wrapper script just to make it easier for other developers to understand what it’s meant to do. So they can predict to some extend what it does.

Managing environment (like a pipeline) and running applications within it can require quite “customizable” setups depending on the complexity of the studio and its needs. For example imagine using a “specific version of Maya” when you run a project. Whenever you go back to that project in a couple of years you’d want to enter that same environment to run the project as it was used in those days whereas new projects will be using the latest and greatest. Packages like be try to set up a framework to manage such a customizable approach much more easily.

For now, start simple. Something like:

  • set up an environment variable (e.g. pyblish’s paths)
  • run an application (e.g. maya)
  • run pyblish from within it.

Once you have that working you actually know the core of what’s required to run an application with a “wrapper script” so it operates within for example your pipeline.

typed on my phone, sorry for any typos


#7

Hey @BigRoy
thanks good points, i will do surely.
just i’m out of office now. i will review your words tomorrow, and i will use your guidance and ask more questions
thanks
[Good Typing] :blush:


#8

Not sure my blob of text did actually clear this up, but the important concept is that you only set the environment “in that application” and don’t mess with any of the default/global environment variables on the system.

In short, the changes you do to the environment are only for that application’s session and you’re not storing anything in the OS. As such, the environment does not persist.

Looking forward to hearing about how you’ll be building it when you’ve started to mess around with it for a bit.


#9

Great info guys, thanks for sharing!

(Updated title and category so it’s easier to find for anyone looking for a network install)


#10

Thank you marcus
I hope we can talk about more topics here.


#11

What application you mean? you mean in server or environment variable in client system that wanna run pyblish?

I think i can do such thing like that:

  1. Installing pyblish manually in server
  2. set the python27 path to server’s environment variable PATH
  3. set the integration paths in client side that have permission access to the pyblish path

didn’t do that, but i will do soon


#12

I mean that you don’t set it in the system/OS itself as something that is stored there. But you customize it only within the wrapper script and run any application from within that, so it has access to those variables.

So you’d run Maya for example with your wrapper script:

set PYTHONPATH=C:/pyblish
call maya

This way your system itself doesn’t get cluttered with environment variables that need to be updated, cleaned and managed but only use a “environment” in which you run the application.

Does that make sense?

Basically if someone would double click the maya.exe that is in your maya installation folder it would run a totally clean maya without access to pyblish since you didn’t run it through the wrapper script and launch within that environment.

So “any application” would launch from within that environment. In the scenario I describe it’s the wrapper script that enters the environment and then could launch the application you want.


#13

yes, make sense.
but i will test all of these.

so “be” is good for using.


#14

Hello guys
i follow instructions of Manual Installation, and i have a few questions:

  • in server i just set pyblish-win/pythonpath in PATH environment Variable?

  • After setting paths to server and client machine, we should call any method of \pyblish-win\pythonpath\_util.py for installing paths, pythonpath or dependencies? or __init__.py file from every related integration path will run _util.py methods?

  • what is differences between pyblish-win\pythonpath and pyblish-win\lib\pyblish-x\pythonpath ?

Thanks


#15

in server i just set pyblish-win/pythonpath in PATH environment Variable?

The question is somewhat ambigiuous. But for an installation on the server where other workstations access it the workstations need the pyblish-win repository on their PYTHONPATH.

So yes, that is the path which you set on the workstations to allow them to use pyblish.

After setting paths to server and client machine, we should call any method of \pyblish-win\pythonpath_util.py for installing paths, pythonpath or dependencies? or init.py file from every related integration path will run _util.py methods?

You don’t need to run what you don’t actually need. Whenever you “start” needing something you just import pyblish_what_i_need and trigger how you want to use it.

Most integrations will have a setup() method within its .lib package. If I’m correct this is often exposed to ots __init__.py. Keeping that in mind “setting up” the Maya integration for example would be:

import pyblish_maya
pyblish_maya.setup()

As stated before. You only run this code when you need it, for example within Maya. In Nuke you wouldn’t need the Maya integration. So triggering this code above is usually only done for Maya. E.g. in userSetup.py.

The pyblish_maya even has its own that you can use to help out with this. For maya to use a userSetup.py it has to be under your PYTHONPATH. So for Maya you can add that particular content there: pyblish-win\lib\pyblish-x\modules\pyblish-maya\pyblish_maya\pythonpath.

Note that this is quite deep within the repository because of how it has nested the submodules:

  • pyblish-win
    • pyblish-x
      • pyblish repositories, for example…
      • pyblish
      • pyblish-maya

Also see the answer to the next question.

what is differences between pyblish-win\pythonpath and pyblish-win\lib\pyblish-x\pythonpath

pyblish-win encapsulates pyblish-x. Think of it like this:

  • pyblish-x is OS independent as a distribution
  • whereas pyblish-win is windows specific.

As such the “windows” variant holds the OS independent contents of Pyblish.
Thus it pyblish-win contains pyblish-x. Currently I don’t think those pythonpaths differ much and pyblish-win is more of a container for the windows specific installer. Over time (with updates) differences could grow further of course.

Because of this “encapsulation” and how it was meant to ease updating through submodules in git you get some “inclusion” levels within the package itself which might be somewhat confusing. But it’s best to just think of it as if the “work has been laid out for you a bit”. Otherwise you’d have to manage all individual repositories of Pyblish yourself and need to know which are compatible with each other. This takes care of that for you.

In this case if you’re using Windows workstations there’s nothing wrong with using pyblish-win\pythonpath.


#16

Hello BigRoy, thanks

I’m testing pyblish on the studio server, so far i did these, please let me know what is wrong?

  • copy pyblish-win on server, with this path for example, \\studio\DFS\Pipeline\pyblish-win
  • in my user i have access permission to \\studio\DFS\Pipeline (full control) (except creating environment variable, that i don’t know this will create problem or not)
  • with admin user i created these environment variables on my user:
  • PATH : \\studio\DFS\Pipeline\pyblish-win\pythonpath
  • PYTHONPATH : \\studio\DFS\Pipeline\pyblish-win\lib\pyblish-x\integrations\maya

Questions:

  • Environment Variables are specific for user or workstation?
  • When i run python in CMD it does not detected as a command
  • because we have many users in the studio how can i set environment variable for all from my user or from server? i mean not setting one by one on every user. it is related to be too?

i should say that DFS is a shortcut like path that give the user a path from original source files.

i will ask many questions soon

thanks


#17

Both. It depends on whether you are setting system-wide environment variables or user environment variables. But this differentiation is only made when storing environment variables that persist. When you set them in a “wrapper script” it’s only local to that session. As explained above that’s recommended as you won’t be “messing” with any administrator stuff of managing the local workstation.

For python to be recognized as a command in a command prompt it’s executable must be on the PATH (or in the running directory cwd)

Think of a computer being completely oblivious to whatever is stored on the hard-disk. It doesn’t know anything. You’ve Python in a specific folder stored somewhere, it doesn’t know! The only reason why an OS will know a command is because it’s told where it is to run it. Similar to PYTHONPATH you’d have to tell it where to look.

For example if Python is in C:\Python27 then add it to your PATH.

set PATH=%PATH%;C:\Python27

Now if you run python as a command the command prompt will look under the current working directory whether it finds it. If not it’s assuming you’re running a command that’s on your path. So it will go through all the paths under PATH until it finds a python executable in there. If it doesn’t find it, the computer will tell you the command doesn’t exist.

Actually, this is what the “wrapper scripts” could be doing. They are not actually “setting” the environment variables that you see when you go to “Computer” > “Advanced” > “Environment Variables” stuff, because we’re not setting them “on the system” but in the session of the wrapper script and then running the implementation.

For example try a .bat file like this:

set PYTHONPATH=%PYTHONPATH%;\\studio\DFS\Pipeline\pythonpath
set PATH=%PATH%;\\studio\DFS\Pipeline\path
call cmd

You’ll have an open command prompt now that has that environment variables set. In there you could start Python, run pyblish, etc. if it’s exposed in the paths set in this file.

Now instead of starting cmd at the end imagine running maya at the end. Basically the command prompt would close, but Maya would start to run (within that environment!) Voila! You’d have your first launcher script to launches Maya. A wrapper script if you will that does all of it for you where you don’t have to “manage” the system itself.

Even better, when updating the environment all you’d need to do is change the script.

it is related to be too?

Somewhat yes. be is a framework that allows you to manage environment that you enter and run an application within it. Again, you’re not setting the environment variables for the user and storing them on the system. You’d be entering the environment each time to launch the application. And that’s when you are running into “wrapper scripts” to ease entering the environment you want for that application.


Even though setting variables on the system itself and then “living with that” works just fine, it’s much harder to manage over time with many many systems. Unless again that whole “hardcoding” of the environment is done with a management system or set of scripts… but again, when going that far? Why not manage the environment on much lower-level, like whenever wrapping the environment?

Again, the recommendation here is to manage this with “wrapper” scripts so that you maintain control in a much more logical sense instead of managing all environment variables system-wide per machine that you have operating. Especially since you might want to change environments depending on the project you’re running, and that’s where the real power comes in.


#18

Look, my pyblish-win is on server and i should set its path to client machine, so as in Manual Installation marcus just talked about pyblish-win/pythonpath PYTHONPATH, but when people install pyblish-win manually on server first they should set python27 in PATH env that system detect python.exe, then set PYTHONPATH.

i did it with a batch file that will set python27 of pyblish-win that is in server then set pythonpath folder of pyblish-win.

but i want to do that for all users and everytime they start their windows.

besides, i create PATH and PYTHONPATH directly in environment variables but pyblish does not work in maya and python does not detect in cmd again

what do you think.


#19

Not sure how to interpret this; but if by “people” you mean the other artists, then you’ve misunderstood what Roy was trying to say.

I’ll try and explain via an example. I’ll assume all computers on the network has access to \\studio\dfs\pipeline\.

  1. You’ve got it right, that you should install Pyblish manually, as per the instructions to \\studio\dfs\pipeline\pyblish-win.
  2. Next, whenever anyone runs Maya, they will need to run it via a .bat script written by you. This .bat script can look something like this.
set PYTHONPATH=\\studio\dfs\pipeline\pyblish-win\pythonpath;%PYTHONPATH%
start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*
  1. If you save this as maya.bat, then this is what artists then use to launch Maya. The first line “installs” Pyblish, and the second physically runs Maya.

The challenge for you, is to get everyone to launch Maya this way, instead of running it from the start-menu or by double-clicking on Maya files in Explorer and so on.

Alternatively, you could walk up to every machine on the network, and add Pyblish to their physical PYTHONPATH, but as you probably understand, that is quite tedious. So as another alternative, you could have a look at be, but I would suggest you try @BigRoy’s recommendation first of using wrapper scripts as they will make it easiest for you and for your artists to adapt to a new workflow.

Let me know if anything seems confusing and I’ll try to explain it differently, otherwise give this a try and let me know how it goes. You could, and should, try this on your own machine first, before you get artists to try it. It will make it easier for the both of you.

Good luck!


#20

Very good @marcus
thank you. i will do and i will use BE too,
and i will study wraooer scripts.
and i will ask many questions.

just how can i see syntax of this command start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*
i mean what is its flags

thank you