Installing Pyblish on a network

Pipeline Development
#10

Thank you marcus
I hope we can talk about more topics here.

#11

What application you mean? you mean in server or environment variable in client system that wanna run pyblish?

I think i can do such thing like that:

  1. Installing pyblish manually in server
  2. set the python27 path to server’s environment variable PATH
  3. set the integration paths in client side that have permission access to the pyblish path

didn’t do that, but i will do soon

#12

I mean that you don’t set it in the system/OS itself as something that is stored there. But you customize it only within the wrapper script and run any application from within that, so it has access to those variables.

So you’d run Maya for example with your wrapper script:

set PYTHONPATH=C:/pyblish
call maya

This way your system itself doesn’t get cluttered with environment variables that need to be updated, cleaned and managed but only use a “environment” in which you run the application.

Does that make sense?

Basically if someone would double click the maya.exe that is in your maya installation folder it would run a totally clean maya without access to pyblish since you didn’t run it through the wrapper script and launch within that environment.

So “any application” would launch from within that environment. In the scenario I describe it’s the wrapper script that enters the environment and then could launch the application you want.

#13

yes, make sense.
but i will test all of these.

so “be” is good for using.

#14

Hello guys
i follow instructions of Manual Installation, and i have a few questions:

  • in server i just set pyblish-win/pythonpath in PATH environment Variable?

  • After setting paths to server and client machine, we should call any method of \pyblish-win\pythonpath\_util.py for installing paths, pythonpath or dependencies? or __init__.py file from every related integration path will run _util.py methods?

  • what is differences between pyblish-win\pythonpath and pyblish-win\lib\pyblish-x\pythonpath ?

Thanks

#15

in server i just set pyblish-win/pythonpath in PATH environment Variable?

The question is somewhat ambigiuous. But for an installation on the server where other workstations access it the workstations need the pyblish-win repository on their PYTHONPATH.

So yes, that is the path which you set on the workstations to allow them to use pyblish.

After setting paths to server and client machine, we should call any method of \pyblish-win\pythonpath_util.py for installing paths, pythonpath or dependencies? or init.py file from every related integration path will run _util.py methods?

You don’t need to run what you don’t actually need. Whenever you “start” needing something you just import pyblish_what_i_need and trigger how you want to use it.

Most integrations will have a setup() method within its .lib package. If I’m correct this is often exposed to ots __init__.py. Keeping that in mind “setting up” the Maya integration for example would be:

import pyblish_maya
pyblish_maya.setup()

As stated before. You only run this code when you need it, for example within Maya. In Nuke you wouldn’t need the Maya integration. So triggering this code above is usually only done for Maya. E.g. in userSetup.py.

The pyblish_maya even has its own that you can use to help out with this. For maya to use a userSetup.py it has to be under your PYTHONPATH. So for Maya you can add that particular content there: pyblish-win\lib\pyblish-x\modules\pyblish-maya\pyblish_maya\pythonpath.

Note that this is quite deep within the repository because of how it has nested the submodules:

  • pyblish-win
    • pyblish-x
      • pyblish repositories, for example…
      • pyblish
      • pyblish-maya

Also see the answer to the next question.

what is differences between pyblish-win\pythonpath and pyblish-win\lib\pyblish-x\pythonpath

pyblish-win encapsulates pyblish-x. Think of it like this:

  • pyblish-x is OS independent as a distribution
  • whereas pyblish-win is windows specific.

As such the “windows” variant holds the OS independent contents of Pyblish.
Thus it pyblish-win contains pyblish-x. Currently I don’t think those pythonpaths differ much and pyblish-win is more of a container for the windows specific installer. Over time (with updates) differences could grow further of course.

Because of this “encapsulation” and how it was meant to ease updating through submodules in git you get some “inclusion” levels within the package itself which might be somewhat confusing. But it’s best to just think of it as if the “work has been laid out for you a bit”. Otherwise you’d have to manage all individual repositories of Pyblish yourself and need to know which are compatible with each other. This takes care of that for you.

In this case if you’re using Windows workstations there’s nothing wrong with using pyblish-win\pythonpath.

#16

Hello BigRoy, thanks

I’m testing pyblish on the studio server, so far i did these, please let me know what is wrong?

  • copy pyblish-win on server, with this path for example, \\studio\DFS\Pipeline\pyblish-win
  • in my user i have access permission to \\studio\DFS\Pipeline (full control) (except creating environment variable, that i don’t know this will create problem or not)
  • with admin user i created these environment variables on my user:
  • PATH : \\studio\DFS\Pipeline\pyblish-win\pythonpath
  • PYTHONPATH : \\studio\DFS\Pipeline\pyblish-win\lib\pyblish-x\integrations\maya

Questions:

  • Environment Variables are specific for user or workstation?
  • When i run python in CMD it does not detected as a command
  • because we have many users in the studio how can i set environment variable for all from my user or from server? i mean not setting one by one on every user. it is related to be too?

i should say that DFS is a shortcut like path that give the user a path from original source files.

i will ask many questions soon

thanks

#17

Both. It depends on whether you are setting system-wide environment variables or user environment variables. But this differentiation is only made when storing environment variables that persist. When you set them in a “wrapper script” it’s only local to that session. As explained above that’s recommended as you won’t be “messing” with any administrator stuff of managing the local workstation.

For python to be recognized as a command in a command prompt it’s executable must be on the PATH (or in the running directory cwd)

Think of a computer being completely oblivious to whatever is stored on the hard-disk. It doesn’t know anything. You’ve Python in a specific folder stored somewhere, it doesn’t know! The only reason why an OS will know a command is because it’s told where it is to run it. Similar to PYTHONPATH you’d have to tell it where to look.

For example if Python is in C:\Python27 then add it to your PATH.

set PATH=%PATH%;C:\Python27

Now if you run python as a command the command prompt will look under the current working directory whether it finds it. If not it’s assuming you’re running a command that’s on your path. So it will go through all the paths under PATH until it finds a python executable in there. If it doesn’t find it, the computer will tell you the command doesn’t exist.

Actually, this is what the “wrapper scripts” could be doing. They are not actually “setting” the environment variables that you see when you go to “Computer” > “Advanced” > “Environment Variables” stuff, because we’re not setting them “on the system” but in the session of the wrapper script and then running the implementation.

For example try a .bat file like this:

set PYTHONPATH=%PYTHONPATH%;\\studio\DFS\Pipeline\pythonpath
set PATH=%PATH%;\\studio\DFS\Pipeline\path
call cmd

You’ll have an open command prompt now that has that environment variables set. In there you could start Python, run pyblish, etc. if it’s exposed in the paths set in this file.

Now instead of starting cmd at the end imagine running maya at the end. Basically the command prompt would close, but Maya would start to run (within that environment!) Voila! You’d have your first launcher script to launches Maya. A wrapper script if you will that does all of it for you where you don’t have to “manage” the system itself.

Even better, when updating the environment all you’d need to do is change the script.

it is related to be too?

Somewhat yes. be is a framework that allows you to manage environment that you enter and run an application within it. Again, you’re not setting the environment variables for the user and storing them on the system. You’d be entering the environment each time to launch the application. And that’s when you are running into “wrapper scripts” to ease entering the environment you want for that application.

Even though setting variables on the system itself and then “living with that” works just fine, it’s much harder to manage over time with many many systems. Unless again that whole “hardcoding” of the environment is done with a management system or set of scripts… but again, when going that far? Why not manage the environment on much lower-level, like whenever wrapping the environment?

Again, the recommendation here is to manage this with “wrapper” scripts so that you maintain control in a much more logical sense instead of managing all environment variables system-wide per machine that you have operating. Especially since you might want to change environments depending on the project you’re running, and that’s where the real power comes in.

#18

Look, my pyblish-win is on server and i should set its path to client machine, so as in Manual Installation marcus just talked about pyblish-win/pythonpath PYTHONPATH, but when people install pyblish-win manually on server first they should set python27 in PATH env that system detect python.exe, then set PYTHONPATH.

i did it with a batch file that will set python27 of pyblish-win that is in server then set pythonpath folder of pyblish-win.

but i want to do that for all users and everytime they start their windows.

besides, i create PATH and PYTHONPATH directly in environment variables but pyblish does not work in maya and python does not detect in cmd again

what do you think.

#19

Not sure how to interpret this; but if by “people” you mean the other artists, then you’ve misunderstood what Roy was trying to say.

I’ll try and explain via an example. I’ll assume all computers on the network has access to \\studio\dfs\pipeline\.

  1. You’ve got it right, that you should install Pyblish manually, as per the instructions to \\studio\dfs\pipeline\pyblish-win.
  2. Next, whenever anyone runs Maya, they will need to run it via a .bat script written by you. This .bat script can look something like this.
set PYTHONPATH=\\studio\dfs\pipeline\pyblish-win\pythonpath;%PYTHONPATH%
start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*
  1. If you save this as maya.bat, then this is what artists then use to launch Maya. The first line “installs” Pyblish, and the second physically runs Maya.

The challenge for you, is to get everyone to launch Maya this way, instead of running it from the start-menu or by double-clicking on Maya files in Explorer and so on.

Alternatively, you could walk up to every machine on the network, and add Pyblish to their physical PYTHONPATH, but as you probably understand, that is quite tedious. So as another alternative, you could have a look at be, but I would suggest you try @BigRoy’s recommendation first of using wrapper scripts as they will make it easiest for you and for your artists to adapt to a new workflow.

Let me know if anything seems confusing and I’ll try to explain it differently, otherwise give this a try and let me know how it goes. You could, and should, try this on your own machine first, before you get artists to try it. It will make it easier for the both of you.

Good luck!

#20

Very good @marcus
thank you. i will do and i will use BE too,
and i will study wraooer scripts.
and i will ask many questions.

just how can i see syntax of this command start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*
i mean what is its flags

thank you

#21

i tested this code on a system that can read from \\studio\dfs\pipeline\pyblish-win

set PYTHONPATH=\\studio\dfs\pipeline\pyblish-win\pythonpath;%PYTHONPATH%
start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*

but does not work. and publish menu does not appear on file menu.
what do you think?

but i put \\studio\dfs\pipeline in PYTHONPATH too, in it i have a simple py file that print a statement
in maya when i import it, it works fine.

#22

This is correct. This particular example doesn’t add the default pyblish_maya's userSetup.py to the pythonpath. As mentioned earlier that’s an additional step you can take to use the default setup.

Does import pyblish_maya work in Maya?

If so, try this:

import pyblish_maya
pyblish_maya.setup()

This should also get you the menu item.
That is also what the pyblish_maya userSetup.py does for you on start-up.
For more details on adding that particular path, see this reply above.

Specifically:

#23

Hey Roy

it works with this bat file:

set PYTHONPATH=%PYTHONPATH%;\\studio\DFS\Pipeline\pyblish-win\pythonpath;\\studio\DFS\Pipeline\pyblish-win\lib\pyblish-x\integrations\maya;
set PYBLISHPLUGINPATH=\\studio\DFS\Pipeline\pyblish_plugins;

call "C:\Program Files\Autodesk\Maya2015\bin\maya.exe"
or
start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*

and publish menu added to file

#24

That’s it!

#25

Yes
So for starting Pyblish on maya (artist PC) from server installation we have a batch file named maya.bat with this content:

set PYTHONPATH=%PYTHONPATH%;\\studio\DFS\Pipeline\pyblish-win\pythonpath;\\studio\DFS\Pipeline\pyblish-win\lib\pyblish-x\integrations\maya;
set PYBLISHPLUGINPATH=\\studio\DFS\Pipeline\pyblish_plugins;

start "" "C:\Program Files\Autodesk\Maya2015\bin\maya.exe" %*

This is a method that i test and it works.

#26

Hi
I tried this way to run pyblish-standalone on network, it works fine.
create a batch file with this content:

@echo off
set PATH=%PATH%;\\studio\pyblish-win\lib\Python27;\\studio\pyblish-win\bin;
set PYTHONPATH=%PYTHONPATH%;\\studio\pyblish-win\pythonpath;\\studio\pyblish-win\lib\pyblish-x\integrations\maya; 
set PYBLISHPLUGINPATH=%PYBLISHPLUGINPATH%;\\studio\pyblish_plugins\_standalone;
pyblish-standalone --data layer9 helloWorld

i used collect_render.py as test in my plugin directory

#27

He men

as new update on my works on studio, i was working on Directory Structure of pipeline and trying to learn using be and i’m trying to connect pyblish and be and some custom tools together to work better in the pipeline.
just i have some questions about be that i don’t know where should i ask. @marcus :wink:

#28

I think pipeline specific discussions can emerge right here on the forum. I remember there already being a topic here on folder structures somewhere, have a look at that. Maybe there was even one for be.

Either way feel free to set up your own topics if you can’t find the information or topic you’re looking for.

#29

however this is related to work on network but sure
thanks