Sharplish - Porting Pyblish to C#

#lish

Having chatted with @mrobbins, this thread is about the porting of Pyblish to C# - hence the name - for use with Unity and the C# APIs of Maya and 3ds Max.


Introduction

The entirety of Pyblish - base library, integrations, graphical user interfaces and more - is written in Python. For use with C# we’ll need a version of pyblish-base written in C#.

At the end of this journey, I expect a fully functional port of pyblish-base, running alongside the pyblish-qml GUI.

Why?

Both Unity, Maya and 3ds Max grant access to their APIs via Python, so why bother with C#?

In the case of Unity, the Python API is both unofficial and exposed via IronPython - a C# implementation of Python. It lacks aspects of Python currently used in pyblish-base, although at this time it is uncertain what and how much - for example the dependency six won’t import successfully.

Porting Pyblish to C# then would expose the entirety of Unity’s API to Pyblish plug-in developers.

The benefits of C# in Maya and 3ds Max are yet to be investigated, although at first glance they seem more performant. I’d imagine that if a shop it Unity based, they may have more developers writing C# and have most of their code base written in it as well, favouring a single language across all content creation software they use.

Where else is C# favourable?

What do we need?

We have a few challenges ahead of us, but if we start small and work our way up I’m certain we’ll succeed. For starters, if we can find a way to port this snippet, we’d be well on our way.

Knowing little about C#, I’ll provide a step-by-step guide on core functionality as we go, assuming it can do nothing of what Python does and work our way towards full feature paridy.

We will need:

  1. Functions
  2. Functions that take arguments
  3. Passing functions as arguments
  4. Functions that can take functions are arguments
  5. A key:value, dictionary/map type object, e.g. {"hello": "world"}
  6. A list/array type object, e.g. [1, 2, 3]
def process(Plugin, instance):
    """Process plugin, and produce result"""
    Plugin(instance)
    return {
        "plugin": Plugin,
        "instance": instance,
    }

def publish(plugins, context):
    """Primary publishing mechanism"""
    for Plugin in plugins:
        for instance in context:
            process(Plugin, instance)

def Plugin1(instance):
    """An InstancePlugin"""
    print("Plugin1 ran")

plugins = [Plugin1]
context = list()
instance = list()
context.append(instance)

publish(plugins, context)

Development

For starters, you are invited to share snippets of code right here in this thread. We can test out our code directly via a C# development environment, or via Docker like this.

$ cd path/to/script.cs
$ docker run -ti --rm -v $(pwd):/csharp ubuntu
$ apt-get install -y mono-complete
$ mono script.cs

mono is the C# equivalent to python, and script.cs is our script. This will be a live connection, so as you save your C# script locally, re-run mono script.cs to run it.

Just wondering why porting to C# and not JavaScript?

Unity uses JavaScript as well, and it would work with the Adobe products more natively.

Good question. @mrobbins, what do you think?

@marcus @tokejepsen C# is the more common coding language when it comes to Unity, it is also the language I use.

Javascript in Unity is not real javascript, it’s a variation based on an old version of javascript. People have come to call it UnityScript. There are certain things in C# that you can’t do in UntiyScript such as switch statements. You also don’t have a direct connection with Unity libraries so it’s more time consuming to write.

It would be much simpler and faster to write the GUI code we may need in C#.

We could do javascript for this so it could be used in multiple software’s but if we were to have this version open to the public to help( if that was going to happen), then people would be more inclined to do that in C#.

Ok, C# it is.

There’s nothing stopping us from having bindings for both; although it would make sense to start off with one. What we could do is pay special attention to and document in full the process of porting, such that it would make it easier to continue porting to additional languages. I’d imagine the Python flavour to need various updates as we port, as we work to flesh out ambiguities currently present in the implementation.

@mrobbins are you able to post a C# equivalent of the above snippet? That would get us started in the right direction.

So I talked to my Technical Director at my work about our plan to make a C# version of pyblish and he suggested this plan.

-Open pyblish from unity
-Run the programs
-output an object from pyblish
-send that into pyro4
-that talks to ironpython
-that triggers a C# class to run a check inside unity that runs like the program we ran in pyblish
-that sends an object back through to pyblish

not sure if this is too confusing or not but that’s what were thinking of having.

Everything you have that is pyblish stays the same and there is a C# script that gets inherited for people to write their own programs. All we need is a way to open the correct pyblish from unity(I currently have pyblish-lite that opens) then we have similar python programs to C# scripts and be able to send this information through ironpython and pyro back and forth from Unity and Pyblish.

Thoughts?

That’s almost exactly what I have in mind as well. Except in place of Pyro, which is an RPC library for use over a network connection (including localhost), we will be using standard input/output directly. This would allow us to skip the part where we talk to IronPython, IronPython talks to C#, and cut straight to C#.

You and your TD can find more information about this here.

Sounds like we’re on the same page!

@marcus does that require PyQt5 including python3? Or can we use PySide with python2.7 ?

It requires a standalone version of Python, either 2 or 3, along with PyQt5.

I’m running into a couple problems trying to test out pyblish_qml

    C:\Windows\system32>python -m pyblish_qml --demo
Traceback (most recent call last):
  File "C:\Python\Python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python\Python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python\Python27\lib\site-packages\pyblish_qml\__main__.py", line 6, in <module>
    from . import app
  File "C:\Python\Python27\lib\site-packages\pyblish_qml\app.py", line 13, in <module>
    from . import util, compat, server, control, rpc, settings
  File "C:\Python\Python27\lib\site-packages\pyblish_qml\control.py", line 11, in <module>
    from . import util, models, version, settings, rpc
  File "C:\Python\Python27\lib\site-packages\pyblish_qml\models.py", line 94, in <module>
    class PropertyType(QtCore.pyqtWrapperType):
AttributeError: 'module' object has no attribute 'pyqtWrapperType'

I tried testing out the installation of pyblish_qml and get this output

Solved privately, the problem was an out-dated version of pyblish-qml with the most recent version of PyQt5.

Hey so i’m taking another stab at converting that snippet of Python from the sharplish thread now that I have a better understanding. Still missing a bit, could you run me through this function so i understand what the variables are exactly.

def process(Plugin, instance):
    """Process plugin, and produce result"""
    Plugin(instance)
    return {
        "plugin": Plugin,
        "instance": instance,
    }

I’m still somewhat new to Python so i might be missing some syntax here. What exactly is happening with Plugin(instance)? What type is being returned here?

What type is context and instance in that snippet?

Excellent!

This is a plug-in being run. A plug-in, in Pyblish for Python, is written as a subclass of pyblish.plugin.Plugin and look like this.

from pyblish import api

class MyPlugin(api.ContextPlugin):
  def process(self, context):
    print("Do something")

We will need a similar mechanism for C#, where the user subclasses a C# class and implements his validation or export or whatever.

The type returned here is a dict, or “dictionary” containing two keys, whose values are Plugin and Instance. Plugin is a plain Python class, whereas Instance is a subclass of a standard Python list.

It doesn’t have to be those types, however. I think we can make life simpler by instead passing both Plugin and instance as just strings.

def process(Plugin, instance):
    """Process plugin, and produce result"""
    Plugin(instance)
    return {
        "plugin": "name of plugin",
        "instance": "name of instance",
    }

Okay so here is how I interpreted the Pyblish Python snippet from a few days ago. A Sharplish base class, Plugin base class and a plugin script.

Sharplish base;

public class BaseClass : MonoBehaviour {

    public IDictionary<string, List<string>> Process(Plugin in_plugin,     List<string> in_instances)
        {
            return new Dictionary<string, List<string>> {
                {
                    in_plugin.name,
                    in_instances // this is a list
                }
            };
        }
    	
    	void Publish (List<Plugin> in_plugins) {
    		
            foreach(Plugin pl in in_plugins)
            {
                Process(pl, pl.instance);
            }
    	}

        void RunIt()
        {
            Publish(plugins);
        }

        public List<Plugin> plugins = new List<Plugin>();
    }

Plugin base;

public class Plugin : BaseClass {

    public string name;
    public List<string> instance;
    public List<string> context;

    public virtual void SetupPlugin()
    {
        name = "";
        instance = new List<string>();
        foreach(string st in instance)
        {
            context.Add(st);
        }
    }

    private void Start()
    {
        base.plugins.Add(this);
    }
}

Plugin1 ;

public class Plugin1 : Plugin {

    public override void SetupPlugin()
    {
        base.SetupPlugin();
        //set name
        //set list of strings(instances)
    }
}

This is just a first pass at it. So let me know if I misunderstood anything.

That’s great! Could you guide me through running this code?

I’ve got a C# interpreter set-up under Linux, like this.

$ cd path/to # script.cs
$ docker run -ti --rm -v $(pwd):/csharp ubuntu
$ apt-get install -y mono-complete
$ mono script.cs

Where script.cs is the name of the file to run.

How can I save your code and run it?

ps. Here’s a tip on code formatting. If you put fences around your code, and indicate a language, the forum will syntax highlight it for you.

For example

```cs
public class Plugin1 : Plugin {
```

Becomes…

public class Plugin1 : Plugin {

I’m not entirely sure how this will run outside of unity. I’m on windows so I can’t test that interpreter.

I’ve never ran C# outside of Unity. So if that’s what were wanting to do then we are going to need more code. What is the end game of this step? That will help figure out how I need to structure this, to run it outside Unity.

What we need, is a function that the end user can run to find and run Plugin1.

Something along these lines (Python).

import sharplish
sharplish.publish()

Ideally executable from the command-line, for testing purposes, but otherwise within an interpreter, such as Unity. I just need some way of running it here as I don’t have access to Unity.

How can I run this within Unity?

I’ve had a closer look at C# just now and realised it does not (cannot?) run scripts the way Python does. That is, files aren’t interpreted but must instead compile to an .exe first - even on Linux where you run the .exe with mono, a cross-platform open-source implementation of the C# compiler.


helloworld.cs

// A Hello World! program in C#.
using System;
namespace HelloWorld
{
    class Hello 
    {
        static void Main() 
        {
            Console.WriteLine("Hello World!");
        }
    }
}

Compile and Run

csc.exe is the C# compiler included with Visual Studio.

$ csc helloworld.cs
$ helloworld.exe
Hello World!

With this in mind, let me refactor the above snippet of code into something more suitable for C#.

publish.py

def process(Plugin, instance):
    """Process `instance` with `Plugin`

    Standalone function meant to take a `Plugin` in combination with an `instance`
    in order to produce a "result".

    """

    success = Plugin(instance)

    return {
        "success": success
    }


def ValidateName(instance):
    """Children must end with _PLY"""
    return all(i.endswith("_PLY") for i in instance)


def main():
    """Primary publishing mechanism"""

    # Plug-ins are normally discovered from disk, as individual Python
    # files stripped of the class they contain.
    plugins = [ValidateName]

    # The Context is where the content and result of publishing is held
    # In Python, this is implemented as a list, where each child is
    # an Instance. A new context is instantiated per publish.
    context = list()

    # Instances are also implemented as lists, where children represent
    # individual objects in a host, such as nodes in Maya or Nuke.
    instance = list()
    instance.append("polyCube_PLY")

    # Instances are always parented directly under the current context.
    context.append(instance)

    # Here each plug-in is run to produce the end-result.
    # Normally, the result of each plug-in is taken into account here,
    # such as to determine whether it has failed, but not for now.
    for Plugin in plugins:
        for instance in context:
            print("Processing %s(%s)" % (Plugin.__name__, instance))
            process(Plugin, instance)

    return True

if __name__ == '__main__':
    main()

Usage

$ python publish.py
Processing ValidateName(['polyCube_PLY'])

Does that make more sense?

Ping @mrobbins, just checking in on your progress? Let me know if there’s anything else you need in order to continue with this!