There are a many ways to go about this. Here’s an example of how that could work using cQuery and Open Metadata.
# Save an asset to disk
asset_name = "MyAsset"
current_file = maya.cmds.file(sn=True)
current_project = cquery.first_match(current_file, ".Project")
asset_dir = om.read(current_project, "assetDir")
final_path = os.path.join(asset_dir, asset_name)
# e.g. /server/projects/projectA/assets/MyAsset
I’ve done some draft work with lucidity to get some extra things in there. The changes so far are:
parse_iter() and format_iter() methods, instead of returning only the first match (#18).They are in my forked repository: https://github.com/BigRoy/lucidity.
Though note that my master branch is my development branch (since I’ve been changing as I go). The Template class barely changed and most of the changes are within parse/format methods in __init__.py or come in additional files, like schema.py and formatter.py.
All tests are still passing, except for the new schema yaml tests that I’ve been setting up. To get to passing those completely we would need to have optional arguments (??) and nested/referenced templates (#2) implemented in lucidity as well.
@marcus what I like about your setup is that it keeps data very close to what it belongs to, but how paths that are ‘to be constructed’ are defined seems cumbersome. Well, at least cumbersome at this low level. I wonder how you’d go about setting this up in a higher level python package that can manage the file structure easier. So more how do you see a full project being managed like that?
At this stage I feel that your solution reduces the freedom on folder structure more than what an extensive schema could offer. Because the data must be nested under its respective parent. In theory it’s confusing for the code (so the computer) that the asset could live in a separated work and publish folder, where your solution has a conflict. That would mean that same data is duplicated, which is exactly what you’re trying to avoid. Right?
For example
work/asset/hero/.Asset
publish/asset/hero/.Asset
Where would you store the data and how would you manage the file structure pattern?
On the other hand there’re other things your solution could solve, like:
It sounds like you’ve got the gist of how it works by now. We should definitely have a chat about how to solve these more specific issues if you’re interested, how about a separate thread?
I’m all up for these discussions.
Seems like this thread is a mixture of pure folder structure and managing a project solely through folder structure. The second one might mostly be irrelevant to the first. That is with tools like ftrack/shotgun or metadata on folders. At least it seems like you’ve got something nice to bring to the table. If you think some separation is in place feel free to kickstart a thread with a clearer separation. 
I’m trying out Lucidity and it looks like tokens cannot contain “/” character? So you have to extract the {project_root} out of the path before parsing it seems.
These would raise an error
path = 'P:/evo/scene'
token = '{project_root}/scene'
While this works.
path = 'P:/evo/scene'
token = 'P:/{project_root}/scene'Correct. You would have to use your own regex format for that token.
To do your own project_root you can do something like:
path = 'P:/evo/scene'
token = r"{project_root:(^[\w]*:*[\/]?(?:(?:(?<=[\/])[^\/]+[\/]?)*))}/scene"
Or for readability (to explain):
# The complex regex part that can match any root path (NT and UNIX)
regex_pattern = r'(^[\w]*:*[\/]?(?:(?:(?<=[\/])[^\/]+[\/]?)*))'
# The lucidity formatted *project_root* key
root = '{project_root:%s}' % regex_pattern
# Your token
token = root + "/scene"
We’re using this same regex pattern for the The Deal test project we’re doing with Pyblish Magenta
Thank you for sharing BigRoy. I’ll need to dig deeper into the tutorial 
I never really got the role of the regex here, could someone explain it to me in layman terms? Why can’t you have a slash?
Lucidity parses each token or key in a template as a name that does not hold any slashes. So the default key will never resolve to nested folders.
Otherwise any key would always match arbitrary amount of nested folders:
template = '{root}/dev/{asset}/test'
path = 'C:/projects/foobar/dev/character/maya/test
# {root}: C:/projects/foobar
# {asset}: character/maya
Though currently it does not match that by default, since {root} and {asset} can only ever be a single folder.
So we overwrite the regex by implementing our custom regex to allow the {root} to match anything up to that point. In theory you could have a very simple regex for {root} to match anything non-greedy up to that point. Something like {root:.+?} where the regex pattern is .+? to match any 1 or more characters in a non-greedy fashion.
Ah, I think I understand. So it’s only relevant to parsing a string into “tokens”, and not formatting a path from them?
Not entirely sure, but if lucidity would confirm that a path is always reversible (can be parsed) than formatting should obey to the same rules.
Theoretically it could confirm that it parses correctly after formatting. Actually now that I think of it, it should!
Got a link from a friend working on a similar solution as Lucidity.
Ade is tackling a slightly different (and important problem) I think - it operates at a higher level than Lucidity which is just a simple library at the end of the day.
Ade solves things like (correct me if I am wrong Lorenzo!) what permissions should a directory have in the hierarchy. One way you can do that is come up with a fun config file format, but an arguably nicer way would be to just setup up a real directory structure as a template and reference that, which I think is what Ade does. This is similar to project templates you see in development such as cookiecutter.
Great to see more solutions being open sourced!
Ah, that does sound interesting (and complex).
I’ll ping Lorenzo, maybe he can give us the tour.
For reference, you can also change the default placeholder regex per Template by passing a custom default_placeholder_expression parameter into the constructor.
So, here I am as well, in the arena .
Ade tries to solve the filesystem management problem using templates, rather than strings (strings are generated from the template folder by the end). It does also provide two commands : create, which will create all the needed folder on disk (file template included, let say for envs) , and parse, so you can do something like : ade parse /foo/projects/smurf/foo/bar and it will return {‘project’: ‘smurf’, ‘sequence’:‘foo’, ‘shot’:‘bar’}
example for create:
http://docs.efestolab.uk/ade/docs/build/html/tutorials/create.html
example for parse:
http://docs.efestolab.uk/ade/docs/build/html/tutorials/parse.html
As martin said, it also handle permissions, some more informations can be found here : http://docs.efestolab.uk/ade/docs/build/html/
we are using this tool on daily basis for all our clients, and so far has been proven quite robust and flexible.
Let me know if you have any question !
L.
p.s
Here some example of api usage :
http://docs.efestolab.uk/ade/docs/build/html/reference/examples.html#initialize-a-config-manager
I haven’t had much experience with Lucidity yet, is it also able to parse paths?
Yes. Lucidity both parses and formats paths, though it doesn’t parse a full schema into its own configuration file. As I understood that is what Ade is capable of?
Had a quick look at Ade but couldn’t get a quick grip it with the documentation. Hopefully I can find some time soon to have another look at it to get a feeling of the differences.
I’ll have a look at Ade too. It looks very interesting, however as you said, the docs are a bit confusing after a quick glance.