https://openbrush.app/ logo
New exporter and replacement for Unity Toolkit/SDK
# unity-integrations
a
I've started work on both sides of the import/export pipeline. Looking for testers...
a
Ya, I would love to look at that! Post a link when you want me to check it out
a
Yeah. I'll do a bit of a write up. Early days but there's definitely some value for some specific workflows.
Roughly speaking it should improve the following: 1. Optionally preserving more structure on export (layers > groups > strokes) 2. The opposite (a single mesh for all strokes with a given brush - without the splitting into batches that happens now) 3. Exporting a broader range of stuff (models, images - and new stuff like text widgets and audio) 4. Eventually - animation export 5. Better fidelity when exporting to apps that don't have special support for Open Brush (Blender, Banter, whatever...) 6. Support for URP and HDRP
Current drawbacks: 1. Particle brushes are broken 2. No audio reactivity support 3. You can't export and preserve groups unless you also export and preserve each brush stroke (i.e. lots of separate objects)
Probably more. Need to do more testing.
Currently export options are configured in Open Brush.cfg. Example:
Copy code
json
{
  "User": {
    "Author": "XXX"
  },
  "Video": {
    "Resolution": 1280,
    "OfflineResolution": 1920,
    "FPS": 30,
    "OfflineFPS": 60,
    "ContainerType": "mp4",
    "CameraSmoothing": 0.98,
    "Encoder": "h.264",
    "SaveCameraPath": true,
    "FOV": 80
  },
  "Flags": {
    "LargeMeshSupport": true,
    "PostEffectsOnCapture": true,
    "ShowWatermark": false,
    "ShowHeadset": false,
    "ShowControllers": true,
    "SnapshotWidth": 4096,
    "SnapshotHeight": 2160,
    "FOV": 55,
    "DisableAudio": false,
    "UnlockScale": true
  },
  "Export": {
    "ExportBinaryFbx": true,
    "ExportFbxVersion": "FBX201400",
    "ExportStrokeMetadata": true,
    "KeepStrokes": false,
    "KeepGroups": false,
    "Formats": {
      "fbx": false,
      "glb": false,
      "newglb": true,
      "json": false,
      "latk": false,
      "obj": false,
      "stl": false,
      "usd": false,
      "wrl": false
    }
  }
}
Note the new stuff under "Export" - 1. formats can be individually chosen. ("newglb" is the new one with the new features) 2. LargeMeshSupport under Flags allows a layer to combine huge numbers of brush strokes into one mesh 3. KeepStrokes and KeepGroups do the opposite - exports are broken into lots of specific objects to allow easier animation etc 4. ExportStrokeMetadata is similar to the old ExportStrokeTimestamp setting. Rather than adding the metadata into the mesh itself, it's stored as an "Extra" field in the glb. The new Unity import reads this and adds a component to each mesh object in Unity for ease of access.
Bit more info on the PR (although that is a bit out of date at the moment): https://github.com/icosa-foundation/open-brush/pull/573
Thanks to @Nick for his work on the new shader graph shaders and @seethrough for his help with the config file options
Next priority is fixing particle brushes. I'm hoping that even apps that don't have any specific support for Open Brush exports can display these in a slightly more accurate way than was previously possible.
a
So it will pull the information out of unity instead of the game itself? How are layers impacted?
Will we be able to use shadergraph effects with this?
a
The exported glb should (eventually) be an accurate representation of what you see while sketching (up to the limits of what GLTF currently supports)
Layers will be respected. Groups and strokes are optionally also respected.
> shadergraph effects Depends what you mean 🙂
a
I'm thinking mostly in terms of materials. Like can we edit the output in unity or is it going to be a passthrough kind of export?
a
Well - give it a go...
It's certainly easier for most people to edit a shader graph than the hlsl code.
but the shader graphs are still fairly complex - because they attempt to reproduce the original shaders faithfully.
Personally - I find code easier to follow that a bunch of boxes and lines - but hey. What do I know...
a
Well, you've got a lot of practice at that form. Lots of people working with spaghetti, visual form so maybe more intuitive for artists.
Just trying to get a sense of the potentials of what can be done this way. Like can I make a shader and/or texture and apply it at some stage in this, or does it come out as a prepackaged export. What are potential limitations or advantages of doing it this way?
aside from those you've mentioned
Expect I'll h ave better questions after I crack it open, downloading now. Not sure how much longer I will stay up to night, but definitely get it installed at least
a
> visual form so maybe more intuitive for artists. Get away with that "left brain right brain" crap. It was debunked a couple of decades ago. 😉
a
Different processing regions in different parts of the brain. I'd think it would be easier for visual thinkers in general, and certainly much faster to get the hang of the basics since it's fairly constrained. If the cord doesn't fit in the plug then you can't do it. Simplifies things
Strengths and weaknesses in it. Easier to rearrange/tune/adjust things and see instant feedback. Harder if you have a lot of things to wire together, becomes a kind of visual maze
Ugh: Didn't update the config file. Assumed it was built in. Will retest.
TESTING RESULTS (using default newglb config above) Initial Test (draw and save): 1. Waveform particle has an issue, doesn't appear to draw or is widely offset from the main scene There is a brush stroke animation but it doesn't appear to be part of the scene and does not show in the camera viewfinder This is an issue with the original brushes. Is either new or I've fixed it in mine because I'm pretty sure I've used that 2. Attempted to export (success) Folder "newglb" created and contains a glb Import Test (Test2) 1. glb imports successfully (with exception of waveform particle, as expected) 2. particle brushes are frozen. They render but don't move 3. Transparency is retained 4. Added text 5. Successfully exports Reimport Test (Test3) 1. Loaded the model exported from test 2 2. Imported a new model (so both old and new) 3. Added new text 4. Exported 5. Reloaded the resulting combined model 6. Successful (same result as test 2). Confirms that models will combine correctly with or without additional text using the current system
Photo appears fuzzy, is there a reason the camera fov is set at 55 and not 80?
Next tests will be attempting imports of these files into other programs
a
I'll need too handle reimporting Open Brush exports back into the Open Brush better at some point. It's hard to decide whether we should swap back in our shaders or not.. On one hand it's the only way to maintain appearance - but on the other it will oveerwrite any intentional user changes to materials.
So I guess it needs to be an option... There's also the question of round-tripping with something like Blender in the middle. How much metadata is preserved? I'm not relying on metadata for matching materials so that should be safe. But there's metadata on stroke timestamps and groups
a
Interesting. I will test the glbs in other software and see what the exports look like there. Seems that it would be best to have the option on import, unless there's a better way to handle internal re-merges. Maybe an option to "repaint all openbrush brushes", So that it's not trying to reinterpret external brushes and is only redoing our brushes if we want them to? Have to think about that some. :☕:: Other programs would rewrite the metadata so its not usable?
a
dunno yet
a
Ha, well I guess we'll see. I'm not great with blender but I'll try stuff with it and see what happens
Further Import Tests. Snapshots from imports in ShapesXR, Blender, AdobeStager, Babylon Sandbox, and the windows preview image for the glb I've also included hierarchy info where available, The text file is the report from SandboxJS, which observes that normals aren't found Pretty much everything picks up the Albedo texture, either as the color of the brush used or the mesh itself (albedo) swatch (stager) Transparency varies a bit, depending on the program that's doing the reading. Imports are off center in every case, although very possibly that's exactly how I built the scene Image is brush panels in sequence https://cdn.discordapp.com/attachments/1199435833183326260/1199847895160070234/GlbImportPhotos.zip?ex=65c4086e&is=65b1936e&hm=04bfc71c28afcb877a86a4a586c99c546a089be087e164f5f170b9990d6ca2dc&
Super happy they export. Looking forward to seeing progress on this branch. Have to check, but I think that blue brush has a custom shader on it too, which is impressive
Getting this error on custom brushes, which is interesting. (they more or less export)
Copy code
Failed to rename node brush_equinox_0 based on brush guid: Object reference not set to an instance of an object
UnityEngine.Debug:LogError (object)
TiltBrush.OpenBrushExportPluginConfig:AfterNodeExport (UnityGLTF.GLTFSceneExporter,GLTF.Schema.GLTFRoot,UnityEngine.Transform,GLTF.Schema.Node) (at Assets/Scripts/Export/OpenBrushExportPlugin.cs:158)
UnityGLTF.GLTFSceneExporter:ExportNode (UnityEngine.Transform) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:1128)
UnityGLTF.GLTFSceneExporter:ExportNode (UnityEngine.Transform) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:1121)
UnityGLTF.GLTFSceneExporter:ExportScene (string,UnityEngine.Transform[]) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:988)
UnityGLTF.GLTFSceneExporter:SaveGLBToStream (System.IO.Stream,string) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:714)
UnityGLTF.GLTFSceneExporter:SaveGLB (string,string) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:665)
TiltBrush.Export:ExportScene () (at Assets/Scripts/Export/Export.cs:307)
TiltBrush.SketchControlsScript/<>c:<ExportCoroutine>b__316_0 () (at Assets/Scripts/SketchControlsScript.cs:3969)
TiltBrush.OverlayManager/<AsCoroutine>d__67:MoveNext () (at Assets/Scripts/OverlayManager.cs:622)
TiltBrush.OverlayManager/<RunInCompositor>d__61:MoveNext () (at Assets/Scripts/OverlayManager.cs:416)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)
So looks like there's either a naming syntax requirement or there needs to be a way to add to the guid list or assign one? Any ideas?
a
You'd have to give me more detail on when this error happens. Custom brushes from where?
I've not even considered how this integrates with the Edit Brushes branch - and if you mean "brushes modified directly in Unity" then all bets are off. It will fall back on whatever UnityGLTF does by itself.
I'll probably change that LogError to LogWarning - yellow rather than red.
I'm currently matching on Brush (durable) Name when you import. I've got a few options: 1. Use material name to match up with brush name (downside: possibility of unintended matches if material names match with Open Brush names) 2. Name the material after the brush guid instead and match on that (downside: ugly guids cluttering up material names) 3. Don't match via the material name - store the brush guid as GLTF metadata (downside: roundtripping via other software might strip off metadata)
I'll probably either do with (1) but with s simple prefix like "ob-" to reduce the likelihood of accidental matches. Or else (2) but I'll use "brushname (guid)" or similar.
Need to fix a few brushes. Toon is broken on the built-in pipeline. Light is invisible from behind. I've got a potential fix for all the particle brushes...
a
Naturally, I was curious. Wanted to see if it could handle that at all, since previously couldn't export those as glb at all. Used in unity brushes, because that's a lot faster than trying to integrate externally modified ones. Apparently the fallback seems to work reasonably well, since they exported I think one of the first two naming approaches is best. Ideally we minimize conflicts with other software out the gate. (1) The first has definite advantages in readability, the second may have advantages when it comes to user created brushes and the display (2) ugly and a bit of work to change, but also very easy to check for potential conflicts or duplicates. (X) Unclear regarding dashes or underscores, potentially a problem area if other software uses those as separators of some kind (string parsing?) Camelcase like ObBrushName might be a better choice for prefix? Drawbacks (personally) would be in backwards compatibility, if its using the hardcoded name. Although maybe possible to do a string substitution in the save files? Haven't tried importing my current glbs into existing sketches, depending on how that works, could be a workaround? Presumably would affect modified brushes as well, unless we use the name format that is currently being used there. Regardless, once we establish a naming format, then I can update things going forward. I'd love to have a structure where I could package brushes individually or in small sets so they could be added using the package manager, without overloading their device memory. Some thinking to do there, but long term might be a good approach so we can incorporate new brush designs without necessarily hardcoding in everything internally.
:: next thing I'm probably looking at is seeing how models, videos, images work in practice
:: Also we currently have a format defined in the modified brushes system, so might be pros/cons to going with something like that?
Whatever you think is best, we should do. Just trying to anticipate potential issues (ie: extra work) down the road
Also occurs that brush names being readable might be an advantage for people working in blender or unity and trying to find things in menus by name
a
Just updated both the new exporter branch and the new unity tools repo
previously - stuff in your scene was left broken after an export. that should be fixed (althought i haven't tested properly yet so other things might now be broken)
The tools now have a UPM branch which is automatically kept up to date so you can always install via the package manager using this url: https://github.com/icosa-foundation/open-brush-unity-tools#upm
No more .unitypackage versions!
You will need to also add the package for UnityGltf until there's an official UPM release of that.
This release also fixes Toon (well - it looks wrong but at least it's not invisible) and Light (which was invisible from one viewing direction)
Oh - and Mylar meshes are the correct shape (although currently we don't have the right shader for Mylar)
a
So a bit unclear about the unity package manager in this context. I did download the new exporter branch Do I need to use the UPM tool, or is that a separate thing? TESTING REPORT Tested the new exporter branch. Results summary 1. Images import successfully 2. Videos import as still frames (as an image, no ability to play from model, just appears to bring the snapshot) 3. Waveformparticles fails to export (error code below) 4. Bubbles and Hypergrid show as present but not visible on import 5. Point clouds fail to export (pretty sure its on the export since they don't show in the windows preview) 6. Animated models import, although animations are lost (expected for now) 7. Imported models (external) retain their textures 8. Animated brushes don't work (expected for now) Not tested 1. groups 2. skyboxes (presumed won't import since they should be tied to the model, didn't test for issues yet) 3. any impacts of scene lighting 4. fbx models (possibly tested, not verified, think I used just glbs) Observations Generally quite useful for anything static, within the above limitations, don't see any obvious new issues that weren't present previously may need to take a closer look for strip transparencies, although they generally look correct
Copy code
Failed to rename node brush_waveformparticles_0 based on brush guid: Object reference not set to an instance of an object
UnityEngine.Debug:LogError (object)
TiltBrush.OpenBrushExportPluginConfig:AfterNodeExport (UnityGLTF.GLTFSceneExporter,GLTF.Schema.GLTFRoot,UnityEngine.Transform,GLTF.Schema.Node) (at Assets/Scripts/Export/OpenBrushExportPlugin.cs:218)
UnityGLTF.GLTFSceneExporter:ExportNode (UnityEngine.Transform) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:1128)
UnityGLTF.GLTFSceneExporter:ExportNode (UnityEngine.Transform) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:1121)
UnityGLTF.GLTFSceneExporter:ExportScene (string,UnityEngine.Transform[]) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:988)
UnityGLTF.GLTFSceneExporter:SaveGLBToStream (System.IO.Stream,string) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:714)
UnityGLTF.GLTFSceneExporter:SaveGLB (string,string) (at Library/PackageCache/org.khronos.unitygltf@4ebe065f27/Runtime/Scripts/GLTFSceneExporter.cs:665)
TiltBrush.Export:ExportScene () (at Assets/Scripts/Export/Export.cs:307)
TiltBrush.SketchControlsScript/<>c:<ExportCoroutine>b__316_0 () (at Assets/Scripts/SketchControlsScript.cs:3969)
TiltBrush.OverlayManager/<AsCoroutine>d__67:MoveNext () (at Assets/Scripts/OverlayManager.cs:622)
TiltBrush.OverlayManager/<RunInCompositor>d__61:MoveNext () (at Assets/Scripts/OverlayManager.cs:416)
UnityEngine.SetupCoroutine:InvokeMoveNext (System.Collections.IEnumerator,intptr)
Finding this is really resilient. Even when running into unexpected content, missing brushes, brushes with broken shaders and so forth, it's still successfully importing/exporting whatever it can actually handle, rather than failing. When animations happen, it may become time to start constructing a monument in your honor
a
lol. i think animation support for imported models is just a checkbox. i'll investigate
a
Easiest monument ever!
Seriously, this thing is amazing
9 Views