{"id":10,"date":"2011-05-30T17:04:22","date_gmt":"2011-05-31T01:04:22","guid":{"rendered":"https:\/\/blog.sigmel.net\/?p=9"},"modified":"2011-05-30T17:04:22","modified_gmt":"2011-05-31T01:04:22","slug":"refactoring-an-asset-pipeline","status":"publish","type":"post","link":"https:\/\/blog.sigmel.net\/?p=10","title":{"rendered":"Refactoring an Asset Pipeline (Part 1)"},"content":{"rendered":"

Lost Project<\/strong><\/p>\n

I want to talk about a project that I planned a lot for but never got the chance to implement: Refactoring an Asset Pipeline.\u00a0 Granted, this isn’t particularly exciting stuff for most people, but it was going to drastically cut down on our asset creation times which would do nothing but good for our company and the iteration speed of designers and artists.\u00a0 However, it ended up not happening since the company changed its focus and we ended up parting ways.\u00a0 So, I have decided that I wanted to post instead about what were planning on doing, since I think it was a good design and in this way I sort of get to do the work by proxy.\u00a0 And maybe someone will even learn a thing or two about it and the world will be a better place somehow.\u00a0 Even if it is just a little.<\/p>\n

The Old Way<\/strong><\/p>\n

On the project we wrote our own engine and the tools to support it.\u00a0 We had very short development cycles so stuff had to be done quickly.\u00a0 I was responsible for designing and writing a few of the asset pipelines, starting with the texture and shader pipelines, while the model\/scene pipeline was handled by the senior programmer.\u00a0 This was my first time ever writing pipelines of this sort from scratch, but I think I did a decent job overall trying to make things flexible and quick.\u00a0 Granted, I would do it a bit differently knowing what I know now, but that is part of the reason for writing about this in the first place!<\/p>\n

We were developing for multiple platforms, and it was decided to create a separate pipeline for each platform so that you only had to build the assets for the platform you are working on.\u00a0 This was accomplished by having a central executable which was responsible for loading and executing the corresponding DLL for that asset and platform.\u00a0 All the common code for the various asset pipelines was shared and the platform specific code put in separate files to keep the code maintenance to a minimum.\u00a0 We standardized on a few asset input types so that we didn’t have to write code to support everything under the sun.\u00a0 For example with textures we were only going to accept DDS (Direct Draw Surface) files since they were very flexible, easy to read, and there were plugins for Photoshop that would read and export them.\u00a0 Granted, this didn’t stop the artists from later wanting support for more image types, but then it was a simple matter of just running some tools to convert the JPEG, PNG, Targa, or whatever else to DDS before running it through the pipeline and into the game.\u00a0 For model data we chose to go with Collada and shaders were simply handled by FX (plain text) files split into separate vertex and pixel shaders.<\/p>\n

Asset Dependencies<\/strong><\/p>\n

For the first project it was decided that scene information (instances and transformations of them) would be combined with the model information.\u00a0 Additionally, materials (shader selection and parameter settings) would also be unique to each mesh (which belong to a single model).\u00a0 This means that we effectively only had three major assets from the rendering pipeline point of view: models, textures, and shaders.\u00a0 While this decision reduced our flexibility somewhat, it allowed us to get the game started faster and artists and designers to see their stuff in game in a shorter period of time.\u00a0 Eventually this would lead to some pretty heavy bottlenecks considering that changing a texture or shader parameter would require the entire scene be rebuilt.\u00a0 However, some real-time communication with the game was set up so that changes could be seen immediately for tweaking purposes to try to improve the workflow.<\/p>\n

After the first game shipped, it was petitioned by the artists to get the materials split out from the meshes so that they could be shared.\u00a0 In addition, our (now much bigger) tools team created a material editor since the one that I made that ran in XSI had some shortcomings (mainly due to weird artifacts of using XSI’s realtime shader API) plus it never really achieved the artists desire of having XSI as a decent previewer for their assets (which it never would be able to since it wasn’t running with our lighting, shadowing, or other visual effect systems).\u00a0 So more effort was also put into beefing up our in-game viewer instead which could be controlled by the material editor tool<\/p>\n

Next Time<\/strong><\/p>\n

This is an overview of what we started with when we set about completely redesigning how assets would be built.\u00a0 It could take several hours if a new build had to be made of our game, which led to many very late nights for our build engineers and those getting the build out.\u00a0 Not to mention the wasted time of the artists and designers due to the inefficiencies present in the system anyway.\u00a0 Next post will focus on how we were planning on tackling the various issues listed here.<\/p>\n

Also will try to get a few visual aids to help show how things used to work, so check back for those and if you have any suggestions on (free) tools that I can use besides Paint.NET, I’d love to hear it.<\/p>\n","protected":false},"excerpt":{"rendered":"

Lost Project I want to talk about a project that I planned a lot for but never got the chance to implement: Refactoring an Asset Pipeline.\u00a0 Granted, this isn’t particularly exciting stuff for most people, but it was going to drastically cut down on our asset creation times which would do nothing but good for […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-10","post","type-post","status-publish","format-standard","hentry","category-programming"],"_links":{"self":[{"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=\/wp\/v2\/posts\/10","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10"}],"version-history":[{"count":0,"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=\/wp\/v2\/posts\/10\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.sigmel.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}