Multithreaded Texture loading

[Zimond]

This might very well be a problem that it totally on my side. I just would like to ask you if there might be concerns in general. ok uhm:

In the editor I am currently building I implemented a loading routine for textures when you switch game scenes or dragn drop textures from the Asset browser to a scene. Currently there is only a Skydome that receives these. The loading routine automaticly loads Normal and Paralax maps if files with specified suffixes exists.

I put this loading routine into another Thread so that loading can happen in the background without freezing the main application. I am of course aware that Threads are always a big can of worms and I spent the last hours trying to find out if in any case a Texture could be in a loading process WHILE the rendering process tries to access it. And I think that shouldnt be the case.

Oh, the problem is random access violations that only happen sometimes when I keep switching quickly between scenes, forcing constant texture clearing and loading. I failed to catch these exceptions. They don't seem to happen directly in my extra thread. Debug window only gives me gpu drivers as the location.

So long story short, what I wanted to ask:
Is loading textures (including special types like ambient and paralax) in an extra thread and rendering other stuff in the main thread, in general something that should work with Afterwarp? Or would you say that this is something that should rather be avoided?

[lifepower]

Afterwarp API calls that deal with graphics are partially thread-safe: when creating multiple devices - each can be created and used in a separate thread without any issues. Direct3D generally hes less issues with multithreading, but in OpenGL things will silently "not work". Objects that are not related to graphics, such as TSurface can created and destroyed in multiple threads without any issues, but objects that have "device" is parent, generally, need to be created within the same thread that created the device. There are some relaxed conditions on how you can use some of these objects once created, however.

The best way to deal with it is to do all the texture loading in a separate thread, including putting the "ready to use" image data in TSurface. Then, as last step, call Synchronize and inside that function create tan actual TTexture instance using previously loaded TSurface. In other words, if your main thread is T1, and secondary thread T2, the texture image is loaded and TSurface is created in T2, but TTexture will be initialized in T1; you can then release TSurface either in T1 or T2. In Delphi, this would be something like that:

var
  LSurface: TSurface;
begin
  // Note: this code is executed in another thread.
  
  LSurface := TSurface.Create('c:\images\mytexture.png');
  try
    Synchronize(
      procedure
      begin
        // Create texture from a previously loaded image data.
        LTexture := TTexture.Create(FDevice, LSurface);        
      end;
  finally
    LSurface.Free;
  end;
  
  // TODO: let the main/rendering thread know that texture can now be used...  
end;

P.S. The screenshot looks good!

[Zimond]

ah I see. This way the more time consuming loading of the file is still multithreaded. That should be easy to implement. Thanks once again.

P.S. The screenshot looks good!

It is going slowly but I told myself that as long as I make progress I will stick to it. I am still laying the foundation. The project treeview is now also completely drawn by hand, so I had to recreate a treeviews functions. That took some time but this way I can add little buttons next to the nodes like a switch or color wheel next to a light source item. Grapher, projected canvas' and mouserays are implemented, as mentioned a general texture loading and unloading, a project file layout, a class structure for the game scenes, automatic backups of project files, full DPI awareness of each window, you can snap them to the sides of another monitor.

I hope that in a few months that I can present a simple but functional 3d editor using the all the mesh primitives that Afterwarp can create.

[lifepower]

Yes, the only thing that goes in the main thread is uploading the texture image to the texture, which doesn't take much advantage of multi-threading anyway. Next major release, which will come with Direct3D 12, can leverage multi-threading better, but it still has long way to go.

Thanks for the screenshots, it looks quite promising!

P.S. Regarding UI, there is an Afterwarp UI system based on older AsphyreUI that is coming in next update (speaking of which, the update is already finished, I just need to update .NET portion), but it doesn't have a tree-view component.

[Zimond]

I just noticed that for the Paralax and Occlusion maps you can only create them from file not from surfaces.

TTexture.CreateFromFilesNormalsAndOcclusion
TTexture.CreateFromFileParallax

I guess I have to seperate these and always load them in the main thread?

[lifepower]

These functions are merely helpers so you don't have to create and work with surfaces:

1. Occlusion uses alpha-channel of "Normals" texture to tell how lit the pixel is: a value of 255 means fully lit, a value of 0 means fully dark. If you use just "Normals" texture, then its alpha-channel is set automatically to 255. So just iterate the pixels and put "occlusion" into alpha-channel. "CreateFromFilesNormalsAndOcclusion" basically creates a surface, sets its format to TPixelFormat.RGBA8 and does the aforementioned operation.
2. "CreateFromFileParallax" converts the file format to TPixelFormat.L8 and inverts it. So if you work with surface, you can just do this yourself.

In your multi-threading approach, since you already work with surfaces, there is no need for these helper functions. You should load the appropriate textures in a separate thread, perform the above conversions, then just create textures as in code I've shown above.

[Zimond]

perfect 👍 thanks once more

[Zimond]

sorry, me again.
Do I have to use a certain pixelformat for Normalmaps when I preload them into surfaces, too?

Pictures show a texture with a normal map (no ambient map). When I load these directly like I do in the preview window always, the effect is correct. But when I do the preloading into a TSurface then the result is off. Like the 3 normal "color" channels are mixed up.

[lifepower]

For normals, in most cases, you should use TPixelFormat.RGBA8, although other options would work too, like TPixelFormat.RGBA4 (which will reduce memory usage at considerable quality loss), or TPixelFormat.RGBA16 (better quality with higher memory usage). CreateFromFilesNormalsAndOcclusion does not modify RGB channels of the normal map, it is quite a simple routine.

By the way, for the purposes of debugging, you can try saving the texture that doesn't look right back to disk and inspecting it to figure out what is going on.

P.S. In an incoming update, there is a support for bump-maps in OBJ files, which can be optionally used instead of normal maps - the loader will create a normal map from a height based bump-map "on the fly", which, in my opinion, from practical stand-point, seems to be a killer feature as you won't need any external tools to generate normal maps anymore.

[Zimond]

Your tip was the right thing. The normal texture appeared to contain the albedo image.

And it was a really stupid mistake:

property Texture : TTexture read FTexture write FTexture;
property NormalTexture : TTexture read FTexture write FNormalTexture;
property ParalaxTexture : TTexture read FTexture write FParalaxTexture;

Yeah i know, facepalm myself already ^^
Now that works perfectly

[lifepower]

Looks really nice!