Back in my initial post on the stencil buffer approach I had used a simple heuristic for setting the correct render queue. As I feared it didn’t hold up for very long. There are not only 2 layers that need to be set. Let’s look at the picture below:
What’s going on here? The back portal is not properly overlayed by the room geometry, it renders also objects in front of the actual portal creating distortions, it sometimes blends into the content of the portal in the front and more. To wrap my head around the issue, I first created a debug tool allowing me to set portal and geometry render queues in each room separately at run-time.
After a lot of tinkering and verification through scanning all kinds of forums (like this really good thread) I finally got a working solution accepting some world building constraints:
- There are what I call “main portals” which are always at the far side of a room pointing inwards.
- “Reverse portals” are created automatically allowing the player to go back to a room he just came from.
- There is no support for recursive portals for now, meaning looking through a portal seeing another portal. Recursion is something discussed nicely in this blog post.
Accepting the constraints above rooms can be rendered correctly in four steps.
- Main portals
- Reachable “Main” rooms
- Reverse portals
- Rooms reachable through reverse portals