Fix default Y-axis direction

[averne]

When compiling with the newest dkA64 release, most deko3d programs render incorrectly. After confirming the issue on a fresh dev environment, I believe the problem stems from a compiler bugfix.

Screenshots

Simple triangle

Textured cube

Mesh loading and lighting

Deferred shading

Checking the code for dkCmdBufSetViewports, the deko3d version currently found in the pacman repos only checks DkDeviceFlags_YAxisPointsDown. The freshly compiled version checks for both this and DkDeviceFlags_YAxisPointsDown.

However to me the current default viewport settings don't make sense, as we have both an origin in the upper left and an Y-axis pointing up, and indeed forcing a downward Y-axis fixes the issues. In any case the code was implicitely relying on a compiler bug.

[fincs]

First of all, I think there is a bit of a misunderstanding here. deko3d defaults to storing images top-to-bottom (that is what UpperLeft vs LowerLeft is about). Currently in deko3d the Y axis in clipspace always points up, like in OpenGL (this has nothing to do with image layout). The issue you see has nothing to do with a compiler update, it's caused by a feature I started working on a few months ago that allows users to make it point down like in Vulkan. There haven't been any releases including this feature, and as you can see, it currently has a bug (oops).

[averne]

Ah I see, thanks for clearing that up. I didn't check whether there has been commits between now and the last deko3d release touching this stuff, so that's why I assumed the different assmebly resulted from a compiler bugfix.

[CyCle1024]

@fincs Hi, recently I stucked into a flip trouble when using deko3d. After reading some codes in this commit e4e89f4. I think I found the cause of this bug:

deko3d/source/dk_device.h

Lines 100 to 102 in 8cb7b4c

	// Rasterizer expects clip space +Y to point down. In UpperLeft mode with Y pointing up
	// (or LowerLeft mode with Y pointing down) we need to flip the incoming Y coordinate.
	bool viewportFlipY() const noexcept { return !isYAxisPointsDown() ^ !isOriginLowerLeft(); }

The implementation is not consistent with the comment. We intend to flip Y when UpperLeft mode with Y pointing up. But in this case, !isYAxisPointsDown() = true and !isOriginLowerLeft() = true, true ^ true = false. Same as LowerLeft mode with Y pointing down.

I think the fix is simple, here is my PR: #26.