path: root/doc/dev notes.txt

These are some development notes I've put together that would be of great
aid to those willing to contribute to the PSn00bSDK project. Many of these
came from my own experience dealing with the PS1 at low-level when I ran
into some unexplained quirks while some are from disassembly observations
and clarification of existing documents. More entries will be added when
I run into more previously undocumented quirks in the future.
	- Lameguy64
	
* When calling C functions (ie. BIOS functions) from assembly code you'll
need to allocate N words on the stack first when calling a function that has
N arguments (addiu $sp, -(4*N) where N = number of arguments of the function
being called) even if the arguments are on registers a0 to a3 and the C
functions don't always use the space allotted in stack. When calling a
function with a variable number of arguments (printf) always allocate
16 bytes of stack.

* Hooking a custom handler using BIOS function HookEntryInt (B(19h), known
as SetCustomExitFromException in nocash docs) is only triggered when there's
an IRQ that is not yet acknowledged by previous IRQ handlers built into the
kernel. This is also the best point to acknowledge any IRQs without breaking
compatibility with built-in BIOS IRQ handlers and is what the official SDK
uses to handle IRQs. To make sure this handler is triggered on every interrupt
you must call ChangeClearPad(0) and ChangeClearRCnt(3, 0) (which are functions
(B(5Bh)) and C(0Ah) respectively) otherwise the pad and root counter handlers
in the kernel will acknowledge the interrupt before your handler, preventing
you from handling them yourself.

* It is not advisable to handle interrupts using event handlers like in
PSXSDK as it breaks BIOS features that depend on interrupts. Clearing the
VBlank IRQ in a event handler for example prevents the BIOS controller
functions from working as it depends on the VBlank IRQ to determine when to
query controllers. Acknowledge interrupts using a custom handler set by BIOS
function HookEntryInt (B(19h), known as SetCustomExitFromException in nocash
docs).

* When running in high resolution mode you must additionally wait for bit 31
in GPUSTAT (1F801814h) to alternate on every frame (frame 0: wait until 0,
frame 1: wait until 1, frame 2: wait until 0) before waiting for VSync
otherwise the GPU will only draw the first field if you don't have drawing
to displayed area enabled. Performing this check in a low resolution/non
interlaced mode is harmless.

* There's a hardware bug in the GPU FillVRAM command GP0(02h) where if you
set the height to 512 pixels the primitive is processed with a height of 0
as the hardware does not appear to interpret the last bit of the height
value. This is most apparent when putting a DRAWENV with the height of 512
pixels (for PAL for example) and background clearing is enabled, hence why
DRAWENV.isbg is not effective in the official SDK.

* In the official SDK, DMA IRQs appear to be enabled only when a callback
function is set (ie. DrawSyncCallback() enables IRQ for DMA channel 2). DMA
IRQs are only triggered on transfer completion.

Additional notes by spicyjpeg:

* The controller/memory card SPI interface is poorly implemented in most
emulators, making custom controller polling code insanely hard to write and
debug. The only emulator that comes close to real hardware is no$psx, which
seems to correctly implement all features of the SPI port (even those not used
by the BIOS pad driver, such as TX/RX interrupts). DuckStation only emulates
the bare minimum required by the BIOS and Sony libraries, and pcsx-redux has
major bugs that break most custom polling implementations. This pretty much
means TX/RX IRQs and many flags in the JOY_* registers should not be used
unless you are willing to break compatibility with emulators.

* As if communicating with controllers wasn't difficult enough already,
DualShock pads also have a built-in watchdog timer that gets enabled when first
putting them in configuration mode (and is NOT disabled after exiting config
mode). If no polling commands are sent to the controller for about 1 second,
vibration motors are switched off and analog mode is disabled; the same happens
if the analog button is pressed while in analog mode. In order to always keep
the pad in analog mode you must:

 * Poll both controller ports at least once per frame by sending command 42h.
   Polling at a higher rate might be desirable in some cases (such as rhythm
   games) to increase timing accuracy.
 * If a digital pad response (type = 4) is received from a port that hasn't
   previously been flagged as digital-only, attempt to put the pad into config
   mode using command 43h *twice* (as the proper response is delayed).
   * If the pad doesn't recognize the config command, flag it as digital-only
     and treat all further digital pad responses from it as valid.
   * If the pad recognizes the command, it will reply by identifying as an
     analog pad in config mode (type = 15). Send command 44h immediately
     (without sending 42h first, otherwise the pad will exit config mode) to
     enable analog mode and turn on the LED.
   * Pressing the analog button will result in the controller identifying as
     digital even though it is not flagged as such, thus re-triggering the
     configuration process and putting it back into analog mode.
 * All analog pad responses (type = 7) can be treated as valid, as they will
   come from controllers that have already been configured.
 * If no valid response is received, assume no controller is connected and
   reset the port's digital-only flag.

* The SDK provides no support yet for replacing the BIOS exception handler with
a custom one, however it can be done (if you are ok with losing all BIOS
controller, memory card and file functionality). In order not to break anything
your exception handler must do the following:

 * prevent GTE opcodes from being executed twice due to a hardware glitch (the
   nocash docs explain how to do this);
 * define _irq_func_table[12] as an *extern* array of function pointers, call
   the appropriate entry when an IRQ occurs and clear the respective flag in
   register 1F801070h;
 * handle syscalls 01h-02h, i.e. EnterCriticalSection and ExitCriticalSection
   properly. You should also handle syscall FF00h (invalid API usage), as well
   as breaks 1800h and 1C00h (division errors injected by GCC), by locking up
   and maybe showing a BSOD or similar;
 * overwrite the default BIOS API vectors with a passthrough that checks no
   controller- or interrupt-related function is being called. This is necessary
   (although ugly) as libpsn00b often calls such functions internally.

* For some reason mipsel-unknown-elf-nm (symbol map generator) insists on
outputting 64-bit addresses (with the top 32 bits set, e.g. FFFFFFFF80010000)
even when feeding it a regular 32-bit MIPS executable, while the standard x86
nm tool that ships with most GCC packages prints the proper 32-bit address.
Unclear whether this is a bug, intended behavior or the result of some ancient
ELF ABI flag crap. DL_ParseSymbolMap() will ignore the top 32 bits, so this
should only bother you if you're implementing your own symbol map parser.

* I haven't worked on psxspu but, for those willing to write some code, this is
the formula to calculate SPU pitch values for playing musical notes ("^" is the
power operator, not xor):

 frequency = (ref / 32) * (2 ^ ((note - 9) / 12))
 spu_pitch = frequency / 44100 * 4096

 ref  = frequency the sample should be played at to play a middle A (MIDI note 69)
 note = MIDI note number (usually 0-127, 60 is middle C)

* If you are overriding any of the memory allocation functions, DO NOT ENABLE
LINK-TIME OPTIMIZATION. GCC has a long-standing bug with LTO and weak functions
written in assembly, also LTO hasn't been tested at all yet.

Obscure CMake issues and related stuff:

* Toolchain files are loaded "early" according to the CMake docs. What this
means in practice is that a lot of commands, such as find_*(), won't work
properly in a toolchain script as they rely on variables initialized by the
project() command. The poorly documented solution to this is to move such
commands to a separate file and set CMAKE_PROJECT_INCLUDE to point to it, so
project() will execute it immediately after initialization.

* After executing the toolchain file, CMake generates and attempts to build
several dummy projects to test the compiler. Each of these projects re-includes
the toolchain script (which is why you'll see commands executed multiple times)
and uses the same variable values as the main project, however CMake will *NOT*
pass custom variables through by default. If your toolchain script has options
that can be set via custom variables (like PSN00BSDK_TC and PSN00BSDK_PREFIX in
PSn00bSDK), you'll have to set CMAKE_TRY_COMPILE_PLATFORM_VARIABLES to a list
of variable names to be exported to generated dummy projects.

* There is no way to use multiple toolchains (PS1 + host) in a single project,
even if you use add_subdirectory() to execute multiple project files (which,
confusingly, adds their targets to the parent project rather than treating them
as separate projects). Thankfully though CMake provides support for automating
the build process of independent CMake projects via the ExternalProject module.
Which brings me to the next issue...

* If you run CPack on a "superbuild" project (i.e. a project that calls
ExternalProject_Add() to configure, compile and install subprojects at build
time), you'll likely run into a weird issue with CPack bundling folders from
your build directory into DEB and RPM packages. This is caused by the DEB/RPM
generators running "cmake --install" in a chroot/fakeroot to prepare the files
to be packaged, which seems to interfere with absolute paths in the project
cache or something like that (?). The only workaround I know of is to use
CPACK_PRE_BUILD_SCRIPTS to trigger a custom script that deletes anything other
than the actual files to be packaged (see cpack/fakeroot_fix.cmake).

* Project installation might fail on macOS (and possibly some Linux distros) if
CMake attempts to set permissions on system directories such as /usr/local.
This is usually caused by install() commands that copy files to the root
installation directory rather than to a subfolder, like this:

 install(
   DIRECTORY   install_tree/
   DESTINATION .
   USE_SOURCE_PERMISSIONS
 )

If the USE_SOURCE_PERMISSIONS flag is specified CMake will attempt to set
permissions on the DESTINATION folder, which in this case would be the root
prefix (/usr/local by default on macOS), to match the source directory. This
will however fail as macOS restricts top-level system directories from having
their permissions changed. The simplest workaround is to avoid using
"DESTINATION ." and install each subdirectory explicitly instead, like this:

 foreach(
   _dir IN ITEMS
     ${CMAKE_INSTALL_BINDIR}
     ${CMAKE_INSTALL_LIBDIR}
     ${CMAKE_INSTALL_INCLUDEDIR}
     ${CMAKE_INSTALL_DATADIR}
 )
   install(
     DIRECTORY   install_tree/${_dir}/
     DESTINATION ${_dir}
     USE_SOURCE_PERMISSIONS
   )
 endforeach()

* Depending on how you find external dependencies (find_package(), vcpkg,
pkg-config...), CMake may end up outputting an executable that relies on a DLL
installed system-wide. To correctly install the DLL alongside the executable
you have to specify a regex as follows:

 install(
   TARGETS my_executable
   RUNTIME_DEPENDENCIES
   PRE_EXCLUDE_REGEXES ".*"
   PRE_INCLUDE_REGEXES "tinyxml2"
 )

CMake will scan the executable at install time and copy all the required DLLs
that match the second regex. If no regex is specified CMake will also copy OS
DLLs like libc or msvcrt, which usually isn't the desired behavior.

* Using interface targets to set include directories can be finicky. Not only
do you have to use generator expressions to conditionally use different paths
depending on whether the targets are installed, but CMake can get confused on
which options to pass to the compiler. I spent hours trying to get CMake to use
-I rather than -isystem for include directories (for some reason GCC would
ignore -isystem completely). I eventually gave up and just set the include
directories manually for each target, and for some reason CMake actually
started passing -I instead of -isystem to GCC.

* When using CPack with NSIS, all CPACK_NSIS_* variables are passed to NSIS
verbatim, i.e. without the usual slash-to-backslash path conversion that CMake
does on Windows. Most Windows programs accept paths with slashes without issue,
unfortunately the NSIS builder is not one of those. To add insult to injury,
CMake doesn't even escape backslashes by default when quoting strings in the
generated CPack config file! So you have to convert the paths manually *and*
tell CMake to enable escaping by setting CPACK_VERBATIM_VARIABLES, like this:

 set(CPACK_VERBATIM_VARIABLES ON)
 foreach(
   _var IN ITEMS
     CPACK_NSIS_MUI_ICON
     CPACK_NSIS_MUI_UNIICON
     CPACK_NSIS_MUI_HEADERIMAGE
     CPACK_NSIS_MUI_WELCOMEFINISHPAGE_BITMAP
     CPACK_NSIS_MUI_UNWELCOMEFINISHPAGE_BITMAP
 )
   cmake_path(NATIVE_PATH ${_var} ${_var})
 endforeach()

* Not a CMake/CPack bug per se, but NSIS is picky about the banner and header
images shown in generated installers. They must be Windows BMP version 3 files
with no alpha channel, no compression and no metadata. They can either be
24-bit RGB or indexed, though it's common to use indexed colors to save space.