From 31fba94b8e8c15c1d127925f01a38efc68933fd1 Mon Sep 17 00:00:00 2001 From: spicyjpeg <88942473+spicyjpeg@users.noreply.github.com> Date: Mon, 29 Nov 2021 00:17:28 +0100 Subject: Add io/system573 example, rewrite dev notes, fix CI --- doc/dev_notes.md | 280 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 280 insertions(+) create mode 100644 doc/dev_notes.md (limited to 'doc/dev_notes.md') diff --git a/doc/dev_notes.md b/doc/dev_notes.md new file mode 100644 index 0000000..3aa2304 --- /dev/null +++ b/doc/dev_notes.md @@ -0,0 +1,280 @@ + +# Development notes + +These are some development notes I've put together that would be of great aid +to those willing to contribute to the PSn00bSDK project. Many of these came +from my own experience dealing with the PS1 at low-level when I ran into some +unexplained quirks while some are from disassembly observations and +clarification of existing documents. More entries will be added when I run into +more previously undocumented quirks in the future. _- Lameguy64_ + +Porting PSn00bSDK to CMake also uncovered a lot of bugs and undocumented +behavior in CMake itself. I documented [below](#cmake) all issues I ran into. +_- spicyjpeg_ + +## MIPS ABI / compiler + +- When calling C functions (ie. BIOS functions) from assembly code you'll need + to allocate N words on the stack first when calling a function that has N + arguments (`addiu $sp, -(4*N)` where N = number of arguments of the function + being called) even if the arguments are on registers `a0` to `a3` and the C + functions don't always use the space allotted in stack. When calling a + function with a variable number of arguments (`printf`) always allocate 16 + bytes of stack. + +- For some reason `mipsel-unknown-elf-nm` and `mipsel-none-elf-nm` (symbol map + generators) insist on outputting 64-bit addresses (with the top 32 bits set, + e.g. `FFFFFFFF80010000`) even when feeding it a regular 32-bit MIPS + executable, while the standard x86 nm tool that ships with most GCC packages + prints the proper 32-bit address. Unclear whether this is a bug, intended + behavior or the result of some ancient ELF ABI flag crap. + `DL_ParseSymbolMap()` will ignore the top 32 bits, so this should only bother + you if you're implementing your own symbol map parser. + +- If you are overriding any of the memory allocation functions, + **DO NOT ENABLE LINK-TIME OPTIMIZATION**. GCC has a long-standing bug with + LTO and weak functions written in assembly, also LTO hasn't been tested at + all yet. + +## BIOS and interrupts + +- Hooking a custom handler using BIOS function `HookEntryInt` (`B(19h)`, known + as `SetCustomExitFromException` in nocash docs) is only triggered when + there's an IRQ that is not yet acknowledged by previous IRQ handlers built + into the kernel. This is also the best point to acknowledge any IRQs without + breaking compatibility with built-in BIOS IRQ handlers and is what the + official SDK uses to handle IRQs. To make sure this handler is triggered on + every interrupt you must call `ChangeClearPad(0)` and `ChangeClearRCnt(3, 0)` + (which are functions `B(5Bh)` and `C(0Ah)` respectively) otherwise the pad + and root counter handlers in the kernel will acknowledge the interrupt before + your handler, preventing you from handling them yourself. + +- It is not advisable to handle interrupts using event handlers like in PSXSDK + as it breaks BIOS features that depend on interrupts. Clearing the VBlank IRQ + in a event handler for example prevents the BIOS controller functions from + working as it depends on the VBlank IRQ to determine when to query + controllers. Acknowledge interrupts using a custom handler set by BIOS + function `HookEntryInt` (`B(19h)`, known as `SetCustomExitFromException` in + nocash docs). + +- In the official SDK, DMA IRQs appear to be enabled only when a callback + function is set (ie. `DrawSyncCallback()` enables IRQ for DMA channel 2). DMA + IRQs are only triggered on transfer completion. + +- PSn00bSDK provides no support yet for replacing the BIOS exception handler + with a custom one, however it can be done (if you are ok with losing all BIOS + controller, memory card and file functionality). In order not to break + anything your exception handler must do the following: + + - prevent GTE opcodes from being executed twice due to a hardware glitch (the + nocash docs explain how to do this); + - define `_irq_func_table[12]` as an *extern* array of function pointers, + call the appropriate entry when an IRQ occurs and clear the respective flag + in register `1F801070h`; + - handle syscalls `01h`-`02h`, i.e. `EnterCriticalSection` and + `ExitCriticalSection`, properly. You should also handle syscall `FF00h` + (invalid API usage), as well as breaks `1800h` and `1C00h` (division errors + injected by GCC), by locking up and maybe showing a BSOD or similar; + - overwrite the default BIOS API vectors with a passthrough that checks no + controller- or interrupt-related function is being called. This is necessary + (although ugly) as `libpsn00b` often calls such functions internally. + +## Hardware + +- When running in high resolution mode you must additionally wait for bit 31 in + `GPUSTAT` (`1F801814h`) to alternate on every frame (frame 0: wait until 0, + frame 1: wait until 1, frame 2: wait until 0) before waiting for VSync + otherwise the GPU will only draw the first field if you don't have drawing to + displayed area enabled. Performing this check in a low resolution/non + interlaced mode is harmless. + +- There's a hardware bug in the GPU `FillVRAM` command `GP0(02h)` where if you + set the height to 512 pixels the primitive is processed with a height of 0 as + the hardware does not appear to interpret the last bit of the height value. + This is most apparent when putting a DRAWENV with the height of 512 pixels + (for PAL for example) and background clearing is enabled, hence why + `DRAWENV.isbg` is not effective in the official SDK. + +- The controller/memory card SPI interface is poorly implemented in most + emulators, making custom controller polling code insanely hard to write and + debug. The only emulator that comes close to real hardware is no$psx, which + seems to correctly implement all features of the SPI port (even those not + used by the BIOS pad driver, such as TX/RX interrupts). DuckStation only + emulates the bare minimum required by the BIOS and Sony libraries, and + pcsx-redux has major bugs that break most custom polling implementations. + This pretty much means TX/RX IRQs and many flags in the `JOY_*` registers + should **not** be used unless you are willing to break compatibility with + emulators. + +- As if communicating with controllers wasn't difficult enough already, + DualShock pads also have a built-in watchdog timer that gets enabled when + first putting them in configuration mode (and is **not** disabled after + exiting config mode). If no polling commands are sent to the controller for + about 1 second, vibration motors are switched off and analog mode is + disabled; the same happens if the analog button is pressed while in analog + mode. In order to always keep the pad in analog mode you must: + + 1. Poll both controller ports at least once per frame by sending command + `42h`. Polling at a higher rate might be desirable in some cases (such as + rhythm games) to increase timing accuracy. + 2. If a digital pad response (type = 4) is received from a port that hasn't + previously been flagged as digital-only, attempt to put the pad into + config mode using command `43h` *twice* (as the proper response is + delayed). + - If the pad doesn't recognize the config command, flag it as digital-only + and treat all further digital pad responses from it as valid. + - If the pad recognizes the command, it will reply by identifying as an + analog pad in config mode (type = 15). Send command `44h` immediately + (without sending `42h` first, otherwise the pad will exit config mode) + to enable analog mode and turn on the LED. + - Pressing the analog button will result in the controller identifying as + digital even though it is not flagged as such, thus re-triggering the + configuration process and putting it back into analog mode. + 3. All analog pad responses (type = 7) can be treated as valid, as they will + come from controllers that have already been configured. + 4. If no valid response is received, assume no controller is connected and + reset the port's digital-only flag. + +- I haven't worked on `psxspu` but, for those willing to write some code, this + is the formula to calculate SPU pitch values for playing musical notes (`^` + is the power operator, not xor): + + ``` + frequency = (ref / 32) * (2 ^ ((note - 9) / 12)) + spu_pitch = frequency / 44100 * 4096 + + ref = frequency the sample should be played at to play a middle A (MIDI note 69) + note = MIDI note number (usually 0-127, 60 is middle C) + ``` + +## CMake + +- Toolchain files are loaded "early" according to the CMake docs. What this + means in practice is that a lot of commands, such as `find_*()`, won't work + properly in a toolchain script as they rely on variables initialized by the + `project()` command. The poorly documented solution to this is to move such + commands to a separate file and set `CMAKE_PROJECT_INCLUDE` to point to it, + so `project()` will execute it immediately after initialization. + +- After executing the toolchain file, CMake generates and attempts to build + several dummy projects to test the compiler. Each of these projects + re-includes the toolchain script (which is why you'll see commands executed + multiple times) and uses the same variable values as the main project, + however CMake will *not* pass custom variables through by default. If your + toolchain script has options that can be set via custom variables (like + `PSN00BSDK_TC` and `PSN00BSDK_PREFIX` in PSn00bSDK), you'll have to set + `CMAKE_TRY_COMPILE_PLATFORM_VARIABLES` to a list of variable names to be + exported to generated dummy projects. + +- There is no way to use multiple toolchains (PS1 + host) in a single project, + even if you use `add_subdirectory()` to execute multiple project files + (which, confusingly, adds their targets to the parent project rather than + treating them as separate projects). Thankfully though CMake provides support + for automating the build process of independent CMake projects via the + `ExternalProject` module. Which brings me to the next issue... + +- If you run CPack on a "superbuild" project (i.e. a project that calls + `ExternalProject_Add()` to configure, compile and install subprojects at + build time), you'll likely run into a weird issue with CPack bundling folders + from your build directory into DEB and RPM packages. This is caused by the + DEB/RPM generators running `cmake --install` in a chroot/fakeroot to prepare + the files to be packaged, which seems to interfere with absolute paths in the + project cache or something like that (?). The only workaround I know of is to + use `CPACK_PRE_BUILD_SCRIPTS` to trigger a custom script that deletes + anything other than the actual files to be packaged (see + `cpack/fakeroot_fix.cmake`). + +- Project installation might fail on macOS (and possibly some Linux distros) if + CMake attempts to set permissions on system directories such as `/usr/local`. + This is usually caused by `install()` commands that copy files to the root + installation directory rather than to a subfolder, like this: + + ```cmake + install( + DIRECTORY install_tree/ + DESTINATION . + USE_SOURCE_PERMISSIONS + ) + ``` + + If the `USE_SOURCE_PERMISSIONS` flag is specified CMake will attempt to set + permissions on the `DESTINATION` folder, which in this case would be the root + prefix (`/usr/local` by default on macOS), to match the source directory. + This will however fail as macOS restricts top-level system directories from + having their permissions changed. The simplest workaround is to avoid using + `DESTINATION .` and install each subdirectory explicitly instead, like this: + + ```cmake + foreach( + _dir IN ITEMS + ${CMAKE_INSTALL_BINDIR} + ${CMAKE_INSTALL_LIBDIR} + ${CMAKE_INSTALL_INCLUDEDIR} + ${CMAKE_INSTALL_DATADIR} + ) + install( + DIRECTORY install_tree/${_dir}/ + DESTINATION ${_dir} + USE_SOURCE_PERMISSIONS + ) + endforeach() + ``` + +- Depending on how you find external dependencies (`find_package()`, vcpkg, + pkg-config...), CMake may end up outputting an executable that relies on a + DLL installed system-wide. To correctly install the DLL alongside the + executable you have to specify a regex as follows: + + ```cmake + install( + TARGETS my_executable + RUNTIME_DEPENDENCIES + PRE_EXCLUDE_REGEXES ".*" + PRE_INCLUDE_REGEXES "tinyxml2" + ) + ``` + + CMake will scan the executable at install time and copy all the required DLLs + that match the second regex. If no regex is specified CMake will also copy OS + DLLs like `libc` or `msvcrt`, which usually isn't the desired behavior. + +- Using interface targets to set include directories can be finicky. Not only + do you have to use generator expressions to conditionally use different paths + depending on whether the targets are installed, but CMake can get confused on + which options to pass to the compiler. I spent hours trying to get CMake to + use `-I` rather than `-isystem` for include directories (for some reason GCC + would ignore `-isystem` completely). I eventually gave up and just set the + include directories manually for each target, and for some reason CMake + actually started passing `-I` instead of `-isystem` to GCC. + +- When using CPack with NSIS, all `CPACK_NSIS_*` variables are passed to NSIS + verbatim, i.e. without the usual slash-to-backslash path conversion that + CMake does on Windows. Most Windows programs accept paths with slashes + without issue, unfortunately the NSIS builder is not one of those. To add + insult to injury, CMake doesn't even escape backslashes by default when + quoting strings in the generated CPack config file! So you have to convert + the paths manually *and* tell CMake to enable escaping by setting + `CPACK_VERBATIM_VARIABLES`, like this: + + ```cmake + set(CPACK_VERBATIM_VARIABLES ON) + foreach( + _var IN ITEMS + CPACK_NSIS_MUI_ICON + CPACK_NSIS_MUI_UNIICON + CPACK_NSIS_MUI_HEADERIMAGE + CPACK_NSIS_MUI_WELCOMEFINISHPAGE_BITMAP + CPACK_NSIS_MUI_UNWELCOMEFINISHPAGE_BITMAP + ) + cmake_path(NATIVE_PATH ${_var} ${_var}) + endforeach() + ``` + +- Not a CMake/CPack bug per se, but NSIS is picky about the banner and header + images shown in generated installers. They must be Windows BMP version 3 + files with no alpha channel, no compression and no metadata. They can either + be 24-bit RGB or indexed, though it's common to use indexed colors to save + space. + +----------------------------------------- +_Last updated on 2021-11-28 by spicyjpeg_ -- cgit v1.2.3