Notes on successful glitch-free operation under Linux: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2020-08-05 *Got it working by disabling interrupts before every v_WaitRecal, enabling them again afterwards. Existing code to avoid Linux "tick" interrupts used to keep system time and scheduler working normally by checking that none of the system timer interrupts will be due before the draw operation is completed (and interrupts are enabled again) *Tested in RealtimePi, with all unused peripherals disabled (in theory everything except the serial terminal). Should work with other peripherals enabled though. *Still a breif but sharp slow-down occouring every so often, reducing ST_GAP_MIN from 3000 to 500 makes no difference to this. *Enabling "RTSCHED" so that process gets maximum realtime priority during v_WaitRecal seems to have fixed the occasional slow-down problem. *Testing in PiCore, still glitchy!? Not sure what's the problem here. Try in Raspberry Pi OS. *Exiting game by pressing buttons 1+2+3+4 crashes Linux because the exit condition is handled within v_WaitRecal so the interrupts never get enabled again. Ctrl-C or exiting within the main game code should be safe, and this problem will be fixed by integrating interrupt avoidance/disabling/enabling in the Vectrex Interface library itself. -This could also cause problems when accessing the Calibration menu. *When this functionality is integrated into the Vectrex Interface library, ST_GAP_MIN should be replaced with a variable that is calculated based on the total drawing time for all lines in the vector buffer, so that no more time is wasted waiting for timer interrupts than is necessary for each individual frame. *Interrupt enable registers seem to have the same value all the time in the RealtimePi test environment, might not need to check these again every v_WaitRecal, though not a big performance penalty so not worth the risk. *Code assumes that Linux will always use timer comparators 0,2, and 3. But that's from observation - only expected it to use 3, so don't know if this behaviour may vary. 2020-08-07 *Tested in Raspberry Pi OS, still glitchy just like with PiCore. Double checked - definately the same version that's working in RealtimePi. Something in the non-realtime patched kernels must be silently preventing the interrupts from being disabled. - Though that forum thread suggests that it worked once. *Tested with setting priority to 99 FIFO in init code rather than just during every v_WaitRecal. This also prevents the intermitent slowdown problem, so the setting before must have been staying on after it was unset or something. Anyway, clearly doing it in the init section is more efficient. *The calibration menu shouldn't cause issues like exiting with 1+2+3+4 because the game code still runs in the background. *Tried enabling all of the hardware features in RealtimePi such as I2C, SPI, Audio, and that temperature monitor thing, that I disabled in config.txt. Also enabled HDMI and Composite video in Linux with tvservice. Zblast still works perfectly. So they're not important (at least while not being used). Still can't see how to get USB working. *Graham says that it's not working for him with a Pi Zero W. Not sure what the problem could be, unless it works after building without -DUART0. 2020-08-08 *Tested latest vectrexInterface library (previously using the one from the "Pitrex10" release), looks very messy. Almost as if there are interrupt-related glitches happening again, but it might just be things getting very messed up within the library. -Possible that it's not using the pipelining, or otherwise not always waiting for v_WaitRecal before drawing? Need to check. *Enabling the EDR in pitrexio-gpio using the "USE_EDR" flag causes Linux to crash. No real need to use it though - not until the PiTrex code uses hardware interrupts itself, in which case it will need to be done in a kernel module in Linux and the rules will have all changed then. *Tried out vHyperoid with the interrupt avoidance code and now it's just as solid as Zblast when running in RealtimePi. 2020-08-11 *The only obvious way that interrupts work differently with the PREEMPT_RT patch is that the "threadirqs" option is enabled by default. This can be enabled in RPi OS too by adding "threadirqs" to cmdline.txt. Not clear from the docs how this would affect things when I'm disabling interrupts anyway, and sure enough it doesn't seem to help at all. 2020-08-12 *Having got a glitch-free display un RealtimePi, now the question is why this does not work in Raspberry Pi OS or PiCore (the latter using a similar kernel, but minimal other processes running by default). *A little earlier I tested the timtest program posted to the Raspberry Pi forums, and in Raspbian it reported no interrupts detected after disabling them in Raspbian. -The forum thread with the timtest program (late 2019 version (with Pi4 support) used, see last page): https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=52393 *While I'd based my interrupt disabling code on timtest, I had forgotten to disable the Fast Interrupt (FIQ). But adding code to do this made no difference. /proc/interrupts suggests that the FIQ was being used for USB, and no USB devices were connected during any tests. *I confirmed that Linux still crashed if the game was exited by pressing 1+2+3+4 (interrupts never re-enabled), so clearly some important interrupts are being disabled in Raspberry Pi OS. *I copied all of timtest.c into svgalib-vectrex.c so that I could use the measureints() function to detect any interrupts after they've been disabled by my code instead of the timtest interrupts() function. The result was zero detected. This surely suggests that my code _is_ disabling all interrupts, yet something else is causing glitches except in RealtimePi. *timtest detects interrupts by checking a hardware timer and checking that it always detects an overflow before the timer counter has incremented any further. If it didn't see the overflow straight away, there must have been an ISR running then, so it counts an interrupt. Besides interrupts, another cause of glitches could be the CPU clock slowing. This would proportionally slow the timer counter frequency, so timtest wouldn't see anything happening, but in the real world (including the Vectrex's world) things would be taking longer to happen. *Linux uses a system called "CPU Performance Scaling", which is controlled in "/sys/devices/system/cpu/cpufreq/". In RPi OS the "scaling policy" defaults to "ondemand". This means that the CPU frequency can be reduced during periods of low CPU load. The policy we want is "performance", which keeps the frequency always at maximum. This can be set with the command: echo -n performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor The current CPU frequency (we want "1000000") can be read with: cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_cur_freq -See: https://www.kernel.org/doc/html/v4.19/admin-guide/pm/cpufreq.html https://wiki.linuxaudio.org/wiki/system_configuration It can be seen that when the CPU is not under high load and scaling_governor is set to ondemand, the Pi0 CPU speed read from scaling_cur_freq is "700000". With the performance policy, it is always "1000000". *In addition to the above setting, "disable_pvt=1" apparantly prevents the Pi from adjusting the refresh rate of RAM every 500ms (measuring RAM temperature), and was added to /boot/config.txt. Also HDMI was disabled with "hdmi_blanking=2". *Glitches were minimised at this stage, but still occasionally happened. These were finally eliminated after executing "sudo tvservice -o" within Raspberry Pi OS, which disables HDMI (should have been already) and Composite video output. Note that it hasn't been determined whether this may cause re-enabling of HDMI or Composite output to be complicated, even after rebooting. *To summarise, with the latest AVOID_TICKS and RTSCHED code in svgalib-vectrex.c (enabled with the corresponding definitions passed to the compiler), the following steps achieve a glitch-free display in current Raspberry Pi OS Buster: 1)Before booting, add the following lines to /boot/config.txt (/boot is the first partition on the SD card): (note that most, maybe all, of these might not be required [EDIT: only "disable_pvt=1" is required]) disable_pvt=1 dtparam=audio=off dtparam=i2c_arm=off dtparam=i2c=off,spi=off,i2s=off hdmi_blanking=2 2)After booted and logged into terminal, execute this command to set CPU policy to "performance": echo -n performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor 3)Execute this command to disable all video output: sudo tvservice -o 4)Run the zblast binary as root: sudo ./zblast -Step 2 and 3 may be made permanent by adding these lines to /etc/rc.local (untested): echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor tvservice -o [EDIT: Bloody Systemd has broken /etc/rc.local] -Use Cron instead (silly way to do it though, grumble grumble...) $ sudo cron -e Add these lines: @reboot echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor @reboot sudo tvservice -o -Nope that's not working either @##$@%@! OK, enable /etc/rc.local with damn Systemd following this tutorial: https://www.linuxbabe.com/linux-server/how-to-enable-etcrc-local-with-systemd except instead of line: ExecStart=/etc/rc.local start in /etc/systemd/system/rc-local.service use: ExecStart=/etc/rc.local Nope, that didn't work either. I give up, give me a Systemd-free distro any day... *I guess there's some chance that the above settings may cause the Pi0 to overheat, because they turn off two apparant methods to reducing operating temperature (reducing CPU frequency, and maybe whatever disable_pvt=1 stops). Heatsink required? *Tested some more - The only line that actually needs to be added to the standard RPi OS config.txt is: disable_pvt=1 2020-08-13 *"disable_pvt=1" actually isn't required, so no changes to config.txt are needed. *Can't get Cron to execute the commands on boot whatever I do (tried with "sudo crontab -e", editing "/etc/crontab", and "crontab -e". Strange. *Vaguely based on this guide: https://www.redhat.com/sysadmin/replacing-rclocal-systemd I was able to strip out apparantly obsolete commands from the Systemd service file that I used from the earlier guide, and that seems to work. *So the final steps to enable a glitch-free environment in RPi OS: 1)Add these lines to /etc/rc.local: ---NOPE, that doesn't survive a reboot after all... *OK, the problem is that systemd doesn't run /etc/rc.local (or, presumably, start Cron) last in the boot process, like sysV init used to. So I change the settings one way, and they get changed back again later in the boot. Furthermore as systemd likes every service to run relative to every other service, and it's not clear which services change back those configuration settings, it's even more complicated to restore this old behaviour. So the process for initial configuration is now: 1)Become root ("sudo su") 2)Edit file "/etc/rc.local", adding the following lines before "exit 0" (all that was needed before Systemd): #Keep CPU frequency at maximum: echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor #Disable HDMI/Composite video output: tvservice -o 2)Create file "/etc/systemd/system/rc-local.service" containing: [Unit] Description=Runs /etc/rc.local After=multi-user.target [Service] Type=simple ExecStart=/etc/rc.local [Install] WantedBy=rc-local.target 3)Create file "/etc/systemd/system/rc-local.target" containing: [Unit] Description=Run service that runs /etc/rc.local Requires=multi-user.target After=multi-user.target AllowIsolate=yes 4)Run mkdir /etc/systemd/system/rc-local.target.wants 5)Run ln -s /etc/systemd/system/rc-local.service /etc/systemd/system/rc-local.target.wants/rc-local.service 6)Run systemctl daemon-reload 7)Run systemctl set-default rc-local.target 8)Optionally run this to test before reboot systemctl isolate rc-local.target -Confirm that it has worked by running (don't need to be root): cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor Should return: performance Not: ondemand -If not working, errors might be reported with the commands: systemctl status rc-local | less journalctl -u rc-local | less -The " | less" bit is needed when connected via serial UART otherwise long lines are truncated (which again is stupid). -The above process is largely based on this, as well as the previously mentioned tutorials: https://superuser.com/a/1543365 2020-08-14 *Monitor Pi0 temperature with this command (divide result by 1000 for degC): sudo cat /sys/class/thermal/thermal_zone0/temp -Seems alright running at max. freq. *Graham reports that the deglitched Zblast binaries aren't working for him, causing a system crash. Notes that he's using USB. *Tried in RPi OS from "2020-05-27-raspios-buster-lite-armhf.zip" with USB. Also crashes instantly. Crashed with a USB wireless keyboard/mouse adapter connected, and also with a USB memory stick connected (but not mounted). *A USB handler is associated with FIQ in /proc/interrupts. Probably related to this: https://github.com/raspberrypi/linux/commit/1f45d3d733d51369b3e21b3152839e24a840ec59 *Tried with these options in cmdline.txt to disable FIQ USB support, but doesn't work and also /proc/interrupts is still the same: dwc_otg.fiq_fix_enable=0 dwc_otg.lpm_enable=0 dwc_otg.fiq_fsm_enable=0 dwc_otg.nak_holdoff=0 *Tried in PiCore 9, /proc/interrupts still shows USB FIQ, but deglitched ZBlast runs fine with either USB keyboard or USB memory stick connected (memory stick mounted too). *Note that PiCore 9 does not need "tvservice -o" to eliminate glitches, just "echo -n performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor". *Try dtoverlay=dwc2 in config.txt to avoid loading alternative "dwc_otg" driver that apparantly uses FIQ and currently shows elsewhere in /proc/interrupts? https://raspberrypi.stackexchange.com/questions/77059/what-does-dtoverlay-dwc2-really-do *Well in Raspberry Pi OS, dtoverlay=dwc2 prevents any FIQ association being shown in /proc/interrupts, and deglitched Zblast runs with USB device connected. BUT, USB doesn't work. No devices are even detected in the output of lsusb besides the standard "Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub". -It seems that the dwc2 USB driver is broken, or needs more configuration, with the Pi0. *Back with the dwc_otg driver, timtest causes a crash when used with USB connected too. So it may not be a fixable problem from the side of the interrupt disabling code. *TODO: -Work out how PiCore 9's USB config. differs from RPi OS. -Try PiCore 11 - uses the same kernel (4.19) as current Raspberry Pi OS. -Look into ways to make the dwc2 driver work. -I guess dig into the reported 45000 lines of dwc_otg USB driver code to find any other cmdline.txt options that might affect it. They don't seem to be documented anywhere. *Tried compiling without the FIQ disabling code (why didn't I try this earlier?!), that works but now there are some very occasional glitches showing. Compile with "-DDISABLE_FIQ" to disable FIQ and allow glitch-free display without USB, or in PiCore 9 with USB. Without "-DDISABLE_FIQ" for very slightly glitchy display in RPi OS with a USB device working via the current dwc_otg driver. 2020-08-25 *Glitches visible without "-DDISABLE_FIQ" in PiCore 11. So problem must be with newer kernel or associated "dwc_otg" USB driver. -PiCore 11 / RPi OS Buster use kernel v. 4.19 -RealtimePi Buseter uses v. 4.19 (no glitches, but also no USB for me) -PiCore 9 uses v. 4.9 (USB still works when compiled with "-DDISABLE_FIQ" for zero glitches) *There are a lot more glitches when a USB device is plugged in. *As with RPi OS Buster, adding dtoverlay=dwc2 to config.txt prevents USB from working at all. -Also tried "dtoverlay=dwc2,dr_mode=host", with the same result. -the overlays readme file says that the dwc2 driver is default for the Pi Zero, but that clearly isn't the case. -I'm increasingly thinking that the solution here is to file a bug report for the dwc2 driver, because it would be the best solution if only it worked. -Should try on the Pi0W for the same of completeness. 2020-08-26 *"dwc_fiq_enable=N" in config.txt does nothing. I'm of the opinion that the function for not using FIQ was removed from the dwc_otg driver recently, so all of these options just do nothing now. I haven't checked the source code though. *"dtoverlay=dwc2" doesn't seem to load any USB driver in piCore 11, I don't think the dwc2 driver is included by default (might be in an extension, but I don't know which - ask on the forums?) *Before I used "dtoverlay=dwc2" in config.txt for RPi OS Buster and it didn't work, but I only tried "dtoverlay=dwc2,dr_mode=host" in PiCore 11. Trying "dtoverlay=dwc2,dr_mode=host" with RPi OS results in working USB and no FIQ interrupt (glitch-free display and no crashes with -DDISABLE_FIQ). Woo hoo! *So working configurations (working USB and no glitches) tested are: RPi OS 10.4 (Buster): ^^^^^^^^^^^^^^^^^^^^^ -"dtoverlay=dwc2,dr_mode=host" in config.txt. Presumably OTG/Gadget mode not working, but detects connected devices. - -DDISABLE_FIQ not required (no FIQs anyway, as shown with "cat /proc/interrupts"). -EDIT: Note that "echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor" and "tvservice -o" are still required prior to running. -"dtoverlay=dwc2,dr_mode=otg" in config.txt. Doesn't detect connected devices, but OTG/Gadget mode might work (and/or with "dtoverlay=dwc2,dr_mode=peripheral"?). - -DDISABLE_FIQ not required (no FIQs anyway, as shown with "cat /proc/interrupts"). -EDIT: Note that "echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor" and "tvservice -o" are still required prior to running. PiCore 11: ^^^^^^^^^^ -NONE: Have to choose between glitches and USB unless dwc2 driver can be loaded (or a dwc_otg that doesn't mind FIQ being disabled). PiCore 9: ^^^^^^^^^ -The dwc_otg driver for this kernel (4.9) doesn't crash the system when FIQ is disabled, so don't need to specify dwc2 in config.txt (and it probably wouldn't load even if you did). - -DDISABLE_FIQ is required. -EDIT: note that "echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor" is still required prior to running. 2020-08-29 * I've now integrated the interrupt disabling code into the current Vectrex Interface library, so "V. 0.2_avoid_ticks" of svgalib-vectrex is no longer required, and non-SVGAlib games should be able to display in Linux without glitches as well. * With the interrupt disabling in the Vectrex Interface library now, exiting with 1+2+3+4 doesn't cause a crash. A total drawing time per frame is now calculated and used to determine the minimum "gap" to wait for before diabling the Linux interrupts, this should be more efficient than the fixed-value minimum gap used in the svgalib-vectrex version (though actual performance hasn't yet been confirmed). * Note that interrupts are only disabled while the vector pipeline is being drawn. So if the pipeline feature is disabled, or some lines are drawn without it, they will be without protection from glitches. * The problem reported earlier with the display looking a mess when using the latest Vectrex Interface lib. was just because I'd left the old settings file there and it wasn't compatible with the later library. With that deleted, and settings readjusted from scratch, the display is actually better than ever. 2020-08-31 *Test: gap duration calculation, with Git repo code, linmenu, Hyperoid. 2020-09-01 *In the PiTrex code from the Git repo, using -DRTSCHED causes pause in execution and visible screen blanking to occour at regular intervals, but this doesn't happen in Zblast for some reason. *Seems to work alright without -DRTSCHED, but only if -DDISABLE_FIQ is used along with -DAVOID_TICKS, otherwise there are still occasional glitches. *If -DRTSCHED is needed later for some reason, there may be some scope for adjusting the scheduler settings in v_init to avoid the problem (whatever it is exactly). 2020-09-02 *The random thrust problem in Hyperoid is due to random pulses of "Down" from the joystick. This is coming from the Vectrex Interface lib. Not sure why. -Messing with associated delay values doesn't help. Well it possibly reduces the frequency significantly. *This isn't likely due to interrupts because it was happening before with the deglitched svgalib-vectrex, where interrupts were disable during controller reading as well.