[Jan 24 2024, I asked Alastair about the internal operation of the serial port. I had broken
file transfer recieves, using the XFER 8080 program. About 256 bytes within the small file was missing. Turned
out later, my NOvasaur RTS signal did not get to the PC. Meanwhile here's Alastair's notes. - Herb]

The Novasaur RS232 port is a DTE like a PC COM port. A male-to-female cable would be used to connect to a DCE device like a modem. If it’s another computer (DTE) then you need a female-to-female cable that crosses over the TX/RX and RTS/CTS, or a null modem. It sounds like you have this set up correctly.

There are a few layers to the onion on the software side. The first is the hardware abstraction layer where the serial is sampled at the end of each video line. The transition from the stop to start bit (high to low) is tracked to recover the serial clock edge and handle framing. This can accommodate up to a 2% error in the serial rate, so anything between 9400 to 9800 baud should work. The actual Novasaur serial rate is 9593 baud.

The firmware maintains a 256-byte buffer for transmit and receive data. Each buffer has a read and write index. Data received over serial goes in the receive buffer at the RXWRIDX and data is sent from the transmit buffer at the TXRDIDX. The firmware reads from the receive buffers at RXRDIDX and writes to the transmit buffer at TXWRIDX. The firmware checks the “distance” between the read and write index and will deassert the RTS line when it gets to 128 bytes. The RTS is then reasserted once the buffer is empty. The LED is connected to the RTS line, so you will see this flash on and off when receiving data as the buffer fills and empties. The Novasaur typically can’t quite keep up with the 9600 baud serial so the buffer will fill and empty under normal operation. The RTS/CTS flow control is critical to the serial operation and takes care of keeping things in sync. Additional flow control since it would be redundant so I haven’t implemented things like XON/XOFF.

There isn’t a way for the 8080 to get access to these buffers though. The serial is accessed via port 8, where the IN 8 instruction will return the byte at RXRDIDX and increment it until it reaches the RXWRIDX (buffer empty) at which point it returns a “null” (ASCII zero). The OUT 8 instruction will write the accumulator to the transmit buffer until it’s full at which point the accumulator is set to zero to indicate the buffer is full. This works great for a console handing text, but this mechanism is not used for binary file transfers used by XFER.

There’s a pair of extended instructions that copy the serial buffers to a partial record. The serial buffers are limited to 128 bytes so they don’t exceed a single record but typically contain less data or nothing at all. The partial records are read/written by CP/M using the PUNCH and READER commands. These functions do not return the records though, they only read/write a single byte. The records are reassembled in a buffer and the PUNCH and READER access this. There’s a bit more control over the data here when the PUNCH command can specify when the transmit the data. The READER command sets the zero flag to indicate if there’s no serial data available, since all byte values of the data are valid.

The serial transfer commands are always handling binary data so you should see no difference between text and binary files. I’m not sure why it skips bytes, but it might be something to do with the RTS/CTS. I had some issues with the Prolific RS232-USB cable I had that sounds similar. I can try using XFER with this and see if I miss bytes. The issue I’m thinking of is when the buffer fills and it deasserts the RTS. The other end should stop sending data right away, but if not then the subsequent bytes might get missed. - Thanks, Alastair


Jan 24, Alastair:

I’ve been taking a closer look at the low-level (hardware abstraction layer) serial code. Basically when it hits the high-water mark of 128 bytes it will deassert the RTS. The other end can still transmit bytes but should really stop to give the Novasaur time to catch up. If the transmission doesn’t stop then the buffer will eventually fill with 256 bytes and will wrap around. This means you will start to lose data after 256 bytes and resulting file will be smaller.

The Novasaur can’t quite keep up with 9600 baud so it needs the hardware flow control to “pump the breaks”. It’s a fairly low-level thing, so it wouldn’t be a setting in TeraTerm. The Windows Device Manager will show the COM port or driver that you are using. Hopefully there’s a setting there to enable hardware flow control. 

Another update! TeraTerm does work. I hadn’t enabled the flow control [previously]. It’s under “Setup”, “Serial Port…”, “Flow control”. I set it to “RTS/CTS” and everything is working correctly now using my FTDI USB cable. The settings in the Window Device Manager for the [FTDI] COM port don’t make any difference. - Alastair

On 1/26/2024 1:56 PM, Alastair Hewitt wrote:

On the [microcode] buffers. These are separate 256 buffers for both TX and RX. There are in fact four buffers:
$KSBUFF        0xF8
$KCBUFF        0xF9
$RXBUFF        0xFA
$TXBUFF        0xFB

The first one is for the keyboard scan codes (KSBUFF). These are the raw scan code bytes that are sent from the keyboard and treated in a similar way to the serial RX data. When a byte is added to this butter it is also decoded to ASCII and put in the keyboard decode buffer (KCBUFF). I use a state machine to track if the scan code is a make or break code and if things like the shift key is being held down. I was originally going to this in 8080 code but I was able to efficiently add it to the firmware. The RXBUFF and TXBUFF are the two buffers for serial data. All 256 bytes are used but the flow control kicks in at 128 bytes and provides a 128 byte overhead to prevent overruns.