GCC for PC-9801 / PC-9821 and graphics hardware information... plus a game launcher

[megatron-uk]

Does anyone know if there is a source for GCC (native or cross-compiler) targetting the PC-9801/9821?

I assume that a normal copy of DJGPP would probably run, but guess that crt0 and some other stuff could be different.

Also I'm interesting in programming for the graphics hardware in C (just like my efforts for the X68000), but there's hardly anything written down about the 256 colour PEGC hardware on the AMate and similar machines.

[megatron-uk]

Well I managed to find a few things today in relation to GCC/DJGPP for PC-98.

The first is a patch against DJGPP v2.03 from around 2000:

https://www.mfp.gr.jp/users/takas/prog/djgpp.html

... and the second were two more recent projects that are working to get PC-98 support back in DJGPP:

https://github.com/PC-98/djlsr
https://github.com/lpproj/djgpp-cvs.nec/commit/ff819a64c2265ea45675489b18d4cecd91b28e4b

I can't say that the DJGPP patch is entirely successful; it's a two step patch - one for the source tree/headers, to change some PC-98 specific function calls (which works, I managed to patch a clean DJGPP v2.03 source tree), and a binary diff against libc.a to merge in those functions. The latter wouldn't work for me (firstly, it's a weird 'ldf' patch, which I had to find a MS-DOS tool to use, secondly I just couldn't get it to apply the patch to libc.a - all attempts ended up with the same md5sum as the original).

So... I'm in the process of using Dosbox to run the version of GCC include with DJGPP (2.95.2 is the latest that will build v2.03) to buil DJGPP from that modified source tree.

In theory, all of the GCC userland tools should work cleanly on both PC and PC-98, as long as we have a suitable DPMI server on the PC-98 (there's a modified version of the DJGPP 'go32' for that), so you can run the compiler and tools from Dosbox on whatever platform you want and link with the modified libc.a and run the linked executable (via go32) on a real (or emulated) PC-98.

We'll see how far that actually works...

[megatron-uk]

I think we have a result!

Test code compiled under Dosbox:



.... and the same exe copied over to NP2Kai:



The same code was:

Code Select Expand

#include <stdio.h>
#include <libc/pc9800.h>

int main(){
int m;
m = __crt0_mtype;
if(ISPCAT(m)){
printf("Hello, PC world\n");
}
if(ISPC98(m)){
printf("Hello, PC98 world\n");
}
return 0;
}

So this demonstrates that the patched DJGPP libc is working correctly, including those additional PC-98 system calls that were added.

Full write-up (and pre-built dev environment) on my wiki: http://www.target-earth.net/wiki/doku.php?id=blog:pc98_devtools

[megatron-uk]

Okay, some more news.

- gcc.exe and friends run on the actual PC-98 hardware (more specifically, my PC-9821An).
- DPMI.exe from the (PC-98) MS-DOS 5.00 floppies runs on the hardware (as expected, you need a VCPI provider like HIMEM+EMM386, or VEMM486)
- go32-v2.exe which I built from the patched DGJPP 2.03 source code works

I also found some bits and pieces with VRAM access, which I'll be documenting shortly (quick version: you need to enable the 15-16MB memory hole in the PC-98 BIOS to access the PEGC VRAM linear framebuffer).

However, the bad new. I cannot get make.exe to work on the actual hardware. No matter what I do it doesn't find the makefile and refuses to run any targets.

GCC and friends run, once you've got your DPMI service active:



But make will simply not find/run the makefile. No matter what I name it to (MAKEFILE, MAKEFILE.TXT, MAKEFILE.SH, etc). Whether you use -f or --file=, Nothing works:



The same binaries run absolutely fine within Dosbox. I haven't tried them yet in NP21Kai.

I will say that my PC-9821An is a bit of an odd duck. I'm using the EXIDE486 IDE patch to support an 80GB IDE drive, FORMATX to create huge partitions, and the PC-98 Win98SE emergency floppy to layout a giant FAT-32 filesystem. It also refuses to load EMM386 and running MEM locks up the machine. Aside from those oddities it plays games and music fine though, so I can't really see why any of that would be related.

[megatron-uk]

So make is perfectly happy running inside NP2Kai:



So it's got to be something to do with the FAT32 filesystem on my real system.

I'll spin up a new filesystem on a CF card or similar and see if that has the same issues. If it doesn't, and make is happy on a FAT16 filesystem, I'll redo my hardisk to have a 2GB FAT16 drive for OS and devtools, with a big fat games/media partition for the rest.

[megatron-uk]

So the behaviour is no different when running from a FAT16 filesystem ("no rule to make target"). I wonder what on earth could be causing the behaviour on a real machine compared to NP21Kai?

[megatron-uk]

It appears as if there is something broken in the version of make distributed alongside GCC for DJGPP 2.03. I was reluctant to try later versions of GCC and other toolchain utilities, as you never know if you're going to break library or binary compatability.

Fortunately, make.exe is distributed without any external dependencies that I can see, and having tried make 4.3, have found that it resolves the "no rule to make target" error that I was finding on real (FAT16 and FAT32) hardware:



As a result, I can now compile modern-ish C code using the PC-98 version of DJGPP libc.a from source on Dosbox, NP21Kai and now on real hardware.

My wiki is also now updated with a new development environment archive, for use on emulated or real platforms.

[kamiboy]

Nice work. Are you perhaps planning on porting your X68k game launcher To the PC98?

That would be awesome.

[megatron-uk]

That's the long term plan!

[megatron-uk]

Some interesting behaviour I've found in the various DPMI servers for the PC-98. I need to use one to map the framebuffer on the PEGC hardware into a space that a DOS programme can access.

Fortunately there are several dpmi servers available, but, they respond differently to the request to map the physical address of the framebuffer to a logical one:

Epson DOS 5.00 (dpmi32.exe), a DPMI 1.0 server, 64752 bytes.

MS-DOS 5.00 (dpmi.exe), a DPMI 1.0 server, 405566 bytes.

MS-DOS 6.22 (DPMI.exe), a DPMI 1.04 server, 63792 bytes.

All bar the DOS 6.22 server refuse to map the framebuffer region via the __dpmi_physical_address_mapping() call. I need to try booting a DOS 5 or DOS 7 system and use the DOS 6.22 DPMI server to check that it works on a different DOS version, I can't see why it wouldn't.

[megatron-uk]

Oh, and having to deal with DPMI, segments, linear and physical address spaces... What an awful thing. The Sharp (well, Motorola 68000) is beautiful by comparison, and we'll, I'm a Unix programmer really, so DOS was always a bit alien...

[megatron-uk]

I'm starting to get a bit of headache trying out the sample code I'm finding out on the net for accessing VRAM directly.

There are a whole host of registers and IO ports to toggle with in order to set up the correct access and display modes, but I think I've got them sorted, but then setting values in VRAM doesn't work.

So I found a pre-made example for accessing VRAM at: http://darudarudan.syuriken.jp/kai/pc9821.htm#e256L

It comes with source for Watcom C, a compiled exe and a stub of the same code for GCC, but it's incomplete.

The biggest issue is that I cannot replicate things between NP21Kai and my real machine.

The example is supposed to set palette entries and copy buffers to VRAM directly, resulting in a lovely, cycling, plasma-esque display, as shown on my 9821An:



You run the same exe on NP21Kai and it just doesn't do anything. It just sits there, no errors, no colours, nothing.

That's with the same memory managers installed (VEMM486) and the same DPMI server (the DOS 6.22 version I mentioned several posts back).

I have a suspicion that NP21Kai doesn't actually have support for those memory mapped IO ranges to the framebuffer, as the traditional way of accessing VRAM is via bank switched 32KB segments and not a linear framebuffer located at either 15-16MB, or at 4095MB.

[megatron-uk]

Some modicum of success was had:



I cannot get NP21Kai to activate the VRAM framebuffer at 16MB, it just doesn't appear to be an option that is available.

However...

If you do emit the correct control value to the framebuffer control port (send 0x01 to 0xE0102), it activates the linear framebuffer at 0xFFF00000 as well. Also, since that control port is above the 16bit address space, you need to farpoke it, rather than use outp()/outportb()/outportl(), all of which are for sending data to unsigned short port addresses.

So a concrete example; enabling 256 colour mode (rather than 16 colour):

Code Select Expand

#define PEGC_MODE_ADDR    0x6A
#define PEGC_BPPMODE_16c   0x20
#define PEGC_BPPMODE_256c  0x21
#define PEGC_BPPMODE_DUAL_PAGE 0x68
#define PEGC_BPPMODE_SINGLE_PAGE 0x69

// 256 color mode
outportb(PEGC_MODE_ADDR, 0x07);
outportb(PEGC_MODE_ADDR, PEGC_BPPMODE_256c);
outportb(PEGC_MODE_ADDR, PEGC_BPPMODE_DUAL_PAGE);
outportb(PEGC_MODE_ADDR, 0x06);

... and similar for the PEGC registers which are at E0100 and upwards:

Code Select Expand

#define PEGC_PIXELMODE_ADDR  0xE0100
#define PEGC_FB_CONTROL_ADDR 0xE0102
#define PEGC_PIXELMODE_PACKED 0x00
#define PEGC_PIXELMODE_PLANAR 0x01
#define PEGC_FB_ON  0x01
#define PEGC_FB_OFF 0x00

_farpokeb(_dos_ds, PEGC_PIXELMODE_ADDR, PEGC_PIXELMODE_PACKED);
_farpokeb(_dos_ds, PEGC_FB_CONTROL_ADDR, PEGC_FB_ON);

Then all I'm doing at the moment is creating a 512KB buffer in memory, filling it with some data (each byte represents one of 256 palette entries (each palette entry being an id 1-256, and 8 bits of red, green and blue), then doing something similar to a memcpy, but slightly more involved, since we have to copy from local memory to the DPMI provided 'far' memory region:

Code Select Expand

        // Set palette to be 256 shades of grey
for(i=1;i<256;i++){
vramSetPalette(i,i,i*2,i*3);
}

printf("Allocating buffer...\n");
buffer = (unsigned char *) malloc(PEGC_H_SIZE * PEGC_V_SIZE);
if (buffer == NULL){
printf("Unable to allocate output buffer\n");
return 1;
}

printf("Drawing into buffer...\n");
colour = 1;
ptr = buffer;
for (i = 0; i < (PEGC_H_SIZE * PEGC_V_SIZE); i++){
buffer[i] = (unsigned char) colour;
if (colour < 256){
colour++;
} else {
colour = 1;
}
}
buffer = ptr;
printf("Copying buffer to VRAM\n");
movedata(_my_ds(), buffer, selector, 0, (PEGC_H_SIZE * PEGC_V_SIZE));

- _my_ds() is a function which returns the segment for the current application
- buffer is my 512KB local malloc'd image buffer
- selector is a DPMI provided 'pointer' which points to the far data
- 0 is, erm... it was in the DJGPP docs, so I left it there!
- (PEGC_H_SIZE * PEGC_V_SIZE)... I think you can guess that specifies the number of bytes to copy.

It's not a solution for altering odd bytes here and there, as it uses 32bit word-aligned data. If you want to punch odd-numbered byte sized holes in the far data, then you use farpoke() and friends.

Okay, so with that out of the way, let's see if we can get anything useful to start showing on the screen!

[megatron-uk]

I've just altered the Watcom C source of that plasma demo I posted about earlier. Amended to use the far framebuffer, rather than the one at 16MB, and updated to use far pointers (as well as getting it to compile with GCC)... here it is, running on NP21Kai for probably the first time ever:



One thing I've noticed is that it's horrendously slow on NP21Kai - no more than 2fps (4-5fps if I use the 'no wait' display option from the menu). On my real 9821An its more like 20fps I would say. There's no maths going on in the display loop from what I can see (other than some trivial coordinate calculation), so my guess is that this isn't properly optimised in NP21Kai.

Edit: Whoops, just realised I compiled that without any optimisations - building it with -O3 had a terrific impact on speed... almost 6fps instead of 2.... :-|

[megatron-uk]

Mini-milestone time!

I'm able to load up and throw bitmaps around from RAM to VRAM...

Most of the code is lifted straight from my X68000 game launcher, so once I get to a certain point with redoing the hardware specific stuff, everything else should fall into place very, very quickly. Aside from commenting out the stuff that isn't implemented yet (keyboard handling, drawing graphics primitives, and a new text handler for my much, much nicer multi-colour 8x16 font) the only thing I've had to redo was the bitmap loading functions to accommodate the paletted display of the PC-98 compared to the raw 16bpp display of the X68000.

There's a few minor tweaks to file/directory handling of the config file parsers too; you'd think being MS-DOS they'd be the same, but no - DJGPP doesn't have a d_type entry in it's default dirent structure (may be in a current version, but not in the one supported in v2.03), but it does have a fairly well specced set of posix-style calls, opendir/readir, etc. Which makes navigating directory trees far easier.

Emulation speed is set at 486 SX-25 levels, so it gives an idea of how much faster it is when not having to byteswap everthing and convert from RGB to wacky GRBi!

[megatron-uk]

I've written up the information I've found and my successes in accessing the PEGC graphics hardware.

This includes:

• Setting up a GCC + DJGPP development environment
• Setting the PEGC display hardware registers
• Detecting and setting up the location of the PEGC framebuffer
• Setting up DOS DPMI mappers to access the framebuffer
• Transferring local memory to framebuffer VRAM

https://www.target-earth.net/wiki/doku.php?id=blog:pc98_devcode

I hope that someone else finds this useful!

[megatron-uk]

Starting to get there now...



Most stuff is working without many changes from the X68000 version, but obviously I've had to re-arrange the UI to suit 640x400 compared to the 512x512 of the Sharp.

One odd thing I'm battling now is a bug in either my code or the export of the UI bitmap data from GIMP - you can see some of the text display panels have a weird horizontally offset title (that's a hardcoded bitmap, rather than font - the font is only used for the data display within the panels. I don't know what's causing it as several of the bitmaps (genre, series) display fine.

Edit: Aha! It's the padded vs unpadded row size. I must have an error in my bmp decoding logic; if a row is not equal to a multiple of 4 bytes, then you're supposed to read the actual number of bytes in a row, then seek by the remainder of bytes to get to the next row. I must be off there somewhere.

[megatron-uk]

Fixed:



Next to add some sample metadata files to the couple of game folders I've got and test that the metadata loading/extracting functions written for the X68000 work without any further changes (shouldn't need any - they just use fopen and the same ini library as the main application config file).

[megatron-uk]

Startup routine now matches the familiar progress bar from the X68000 launcher, and image loading from an individual games metadata file (launch.dat) is working, including scrolling left and right through the images:







One thing I'm having a bit of trouble with at the moment is the mapping of palette entries - obviously the display is in 256 colour, I'm reserving the top 40 or so entries for the user interface and a dozen or so primary colours for shading, basic graphics primitive drawing, etc, so I'm left with about 210 free palette entries.

I've got ways for both GIMP and ImageMagick to produce images of no-more than 210 colours in indexed 8bpp mode (same as I used for the 32 colour UI bitmaps), but the colour mapping when I set the palette entries is off somewhat... the "Hell Ogre" title screen should be in shades of red/orange for example, and the "blue" water level screenshot is actually a orange desert planet!

I figure I'm switching the red and blue channel values somewhere, but since the same routines are shared with the UI image loading (but of course they're shades of grey, so mis-matched channels could be hard to spot!!!), I'm a little confused right now.

Anyway, it's coming along nicely.

[megatron-uk]

Yeah, red and blue channel data was reversed when reading from the colour table in the source bitmap. Here's how it looks now (desert planet!):



I'm sure the documentation I was reading for the BMP file format stated that the format of the entries in the colour table were in RGBA format, with 8 bits for each. So I was working with this code:

Code Select Expand

// Where i is the number of the colour entry.
fseek(bmp_image, bmpdata->colours_offset + (i * 4), SEEK_SET);
fread(&r, 1, 1, bmp_image);
fread(&g, 1, 1, bmp_image);
fread(&b, 1, 1, bmp_image);

Which gave me the rgb values for each colour entry in the table. But it seems that it's ordered as BGRA, and the following set of reads gives me the correct values:

Code Select Expand

fseek(bmp_image, bmpdata->colours_offset + (i * 4), SEEK_SET);
fread(&b, 1, 1, bmp_image);
fread(&g, 1, 1, bmp_image);
fread(&r, 1, 1, bmp_image);

I don't quite understand why the values of the blue and red channels are reversed. That's at least two graphics programmes that produced files that look like that though.

[megatron-uk]

Image loading now working great, also added a selection icon so that you know which line/game is highlighted.

I'm about at the point to start writing the launcher code itself - it will almost certainly be nothing more complex that writing a batch file which the launcher expects to call on exit; somewhat the same as LHES does on the X68000.

[kamiboy]

Well, it certainly is faster than the X68000 version, which is to be expected. Then again it will depend very much on ones model, and there is a lot more variability in the PC98 hardware ecosystem than in the X68000.

I know the Cx machine I got is a bit of an onion since some popular titles don't work right on it, or at all, due to hardware incompatibilities.

I believe the Cs2 are the most desired models. Having the right mix of fast CPU performance and graphics mode compatibility.

[megatron-uk]

Code now in github: https://github.com/megatron-uk/pc98launcher

The X68000 version is also there: https://github.com/megatron-uk/x68klauncher

There's a few bits and pieces I've improved on with this version (streamlining the artwork display, the scrolling/selection routine etc), so I'll port those changes across. Then its working on the launcher functionality for both of them.

If I drop the emulated speed right down to 386 20MHz (as indicated by NP21Kai, anyway), it still performs quite well. It also runs with just 3.6MB, of which go32-v2.exe reports 2474kb of DPMI memory available (DPMI ignores the first 1MB of RAM). Bear in mind that in that amount of RAM there's also my 256KB screen buffer and approx 400KB of bitmapped UI assets and fonts.

Of course, that's with just 6 game entries, it may vary as you increase the number of installed games (although each game 'object' in memory should be under 1KB in size).

[megatron-uk]

Started putting compiled versions of the launcher on my website, including some pre-packaged games, pre-filled metadata files and artwork (screenshots and box art):

https://www.target-earth.net/wiki/doku.php?id=blog:pc98_launcher_b#application_binaries

It's still got random bugs in (sometimes it doesn't start, but you run the command again and it launches fine), and there's no launch feature yet, but it's getting there, and most of the hard work is out of the way now.

Eventually I'll do the same as I'm doing for the Sharp X68000 launcher in my wiki and write a suite of documentation for how the code was written and what everything does, as a sort of 'how to write a PC-98/X68000 graphical application' guide.

[megatron-uk]

Latest update now adds support for pre-loading the full, user-defined game name from metadata at scrape-time (as opposed to using the directory name), rather than at the time you select the game. This is before sorting is applied, so the list is still alphabetised.

Obviously if you have a large number of games this will slow down the load time somewhat, so it's configurable via the launcher.ini file.

[megatron-uk]

The game browser/launcher is now at a point where it can:

- Scrape games from the hard drive
- Load metadata where provided
- Display artwork and screenshots
- Start a named exe, or one alternate exe (e.g. the main game exe and a config tool, or opening cinema)

Source code is up at: https://github.com/megatron-uk/pc98launcher

I've started adding pre-compiled binaries to my website: https://www.target-earth.net/wiki/doku.php?id=blog:pc98_launcher_b - I'll get the latest build up there, along with a basic README at some point today.

You'll also find some pre-packaged games on there, complete with all of the metadata and artwork ready to just unzip on your (real or virtual) PC-98 hard drive.

There's a few bugs left to squash, as well as some features that remain to be implemented (filter list by genre or series, for example). But I'm pretty pleased with what is working now.

[kamiboy]

Amazing work. I'll have to transfer this to my machine this weekend and test it out.

[megatron-uk]

On writing the filtering code now, should have it done by end of week; so you can say "filter -> RPG" and then just show all the games that are tagged as RPG within their metadata. Same goes for "series", so you can show just the Power Dolls or Lodoss War games.

Probably not needed if you just have a few dozen games installed, but if you've got hundreds, it will come in quite handy I believe.

[megatron-uk]

Can now filter games by genre or series keywords, as embedded in your metadata files within each game directory: