Bulkding banked code questions

If you like transforming your statements into code, this is the place for you

Moderator: Programming Moderators

User avatar
sol_hsa
Posts: 91
Joined: Fri Jun 02, 2017 10:10 am

Re: Bulkding banked code questions

Post by sol_hsa » Tue May 15, 2018 6:55 am

A related, but slightly tangential question; can the z80-targeting c compilers generate position independent code? (or is position independent code even possible/feasible on z80)

User avatar
varmfskii
Posts: 186
Joined: Fri Jun 23, 2017 1:13 pm
Location: Albuquerque, NM USA

Re: Bulkding banked code questions

Post by varmfskii » Tue May 15, 2018 2:42 pm

PIC: Possible, yes. In general, not practical.
Backer #2741 - TS2068, Byte, ZX Evolution

Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

Re: Bulkding banked code questions

Post by Alcoholics Anonymous » Tue May 15, 2018 3:24 pm

Position independent code is not a thing on the z80.. too many restrictions.
(and now that it's spelled out PIC=position independent code; for some reason I was thinking pic processor while reading varmfski's post)

However, you can generate code that can be patched after loading. Extra information can be provided that indicates where addresses have to be adjusted inside the binary to change the org address. A real example is SymbOS which loads programs into available memory and then patches them to run from any org address loaded to.

This is exposing information from the linking step so it's up to the linker to generate relocation information. We've tried that several times in z88dk's past and will come back to it if there is demand for it. There are problems for generating efficient relocation data when the org is adjusted in steps other than 256 bytes and there are problems when disjoint binary blocks are relocated independently (SymbOS collects disjoint blocks into one binary divided into parts, which makes it possible to know which part adjusted addresses originate in).

NextOS has introduced drivers which are 512 bytes long and have relocation information at their end. Drivers are loaded into divmmc memory and can be relocated by multiples of 256 (512?) bytes. At minimum we'll be adding this type to z88dk so that these relocatable drivers are automatically generated.
Last edited by Alcoholics Anonymous on Tue May 15, 2018 3:48 pm, edited 3 times in total.

Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

Re: Bulkding banked code questions

Post by Alcoholics Anonymous » Tue May 15, 2018 3:32 pm

Position independent data is less a problem of course. As long as the data does not contain pointers you don't have to do anything to it. Even if it does contain pointers it's not too difficult to iterate through the data and adjust pointer addresses.

Data is assigned to physical BANKs and PAGEs and labels attached to it have ORG addresses. But if you place memory holding data into a different mmu than the ORG indicates, you can still access the data as an offset into a different mmu page.

There is a function in arch/zxn.h that performs this function already:

Code: Select all

unsigned int zxn_addr_in_mmu(unsigned char mmu,unsigned int addr) __preserves_regs(b,c,d,iyl,iyh);

Code: Select all

asm_zxn_addr_in_mmu:

   ; return address at the same offset in another mmu slot
   ;
   ; enter :  a = new mmu slot
   ;         hl = address
   ;
   ; exit  : hl = address at same offset in new mmu slot
   ;
   ; uses  : af, e, hl

   rrca
   rrca
   rrca
   and $e0
   ld e,a
   
   ld a,h
   and $1f
   or e
   ld h,a
   
   ret
It takes an address and returns a new one in a different mmu slot. All it does is strip the top three bits off the address and replace with the mmu number.

There are a bunch of these small subroutines in the zx next library that ultimately we'll be inlining.

Bagpuss
Posts: 27
Joined: Tue Jun 06, 2017 9:13 pm

Re: Bulkding banked code questions

Post by Bagpuss » Tue May 15, 2018 10:33 pm

Bugger just wrote up some notes and it failed to submit :)

Still trying to get my head around it (juggling with having to learn AWS & Azure development too at the moment).

So what I think I need to do is:
Switch to using pages rather than banks as this is the Next preferred way
Put my code into pages and constants in there too
Keep main memory for globals variables and main loop which then swaps pages in.
I think to keep things simple, each page will only have one function.
I'll have to use #pragmas to set Org in each .c file won't I?
I can then compile each .c into its own bin file with --codesegPAGExx etc.
In terms of building a resultant program. I guess .sna won't cut it and I can only assume there isn't a zxn equivalent ready, so would I just need a routine in the .sna to load each .bin. page in the right bank & then poke it in? Sort of like a good old bootstrap loader?

Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

Re: Bulkding banked code questions

Post by Alcoholics Anonymous » Wed May 16, 2018 2:55 am

Bagpuss wrote:
Tue May 15, 2018 10:33 pm
Switch to using pages rather than banks as this is the Next preferred way
8k pages numbers yes, it's better. You can use sections "BANK_n_L" and "BANK_n_H" if you want the tools to enforce 8k page sizes but this does have n = 16k bank number. Sections "PAGE_n" are 8k page numbers but no enforcement is done; in all the code I've seen so far people are using PAGE_n and then checking that the output binaries stay under 8k themselves.
I'll have to use #pragmas to set Org in each .c file won't I?
No, all pragmas can be collected into your central pragma.inc file which we usually call "zpragma.inc" because "z" makes it show up at the bottom of listings. This is better than having them distributed in .c files. So just one pragma for main binary org ("#pragma output CRT_ORG_CODE = 32768"), one for stack location ("#pragma output REGISTER_SP = 32768"), org for pages you are using ("#pragma output CRT_ORG_PAGE_20 = 0x2000"), reduce printf, change font, alter text window dimensions, and so on.
In terms of building a resultant program. I guess .sna won't cut it and I can only assume there isn't a zxn equivalent ready, so would I just need a routine in the .sna to load each .bin. page in the right bank & then poke it in? Sort of like a good old bootstrap loader?
Yes that's right. The .sna will contain 16k banks 0-7. z88dk will output any other occupied pages or banks as separate binary files. You'll see them the first time you try it. There are some additional options you may want to pass to appmake on the compile line '-Cz"--clean --pages --fullsize"'. "--clean" is going to erase all the binary files that were put into the .sna. Any remaining are your responsibility to load using the esxdos api. "--pages" tells z88k you prefer the extra banks to be emitted as 8k binaries representing one 8k page (the default is 16k banks). "--fullsize" makes z88dk output a full 8k binary (or 16k for the default) representing an entire 8k page. Without this, it would generate a minimum size binary for the occupied page and let you know what the org is within the page.

NextOS allows the .sna to exceed the 128k sna size so another step we will probably take is append these extra banks to the .sna and put in a loader that will automatically load these pages for you. kev on facebook (and here but maybe under different name) did this for his terminator video where the whole thing is a single .sna with extra data appended to the end.

Another thing coming up is a .tap file that will automatically load the full program using a small basic loader. The basic loader sits in the "main bank" where the main binary is loaded so, just like regular 48k programs, you have to make sure you're ORGing the "main binary" into a high enough address that you're not clobbering the loader. In .tap form, the program will be able to properly return to basic if that's a thing you want.

Bagpuss
Posts: 27
Joined: Tue Jun 06, 2017 9:13 pm

Re: Bulkding banked code questions

Post by Bagpuss » Wed May 16, 2018 8:40 pm

I've switched back to basics a little now I'm heading in the right direction.
I've decided to page switch at 0xC000

Compile with

Code: Select all

zcc +zxn -c -m -clib=new -subtype=bin --codesegPAGE_32 --constsegPAGE_32 displayBank3Function.c -pragma-include:zpragma.inc -o bank32.bin -Cz"--clean --pages --fullsize"
the code is:

Code: Select all

#include <stdio.h>
void displayBank3Function()
{
    printf("In display Bank#3 Function");
}
This compiles to 332 bytes. What I was expecting based on the above is an 8K block by using the --pages --fullsize which I'm perplexed by. I've also tried the --filler 0 option too.
Ultimately I don't think padding out pages matters but I'm also looking at why my load routine is now in an infinite loop (well until it crashes). I suspect something with not re-allocating the stack as a first guess.

Bagpuss
Posts: 27
Joined: Tue Jun 06, 2017 9:13 pm

Re: Bulkding banked code questions

Post by Bagpuss » Wed May 16, 2018 10:06 pm

So where I am now is, that esxdos doesn't seem to care if you read beyond end of file, so I'm just reading a fixed length for now until I understand if that is me or esxdos

I have the binary loading into the correct bank (and can peek it to prove it now).

What I get is an issue with linking. Now I've got the error during link of :

Code: Select all

Error at file 'bankSwitchTest.c' line 63: symbol '_displayBank3Function' not defined
                   ^ ----     displayBank3Function();
What I think is happening is that it can't work out where the call will jump to as it has no knowledge of where its located. I guess somehow it needs to know from symbols what address within the org 0XC000 area that function will be placed in order to call it.

Update: I've now changed by compilation to:

Code: Select all

zcc +zxn -c -m -clib=new --c-code-in-asm --list -subtype=bin --codesegCODESWITCHING --constsegCONSTSWITCHING displayBank3Function.c -pragma-include:zpragma.inc -o bank32.bin -Cz"--clean --pages --fullsize"
zcc +zxn -c -m -clib=new --c-code-in-asm --list --codesegCODESWITCHING --constsegCONSTSWITCHING displayBank3Function.c -pragma-include:zpragma.inc -Cz"--clean --pages --fullsize"
zcc +zxn -c -m -clib=new --c-code-in-asm --list -pragma-include:zpragma.inc -Cz"--clean --pages --fullsize" displayLocalFunction.c
zcc +zxn -c -m -clib=new --c-code-in-asm --list @zprojectMain.lst -pragma-include:zpragma.inc -Cz"--clean --pages --fullsize"
zcc +zxn  -subtype=sna -startup=1 -m -O3 -SO3 -clib=new -m -o bankSwitchTest -clib=new  -lzxnext_layer2 -lzxnext_sprite bankSwitchTest.o displayLocalFunction.o displayBank3Function.o -create-app -Cz"--clean --pages --fullsize --exclude-sections CODESWITCHING" -pragma-include:zpragma.inc 
i.e. compile displayBank2Function.o and have a binary called bank3.bin
in the final linking I'm including the .o but also adding --exclude-sections which I assume means it should be generating symbols for calls but not binding the object into the .sna
It compiles, links then crashes on the call to displayBank2Function.
In the bankSwitchTest.map file there is a line for "_displayBank3Function = $C000 ; addr, public, , displayBank3Function_c, CODESWITCHING, displayBank3Function.c:4" so I assume it has the correct symbol.
By printing the address of the function it is indeed 0xC000 so my next dilema is why its crashing immediatly on calling this

Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

Re: Bulkding banked code questions

Post by Alcoholics Anonymous » Wed May 16, 2018 11:24 pm

Bagpuss wrote:
Wed May 16, 2018 8:40 pm

Code: Select all

zcc +zxn -c -m -clib=new -subtype=bin --codesegPAGE_32 --constsegPAGE_32 displayBank3Function.c -pragma-include:zpragma.inc -o bank32.bin -Cz"--clean --pages --fullsize"
the code is:

Code: Select all

#include <stdio.h>
void displayBank3Function()
{
    printf("In display Bank#3 Function");
}
This compiles to 332 bytes. What I was expecting based on the above is an 8K block by using the --pages --fullsize which I'm perplexed by. I've also tried the --filler 0 option too.
You're only making an object file here ("-c") so the building binary step does not happen. What zcc does is work top to bottom. c->asm->o, asm->o, asm.m4->asm->o, etc. The final destination is always an object file and then the last step is formation of a binary from all the object files. You can stop the process early. If you have "-m4" on zcc's line, it will stop after processing macros. If you have "-a" on the line, it will stop after translating to asm. If you have "-c", it stops after object files are made.

Here's an example:

display32.c (will put this into PAGE_32)

Code: Select all

#include <stdio.h>

void display32(void)
{
    printf("In display Page #32 Function");
}
main.c (will call display32() in PAGE_32)

Code: Select all

#include <arch/zxn.h>

extern void display32(void);

void main(void)
{
   // note that interrupts are disabled by default
   
   // this is important because we're about to page
   // the rom out of the bottom 8k and that's where
   // the rom's interrupt routine is at 0x38.
   
   // page 32 into mmu0 (the bottom 8k)
   
   ZXN_WRITE_MMU0(32);
   
   // display32()
   
   display32();
   
   // rom back into mmu0
   
   ZXN_WRITE_MMU0(0xff);
}
zpragma.inc (set org of PAGE_32 to 0 so we can map it into the bottom 8k mmu0)

Code: Select all

// if you need to move the stack pointer

// #pragma output REGISTER_SP = ...

// org page 32 into bottom 8k

#pragma output CRT_ORG_PAGE_32 = 0

// get rid of the heaps

#pragma output CLIB_MALLOC_HEAP_SIZE = 0
#pragma output CLIB_STDIO_HEAP_SIZE = 0

// reduce size of printf

#pragma printf = ""
The compile:

Code: Select all

zcc +zxn -c -vn -clib=sdcc_iy -SO3 --max-allocs-per-node200000 --codesegPAGE_32 --constsegPAGE_32 display32.c
zcc +zxn -c -vn -clib=sdcc_iy -SO3 --max-allocs-per-node200000 main.c
zcc +zxn -vn -m -startup=0 -clib=sdcc_iy display32.o main.o -o test -pragma-include:zpragma.inc -subtype=bin -Cz"--clean --pages --fullsize" -create-app
The first zcc puts display32.c into PAGE_32 (code and constant data. The constant data is the text string here). Library code still goes to the "main binary" so while dispay32() runs it needs to be able to see the printf code in the main binary. An object file "display32.o" is the output.

The second zcc puts main() into the (default) "main binary". The output is an object file "main.o".

The last zcc puts the object files together to make the output binary.

My output is:

Code: Select all

Creating test__.bin (org 0x8000)
Creating test__PAGE_032.bin (org 0x0000, 8154 tail bytes free)
The org shown for PAGEs and BANKs are the offset into the start of the page or bank. So an org of 0 for PAGE_32 means the binary "test__PAGE_032.bin" should be loaded into the start of PAGE_32. It's also saying there are 8154 bytes still available after the data stored in PAGE_32.

"test__.bin" is 3698 bytes and will contain all the library code and main(). It should be loaded at address 32768 (the default org for the main binary) into the "main bank" ie this usually means the standard bank 5,2,0 configuration seen in basic.

"test__PAGE_032.bin" will contain display32() and is output as an 8k block because of the "--fullsize" option.

Now because this is a "subtype=bin" compile, everything is output as raw binary files and it's your job to get them into memory somehow.

A "subtype=sna" file would put 16k banks 0-7 into the .sna file and then would also generate this "test__PAGE_032.bin" file because it can't be put into an sna. Your main() would then be responsible for loading from disk using esxdos.

Alcoholics Anonymous
Posts: 513
Joined: Mon May 29, 2017 7:00 pm

Re: Bulkding banked code questions

Post by Alcoholics Anonymous » Thu May 17, 2018 12:43 am

Bagpuss wrote:
Wed May 16, 2018 10:06 pm
So where I am now is, that esxdos doesn't seem to care if you read beyond end of file, so I'm just reading a fixed length for now until I understand if that is me or esxdos
It should stop at eof by returning 0 bytes read from esxdos_f_read() but I haven't personally tested it. Right now esxdos_f_read() is also returning 0 if there is an error but that will be changing when the api is updated. To detect an error you can check that "errno != 0" after an esxdos call (errno coming from the "<errno.h>" header). Looking at my own code, I have been using fstat to find out how big the file is but that shouldn't be necessary.

Code: Select all

Error at file 'bankSwitchTest.c' line 63: symbol '_displayBank3Function' not defined
                   ^ ----     displayBank3Function();
What I think is happening is that it can't work out where the call will jump to as it has no knowledge of where its located. I guess somehow it needs to know from symbols what address within the org 0XC000 area that function will be placed in order to call it.
(this is not what you're doing but I will leave here for others)

The compiler knows where it is - it's being placed in a section that has an org. I think what happened is you forgot to supply a prototype in "bankSwitchTest.c" to call it:

Code: Select all

extern void displayBank3Function(void);
This is what you'd normally stick in a header but you can just type it into a file calling the function too. The compiler treats each file independently and without this it won't have seen the function and won't know what you're referring to.
Update: I've now changed by compilation to:
Some edits below to remove stuff not needed in each line.

Code: Select all

zcc +zxn -c -clib=new --c-code-in-asm --list --codesegBANK_3 --constsegBANK_3 displayBank3Function.c
zcc +zxn -c -clib=new --c-code-in-asm --list displayLocalFunction.c
zcc +zxn -c -clib=new --c-code-in-asm --list @zprojectMain.lst -o main.o
zcc +zxn -startup=1 -m -clib=new -O3 main.o displayLocalFunction.o displayBank3Function.o -lzxnext_layer2 -lzxnext_sprite -o bankSwitchTest -pragma-include:zpragma.inc -subtype=sna -Cz"--clean --pages --fullsize" -create-app
When "-c" is there zcc is stopping after object files are made. Information about what to do when making a binary is not looked at.

In the third zcc line with the project list "@zprojectMain.lst" I've specified the name of an output object file "main.o". This will make a single consolidate object file out of all the source files specified in "@zprojectMain.lst". Otherwise there will be one object file for each source file in "@zprojectMain.lst". It's simpler to deal with one object file.

The last line is putting the binary together. "-SO3" is in optimizer option for zsdcc so it's not applicable to sccz80 (-clib=new). "-O3" for sccz80 means try to make smaller code and happens on top of -O2. "-m" is the map file which can only be made at the binary step because this is when the objects are put together in memory and where it will be know where things are. The three object files previously made are listed as input. I took out "--exclude-sections" - what this does is tell appmake not to include anything placed in those sections in processing. So any sections listed there will not be in the sna or processed at all as part of the extended pages or banks. Instead you'd just have raw binary files for those listed sections.
in the final linking I'm including the .o but also adding --exclude-sections which I assume means it should be generating symbols for calls but not binding the object into the .sna
Yes it will be excluded from the sna and if it's in a page or bank, it will be excluded from those too. Normally you don't want this.
Last edited by Alcoholics Anonymous on Thu May 17, 2018 1:10 am, edited 2 times in total.

Post Reply