m.py, my accidental build system

If you like transforming your statements into code, this is the place for you

Moderator: Programming Moderators

Post Reply
User avatar
sol_hsa
Posts: 91
Joined: Fri Jun 02, 2017 10:10 am

m.py, my accidental build system

Post by sol_hsa » Fri Sep 07, 2018 5:31 am

This is a bit tangential, but may be useful for someone.

I'm (slowly) working on a new speccy game, mostly using SDCC. I used a simple batch file to call the assembler, sdcc, linker, my .tap file builder and finally to launch the result in an emulator.

When something didn't compile, the batch file marched on, which was a bit annoying, so I added some early outs - if an object didn't appear on disk, the batch file would quit. The total build time balooned to about 10 seconds, so I split the sources to several files and considered adding checks to only build changed files. While that's possible in a bat file, it's messy, so I figured I'd make a simple python script for file age checking. Then it became obvious that things would be simpler if I just move everything to python. And while there, why not recursively check for include file dates too. And why not build things in parallel..

Anyway, I had accidentally made a configurable build system. Feel free to use and abuse, but don't sue me.

The latest version lives here: https://gist.github.com/jarikomppa/4c13 ... 246b5b2805

Configuring the script is pretty simple. First, add your source files to the "src" array:

Code: Select all

src = [
	"some_image1.png",
	"some_image2.png",
	"+custom_step.bat",
	"*",    
	"some_asm1.s",
	"some_asm.2s",
	"some_asm.3s",
	"some_c1.c"
	"some_c.2c"
	"some_c.3c"
	]
If there's build dependencies between files (such as some file generating a file another includes) a special sync marker "*" is needed. That waits until all previous tasks have finished. If custom build steps are needed (like deleting temp files or something), prefix the command with "+".

Next, add targets to the "product" array. These are only processed if the source files build correctly and something has changed.

Code: Select all

product = [
	"crt0.ihx", 
	"*", 
	"game.tap",
	"*",
	"+speccy game.tap"
	]
Same rules as before.

Finally, the build system needs to know how to deal with the source files. This is done via the "methods" dictionary:

Code: Select all

methods = {
	"png": ["h", "..\\tools\\png2bin %SRCFILE %DSTFILE -h" ],
	"c" : [ "rel", "..\\cc\\sdcc360\\bin\\sdcc -c -o %DSTFILE %SRCFILE -mz80 --no-std-crt0 --opt-code-speed --Werror --peep-asm --peep-return " ],
	"s" : [ "rel", "..\\cc\\sdcc360\\bin\\sdasz80 -xlos -g %DSTFILE %SRCFILE" ],
	"ihx" : [ "ihx", "..\\cc\\sdcc360\\bin\\sdcc -mz80 --no-std-crt0 --opt-code-speed --nostdlib --code-loc 0x6007 --data-loc 0x5b00 -Wl -b_HEADER=0x6000 %REL"],
	"tap" : [ "tap", "..\\tools\\mackarel crt0.ihx %DSTFILE gamename loader.scr -nosprestore -noei"]
	}
So when the source list includes "some_image.png", the build system sees that yep, this is a .png file, finds the method to turn .png into .h, checks that the .h file is older than the .png, and then runs the method, checking that the .h file is produced, and moves on.

If a .c file includes that .h file, it will also be rebuilt if the .h file changes.

The tags %SRCFILE and %DSTFILE are changed to the appropriate filenames, and new tags are dynamically generated for all target file types, so %REL is a list of all .rel files generated. (and %IHX is list of all .ihx files, %TAP is a list of all .tap files, etc)

And that's it!

Ped7g
Posts: 112
Joined: Mon Jul 16, 2018 7:11 pm

Re: m.py, my accidental build system

Post by Ped7g » Fri Sep 07, 2018 9:56 am

Sounds like what makefile does for 40 years, but now you have it with your own custom code and unclear rules... :) (don't take it like *personal* attack, I fully understand mastering makefiles is not simple, and actually most of the makefile use in real world projects makes me cringe anyway, it's just sort of sad/funny how much that tool misses its purpose).

Then again, to make this constructive, it would be nice if I did provide some decent Makefile example, and currently I don't have any for ZX projects, so feel free to ignore this as pure useless hate... :) ... maybe one day I will do one and post it too, so anyone can then pick whichever prefers.

User avatar
sol_hsa
Posts: 91
Joined: Fri Jun 02, 2017 10:10 am

Re: m.py, my accidental build system

Post by sol_hsa » Fri Sep 07, 2018 10:50 am

"unclear rules"? That's the very definition of makefiles.. =)

Anyway, like I explained: it was a total accident. I didn't start out with a goal of making a build system.

Post Reply