overview.md

(temporary overview)

An Introduction to Z88DK

Z88DK is a complete development toolkit for the z80, z180 and rabbit processors.

It contains two C compilers, an assembler / linker / librarian, data compression tools and a utility for processing the raw binaries into forms needed by specific targets.

It comes with an extensive library of functions written in assembly language that implements the C standard and many extensions. It holds the largest repository of z80 code on the internet.

Development in assembly language or C is completely integrated; projects can be 100% assembler, 100% C or any mixture of the two. The toolset treats both as first-class languages and is designed to make it very easy to mix them at will with C and asm functions being able to call each other or make use of the hand-optimized library functions.

The toolset is modern and has modern features. Most z80 assembly tools are simple in that they assume a small memory space (confined to 64k) or lack linking capability. Z88DK is able to generate ROMable code and code for multiple memory banks (bank assignment still requires human direction) and the linking capability means large bodies of code or data can be shared in libraries between projects with the linker only drawing out code and data that is actually used by a linking program.

We compare Z88DK with other commercial and non-commercial offerings using benchmarks to identify where Z88DK can be improved. Z88DK compares favourably and you can see that for yourself by investigating the benchmarks if you are interested.

The Tools

This is a quick overview of the tools included in Z88DK.

• ZCC is the toolchain's front end. zcc can generate an output binary out of any set of input source files.

• SCCZ80 is z88dk's native c compiler. sccz80 is derived from small c but has seen much development to the point that it is nearly c89 compliant.

• ZSDCC is z88dk's customization of the sdcc optimizing c compiler. Our patch makes sdcc compatible with the z88dk toolchain, gives it access to z88dk's extensive assembly language libraries and ready-made crts, addresses some of sdcc's code generation bugs and improves on sdcc's generated code. It has very good standards compliance with c89, some c99 and a little c11.

• Z80ASM (not to be confused with several external projects called z80asm) is a fully featured assembler / linker / librarian implementing sections.

• Z80NM is z80asm's companion archiver. It can provide a listing of functions or data encoded in an object or library file.

• APPMAKE processes the raw binaries generated by the toolkit into a form suitable for specific target machines. For example, it can generate intel hex files, tapes, ROMs, etc.

• TICKS is a command line z80 emulator that can be used to time execution speed of code fragments.

• ZX7 is a PC-side optimal lz77 data compression tool with companion decompression functions in the z80 library.

• DZX7 is a PC-side decompressor counterpart to zx7.

These tools are not normally directly invoked by the user:

• M4 acts as z88dk's macro preprocessor and can optionally process files ahead of the c preprocessor or assembler.

• ZCPP is the c preprocessor invoked for sccz80.

• ZSDCPP is the c preprocessor invoked for zsdcc.

• ZPRAGMA is used by the toolchain to process pragmas embedded in c source.

• COPT is a regular expression engine that is used as peephole optimizer for sccz80 and as a post-processing tool for both sccz80 and zsdcc.

Quick Guide to Using Z88DK

It is possible to use any of the tools listed above directly. However, it is much easier to use the front-end tool zcc to take care of everything.

zcc is a kind of magic bullet - it knows how to generate output files from a list of source files of any type (.c, .asm, .o, .asm.m4, .c.m4, etc). It can generate different types of output too - raw binaries, tape/rom/disk/ihx depending on target machine, object files, libraries, translations of C to asm, etc. In addition to that, it properly sets up the compile environment for a specific target machine. The zx spectrum does not have the same kind of display as the sega master system, but the C library must be able to print to screen on both systems. ZCC makes sure the right libraries are involved in the compile.

This brief guide will be concerned with using zcc to generate output. Later you can look at using the tools directly in conjunction with or separately from zcc but even if you become an expert in all of Z88DK's tools, you will find that zcc will be the primary avenue for using them.

Let's look at a simple but real example of a compile line using zcc found in a zx spectrum sp1 demo

This example contains some C code in "sp1_demo.c", an assembly language collision function in "collision.asm" and some graphics defined in assembly in "graphics.asm".

The two compile lines given are:

sccz80

zcc +zx -vn -clib=new -startup=31 @sp1demo.lst -o demo -create-app

zsdcc

zcc +zx -vn -SO3 -clib=sdcc_iy -startup=31 --max-allocs-per-node200000 @sp1demo.lst -o demo -create-app

There are two C compilers available in Z88DK (linux users have to compile zsdcc separately). In the first compile line, sccz80 is being selected to compile "sp1_demo.c" whereas in the second zsdcc is being selected. The differences in the two compile lines are down to different options being forwarded to the compilers. At this time, all C source must be compiled by only one of the compilers; it is not currently possible to mix object files generated by the two C compilers for technical reasons. So if you have C source to compile you will have to choose one of the compilers to compile all of it with.

• +zx Choose the zx spectrum as target. zcc will make sure the zx spectrum's library will be linked against and will set up the default options for the tools in the toolchain. (These are read from z88dk/lib/config/zx.cfg).

• -vn Keep zcc quiet. Changing to -v will have zcc print out every step it's taking to generate the output. Try a compile with -v so you can see what it is doing.

• -clib=new, -clib=sdcc_ix, -clib=sdcc_iy These select the newlib and which C compiler will be used (sccz80 for new and zsdcc for sdcc_*). As mentioned in the introduction there are two C libraries; if a clib switch does not appear, the compile will be using the classic library. More on this later. For zsdcc compiles, "sdcc_iy" is preferred. The choice between _ix and _iy is there because some targets reserve one index register for their own use. In a _iy compile, zsdcc is granted ownership of ix and the library will use iy. In a _ix compile, zsdcc and the library share ix.

• -startup=31 Selects which CRT will be used. The "C RunTime" is the short bit of startup code that runs before the program is started. Its responsibilities include setting up the memory map, initializing memory (the DATA and BSS sections) and deciding what to do on program exit.

Don't be alarmed if you only want to program in assembly language - the CRT is highly configurable and can be reduced to a very short stub. The setting up of the memory map makes sure that assembly language programs can make use of library functions without complication and can be easily targeted at ROMs or bankswitched memory.

CRTs and startup values vary by target but the default startup value is typically 0. For the newlib, which this compile is using, CRTs for the +zx target are documented in libsrc/_DEVELOPMENT/target/zx/zx_crt.asm.m4. They differ primarily by what drivers are attached to stdin, stdout and stderr. This startup value of 31 selects a CRT without anything in it. This is a "gamer" CRT intended to keep code size as small as possible.

CRTs and the compile environment can be further customized using pragmas. These pragmas can be specified on the compile line, embedded in C source or kept in a separate file (this latter option is the best). In this example, there are a number of pragmas embedded in "sp1demo.c". These are modifying the defaults set up by Z88DK in libsrc/_DEVELOPMENT/target/zx/crt_config.inc. This is a more advanced topic so we'll stop here but mention that these pragmas are documented on this page and specific ones for the zx here (Each IFNDEF quantity is an optional pragma).

For an example using a separate pragma file see two terminals.

For advanced users, the actual CRT code for the zx target can be found in libsrc/_DEVELOPMENT/target/zx/startup.

The set of pragmas supported is also library specific. The pragmas defined above are for the newlib. Some will also work with the classiclib but it's better to consult a separate reference.

The intention here is to be complete with the information but not to discuss in detail. The startup value is just a means to select a CRT with sensible defaults built in.

• @sp1demo.lst The leading @ indicates to zcc that this is a list file containing a list of source files to compile. You can open up "sp1demo.lst" and see the three source files to be compiled. In this case it would be just as easy to list these directly on the compile line and you can do that by replacing "@sp1demo.lst" with "sp1demo.c graphics.asm collision.asm". For larger projects list files are invaluable. See the +sms game Space Hawks for an example.

This method of compilation compiles everything all the time. It's fine for a release build that will run on both windows and linux, eg, but during development it may be preferable to set up a Makefile to take advantage of incremental compilation. This is especially the case if using zsdcc to compile C source because zsdcc can be very slow in even moderatley sized projects.

• -o demo "demo" will the root name of all output files. The default memory map supplied by the newlib broadly divides the binary into CODE, DATA and BSS sections. The output filenames will be some combination of "demo_CODE.bin", "demo_DATA.bin" and "demo_BSS.bin". Since the zx target is set up to generate programs resident in ram there will only be one output file "demo_CODE.bin" that contains everything unless the program places items into bankswitched memory.

• -create-app zcc will call appmake to generate suitable output from the binaries generated by the toolchain. For the zx, this means a tap file will be automatically generated (or an if2 cartridge if an if2 CRT is selected). As an example of more hands-on involvement in generating a tap file see the Black Star example. You can also enter appmake and appmake +zx on the command line to learn more about appmake.

• -SO3 --max-allocs-per-node200000 This is specific to zsdcc. "SO3" selects the aggressive peephole set used during code generation. It can substantially improve the quality of code generated by the compiler. As with any optimizer, there is a small chance that incorrect code can result and we do fix bugs from time to time. If you suspect a bugged compile, reduce the level to SO2. "max-allocs-per-node" controls how deeply zsdcc looks at code alternatives. This is a very high number that will lead to long compile times but also much better code quality. The default value is 3000.

Compiling an All-C Project

How to compile an all C project should be straightforward given the above. Just keep adding .c files to the compile line.

Compiling a Mixed C and Asm Project

The example was a mixed C and asm project. As you can see, zcc makes it easy to mix any set of source files on the compile line.

The interface between C and asm is also straightforward but there are some details best left explained elsewhere for asm programmers. For C, all interfaces are specified in header files. The purpose of the header file is to inform the C compiler that a function or piece of data is "somewhere out there" (extern) and to tell it how to access it properly. It's the linker's job to find these "somewhere out there" pieces and patch calls or data references as they are resolved. The linker will first try to find unresolved references among files in your project and then it will look at libraries in the order they are given to zcc.

In this example, there is no separate header file but there are extern declaration in "sp1demo.c":

extern int sp1_TestCollision(struct sp1_ss *s1, struct sp1_ss *s2, unsigned int tolerance) __z88dk_callee;
extern unsigned char sprite01[];  // gr_window will hold the address of the asm label _gr_window
extern unsigned char sprite02[];  // gr_window will hold the address of the asm label _gr_window
extern unsigned char sprite03[];  // gr_window will hold the address of the asm label _gr_window
extern unsigned char sprite04[];  // gr_window will hold the address of the asm label _gr_window

This tells the compiler that address "sp1_TestCollision" holds a function with "z88dk_callee" linkage (this linkage type explains how the parameters are passed).

The "sprite0n" names are declared arrays so they will be aliases for the address of an array.

On translation to asm, the C compiler will mangle names by prepending with a leading underscore. You can find these names defined in "graphics.asm" and "collision.asm". And that's how the connection is made from C to asm.

The connection from asm to C or asm to asm works similarly. Asm files define labels they want seen outside a file by making them "PUBLIC". They can refer to names defined elsewhere with "EXTERN". These EXTERN names can be names defined by you in another file (asm or C) or it can be a name defined in the library like "asm_printf" or "asm_float_mul". In this way asm can call or refer to functions and data in other asm files, other c files or the library.

Compiling an All-ASM Project

To compile an all asm project, don't add any C to the compile line :) The entry point to your asm program must be labelled "_main" and publicly exported.

By using Z88DK for an all-asm or nearly all-asm project, you gain optional access to Z88DK's assembly language libraries (this includes things like file i/o) and you benefit from the toolchain's ability to generate ROMable code if your code is destined for ROM or to generate bankswtichable code if your program will distribute across memory banks.

If you will be sharing your ASM code/data/music for others to use, Z88DK provides an ideal mechanism through libraries which few z80 assemblers support.

A Quick Note for ASM Code

Z88DK uses z80asm as a section-aware linking assembler. What this means is placement of code and data in memory is controlled by section assignment rather than ORG. This is an essential feature of a modern development environment.

A memory map is defined by the CRT to inform the linker how sections are placed in memory. It is also easy for users to define a custom memory map but this can be explained elsewhere. This memory map is the default one used by the newlib. It has many tiny sections aggregated into three main ones: CODE, DATA and BSS. Some targets may define more sections. For example, the +sms and +zx targets define additional sections BANK_XX (XX from 00 and up in hex) for bankswitched memory banks.

The CODE section will contain read-only code and data that can be held in ROM. The DATA section will contain self-modifying code or data that is initially non-zero that must be in RAM. If the program will be stored in ROM, this DATA section must be constructed in RAM before main() is called. The BSS section holds initially zero RAM variables.

This separation allows Z88DK to generate programs that can be stored in ROM. There are three compile models that Z88DK supports that are variations on this: the ROM model, the compressed ROM model and the RAM model. These are explained here. Compiles for home computers like the zx spectrum are almost always RAM model compiles and Z88DK sets that up by default. Games consoles like the Sega Master System store programs on ROM cartridge so for them a ROM model compile is set up by default.

z80asm can be used as a simpler non-sectioned assembler but as a component of Z88DK, interacting with the libraries and C code, and called through zcc, asm code and data needs to be assigned to appropriate sections to be made part of the output binary.

Z88DK defines a number of sections for ASM code that asm programmers can use: code_user, rodata_user, smc_user, data_user, bss_user.

Example use and explanation can be see below.

SECTION code_user
PUBLIC _main

; code_user is for read-only code
; called by the crt as entry to program
_main:
   ld hl,10
   ret

SECTION bss_user
PUBLIC myvar

; bss_user is for zeroed ram variables
; zeroed by crt when program started
myvar:
   defs 100

SECTION data_user
PUBLIC name

; data_user is for initially non-zero ram variables
; initialized in ram by crt if the program is in rom
name: 
   defm "John Smith"
   defb 0

SECTION smc_user
PUBLIC oops

; smc_user is for self-modifying code
; self modifying code is moved to ram if the program is in rom
oops:
   ld hl,0
   ld (oops),hl
   ret

SECTION rodata_user
PUBLIC placedinrom

; rodata_user if for constant data
; kept in rom if program is in rom
placedinrom:
   defb 1,2,3,4

SECTION BANK_06
PUBLIC aymusic

; assignment to a specific memory bank
; varies by target
aymusic:
   BINARY "rawdata.bin"

SECTION code_crt_init
EXTERN stufftodo

; wedged before call to main inside crt so initialization
; can happen before program is started

call stufftodo

; notice NO RET!

Assignment to a section is done with the SECTION directive. The currently active SECTION can be changed at any time within an asm file. (It should be noted that the C compilers can change active section on a file granularity only see AstroForce for the +sms). SECTION use is straightforward but for more examples have a look at the library code.

ASM code is not confined to the list of sections given above. ASM code and data can be added to any section defined in the memory map. One such special section is code_crt_init (see above) that can wedge in initialization code before main() is called. You can also invent your own sections with their own ORG addresses. Unknown sections will be output as a separate binary by the linker.

Distributing Code or Data as a Library to Share with Others

If you have subroutines or graphics and sound to share, you can place that in a library. People using your code or graphics can link against your library on the compile line with "-lname". Z88DK will automatically pull stuff out of your library as needed but only what is actually used.

The linker works on a file granularity so in order to minimize the amount that is extracted from your library, your code and data should be separated into logically independent individual files as finely as is reasonable. Z88DK's library is written like that and you can see an example of this function separation in stdlib.

It would be a good idea to follow the same conventions used by Z88DK when choosing function names, that is any assembly language entry point should be prefixed with "asm_" and any C entry point without that prefix. Z88DK also does namespacing in the form "asm_libname_make_sprite" for asm and "libname_make_sprite" for C. When making libraries make sure you are placing code and data into appropriate sections so that the user can control where stuff goes in memory either via the default Z88DK memory map or a custom memory map.

zcc can be told to make a library instead of a binary by adding the "-x" switch. See these selected command line options.

Further Information

There is more information connected to the newclib on this page. This page describes the +z80 generic target which is a bare bones target on which all newlib targets are based. The description of the tools and how they are used apply across all newlib targets.

See the examples directory for the newlib and for the classiclib. The examples only touch on a small portion of the libraries' features but they will give an idea of how things are done.

Use the header files to see how to call library functions or get an idea of what is there. For the newlib: https://github.com/z88dk/z88dk/tree/master/include/_DEVELOPMENT/clang . For the classic lib: https://github.com/z88dk/z88dk/tree/master/include

Visit us on the forums or at github to ask questions. Z88DK suffers from a lack of documentation and we know this. The size of Z88DK prevents us from quickly slapping documentation together so very brief overviews like this are all that is available now. If you have a "Can I...?" question the answer is probably yes, just ask and save yourself some frustration poking around.

This discussion has been newlib-centric. classiclib information can be found here. It should be stated that the newlib supports far fewer target z80 machines currently: +cpm, +rc2014, +sms, +z80, +z180, +zx. The classiclib supports over 70 machines.