2.1.1. The advantages of Assembly

Assembly can express very low-level things:

• you can access machine-dependent registers and I/O

• you can control the exact code behavior in critical sections that might otherwise involve deadlock between multiple software threads or hardware devices

• you can break the conventions of your usual compiler, which might allow some optimizations (like temporarily breaking rules about memory allocation, threading, calling conventions, etc)

• you can build interfaces between code fragments using incompatible conventions (e.g. produced by different compilers, or separated by a low-level interface)

• you can get access to unusual programming modes of your processor (e.g. 16 bit mode to interface startup, firmware, or legacy code on Intel PCs)

• you can produce reasonably fast code for tight loops to cope with a bad non-optimizing compiler (but then, there are free optimizing compilers available!)

• you can produce hand-optimized code perfectly tuned for your particular hardware setup, though not to someone else's

• you can write some code for your new language's optimizing compiler (that is something what very few ones will ever do, and even they not often)

• i.e. you can be in complete control of your code

2.1.2. The disadvantages of Assembly

Assembly is a very low-level language (the lowest above hand-coding the binary instruction patterns). This means

• it is long and tedious to write initially

• it is quite bug-prone

• your bugs can be very difficult to chase

• your code can be fairly difficult to understand and modify, i.e. to maintain

• the result is non-portable to other architectures, existing or upcoming

• your code will be optimized only for a certain implementation of a same architecture: for instance, among Intel-compatible platforms each CPU design and its variations (relative latency, through-output, and capacity, of processing units, caches, RAM, bus, disks, presence of FPU, MMX, 3DNOW, SIMD extensions, etc) implies potentially completely different optimization techniques. CPU designs already include: Intel 386, 486, Pentium, PPro, PII, PIII, PIV; Cyrix 5x86, 6x86, M2; AMD K5, K6 (K6-2, K6-III), K7 (Athlon, Duron). New designs keep popping up, so don't expect either this listing and your code to be up-to-date.

• you spend more time on a few details and can't focus on small and large algorithmic design, that are known to bring the largest part of the speed up (e.g. you might spend some time building very fast list/array manipulation primitives in assembly; only a hash table would have sped up your program much more; or, in another context, a binary tree; or some high-level structure distributed over a cluster of CPUs)

• a small change in algorithmic design might completely invalidate all your existing assembly code. So that either you're ready (and able) to rewrite it all, or you're tied to a particular algorithmic design

• On code that ain't too far from what's in standard benchmarks, commercial optimizing compilers outperform hand-coded assembly (well, that's less true on the x86 architecture than on RISC architectures, and perhaps less true for widely available/free compilers; anyway, for typical C code, GCC is fairly good);

• And in any case, as moderator John Levine says on comp.compilers,

  
  "compilers make it a lot easier to use complex data structures,
  and compilers don't get bored halfway through
  and generate reliably pretty good code."
  

  They will also correctly propagate code transformations throughout the whole (huge) program when optimizing code between procedures and module boundaries.

2.1.3. Assessment

All in all, you might find that though using assembly is sometimes needed, and might even be useful in a few cases where it is not, you'll want to:

• minimize use of assembly code

• encapsulate this code in well-defined interfaces

• have your assembly code automatically generated from patterns expressed in a higher-level language than assembly (e.g. GCC inline assembly macros)

• have automatic tools translate these programs into assembly code

• have this code be optimized if possible

• All of the above, i.e. write (an extension to) an optimizing compiler back-end.

Even when assembly is needed (e.g. OS development), you'll find that not so much of it is required, and that the above principles retain.

See the Linux kernel sources concerning this: as little assembly as needed, resulting in a fast, reliable, portable, maintainable OS. Even a successful game like DOOM was almost massively written in C, with a tiny part only being written in assembly for speed up.

2.2.1. General procedure to achieve efficient code

As Charles Fiterman says on comp.compilers about human vs computer-generated assembly code:

The human should always win and here is why.

First the human writes the whole thing in a high level language.
Second he profiles it to find the hot spots where it spends its time.
Third he has the compiler produce assembly for those small sections of code.
Fourth he hand tunes them looking for tiny improvements over the machine
generated code.

The human wins because he can use the machine.

2.2.2. Languages with optimizing compilers

Languages like ObjectiveCAML, SML, CommonLISP, Scheme, ADA, Pascal, C, C++, among others, all have free optimizing compilers that will optimize the bulk of your programs, and often do better than hand-coded assembly even for tight loops, while allowing you to focus on higher-level details, and without forbidding you to grab a few percent of extra performance in the above-mentioned way, once you've reached a stable design. Of course, there are also commercial optimizing compilers for most of these languages, too!

Some languages have compilers that produce C code, which can be further optimized by a C compiler: LISP, Scheme, Perl, and many other. Speed is fairly good.

2.2.3. General procedure to speed your code up

As for speeding code up, you should do it only for parts of a program that a profiling tool has consistently identified as being a performance bottleneck.

Hence, if you identify some code portion as being too slow, you should

• first try to use a better algorithm;

• then try to compile it rather than interpret it;

• then try to enable and tweak optimization from your compiler;

• then give the compiler hints about how to optimize (typing information in LISP; register usage with GCC; lots of options in most compilers, etc).

• then possibly fallback to assembly programming

Finally, before you end up writing assembly, you should inspect generated code, to check that the problem really is with bad code generation, as this might really not be the case: compiler-generated code might be better than what you'd have written, particularly on modern multi-pipelined architectures! Slow parts of a program might be intrinsically so. The biggest problems on modern architectures with fast processors are due to delays from memory access, cache-misses, TLB-misses, and page-faults; register optimization becomes useless, and you'll more profitably re-think data structures and threading to achieve better locality in memory access. Perhaps a completely different approach to the problem might help, then.

2.2.4. Inspecting compiler-generated code

There are many reasons to inspect compiler-generated assembly code. Here is what you'll do with such code:

• check whether generated code can be obviously enhanced with hand-coded assembly (or by tweaking compiler switches)

• when that's the case, start from generated code and modify it instead of starting from scratch

• more generally, use generated code as stubs to modify, which at least gets right the way your assembly routines interface to the external world

• track down bugs in your compiler (hopefully the rarer)

The standard way to have assembly code be generated is to invoke your compiler with the -S flag. This works with most Unix compilers, including the GNU C Compiler (GCC), but YMMV. As for GCC, it will produce more understandable assembly code with the -fverbose-asm command-line option. Of course, if you want to get good assembly code, don't forget your usual optimization options and hints!

3.1.1. Where to find GCC

GCC home page is http://gcc.gnu.org.

DOS port of GCC is called DJGPP.

There are two Win32 GCC ports: cygwin and mingw

There is also an OS/2 port of GCC called EMX; it works under DOS too, and includes lots of unix-emulation library routines. Look around the following site: ftp://ftp.leo.org/pub/comp/os/os2/leo/gnu/emx+gcc/.

3.1.2. Where to find docs for GCC Inline Asm

The documentation of GCC includes documentation files in TeXinfo format. You can compile them with TeX and print then result, or convert them to .info, and browse them with emacs, or convert them to .html, or nearly whatever you like; convert (with the right tools) to whatever you like, or just read as is. The .info files are generally found on any good installation for GCC.

The right section to look for is C Extensions::Extended Asm::

Section Invoking GCC::Submodel Options::i386 Options:: might help too. Particularly, it gives the i386 specific constraint names for registers: abcdSDB correspond to %eax, %ebx, %ecx, %edx, %esi, %edi and %ebp respectively (no letter for %esp).

The DJGPP Games resource (not only for game hackers) had page specifically about assembly, but it's down. Its data have nonetheless been recovered on the DJGPP site, that contains a mine of other useful information: http://www.delorie.com/djgpp/doc/brennan/.

GCC depends on GAS for assembling and follows its syntax (see below); do mind that inline asm needs percent characters to be quoted, they will be passed to GAS. See the section about GAS below.

Find lots of useful examples in the linux/include/asm-i386/ subdirectory of the sources for the Linux kernel.

3.1.3. Invoking GCC to build proper inline assembly code

Because assembly routines from the kernel headers (and most likely your own headers, if you try making your assembly programming as clean as it is in the linux kernel) are embedded in extern inline functions, GCC must be invoked with the -O flag (or -O2, -O3, etc), for these routines to be available. If not, your code may compile, but not link properly, since it will be looking for non-inlined extern functions in the libraries against which your program is being linked! Another way is to link against libraries that include fallback versions of the routines.

Inline assembly can be disabled with -fno-asm, which will have the compiler die when using extended inline asm syntax, or else generate calls to an external function named asm() that the linker can't resolve. To counter such flag, -fasm restores treatment of the asm keyword.

More generally, good compile flags for GCC on the x86 platform are

gcc -O2 -fomit-frame-pointer -W -Wall

-O2 is the good optimization level in most cases. Optimizing besides it takes more time, and yields code that is much larger, but only a bit faster; such over-optimization might be useful for tight loops only (if any), which you may be doing in assembly anyway. In cases when you need really strong compiler optimization for a few files, do consider using up to -O6.

-fomit-frame-pointer allows generated code to skip the stupid frame pointer maintenance, which makes code smaller and faster, and frees a register for further optimizations. It precludes the easy use of debugging tools (gdb), but when you use these, you just don't care about size and speed anymore anyway.

-W -Wall enables all useful warnings and helps you to catch obvious stupid errors.

You can add some CPU-specific -m486 or such flag so that GCC will produce code that is more adapted to your precise CPU. Note that modern GCC has -mpentium and such flags (and PGCC has even more), whereas GCC 2.7.x and older versions do not. A good choice of CPU-specific flags should be in the Linux kernel. Check the TeXinfo documentation of your current GCC installation for more.

-m386 will help optimize for size, hence also for speed on computers whose memory is tight and/or loaded, since big programs cause swap, which more than counters any "optimization" intended by the larger code. In such settings, it might be useful to stop using C, and use instead a language that favors code factorization, such as a functional language and/or FORTH, and use a bytecode- or wordcode- based implementation.

Note that you can vary code generation flags from file to file, so performance-critical files will use maximum optimization, whereas other files will be optimized for size.

To optimize even more, option -mregparm=2 and/or corresponding function attribute might help, but might pose lots of problems when linking to foreign code, including libc. There are ways to correctly declare foreign functions so the right call sequences be generated, or you might want to recompile the foreign libraries to use the same register-based calling convention...

Note that you can add make these flags the default by editing file /usr/lib/gcc-lib/i486-linux/2.7.2.3/specs or wherever that is on your system (better not add -W -Wall there, though). The exact location of the GCC specs files on system can be found by gcc -v.

3.1.4. Macro support

GCC allows (and requires) you to specify register constraints in your inline assembly code, so the optimizer always know about it; thus, inline assembly code is really made of patterns, not forcibly exact code.

Thus, you can put your assembly into CPP macros, and inline C functions, so anyone can use it in as any C function/macro. Inline functions resemble macros very much, but are sometimes cleaner to use. Beware that in all those cases, code will be duplicated, so only local labels (of 1: style) should be defined in that asm code. However, a macro would allow the name for a non local defined label to be passed as a parameter (or else, you should use additional meta-programming methods). Also, note that propagating inline asm code will spread potential bugs in them; so watch out doubly for register constraints in such inline asm code.

Lastly, the C language itself may be considered as a good abstraction to assembly programming, which relieves you from most of the trouble of assembling.

3.2.1. Where to find it

Find it at the same place where you've found GCC, in the binutils package. The latest version of binutils is available from http://sources.redhat.com/binutils/.

3.2.2. What is this AT&T syntax

Because GAS was invented to support a 32-bit unix compiler, it uses standard AT&T syntax, which resembles a lot the syntax for standard m68k assemblers, and is standard in the UNIX world. This syntax is neither worse, nor better than the Intel syntax. It's just different. When you get used to it, you find it much more regular than the Intel syntax, though a bit boring.

Here are the major caveats about GAS syntax:

• Register names are prefixed with %, so that registers are %eax, %dl and so on, instead of just eax, dl, etc. This makes it possible to include external C symbols directly in assembly source, without any risk of confusion, or any need for ugly underscore prefixes.

• The order of operands is source(s) first, and destination last, as opposed to the Intel convention of destination first and sources last. Hence, what in Intel syntax is mov eax,edx (move contents of register edx into register eax) will be in GAS syntax mov %edx,%eax.

• The operand size is specified as a suffix to the instruction name. The suffix is b for (8-bit) byte, w for (16-bit) word, and l for (32-bit) long. For instance, the correct syntax for the above instruction would have been movl %edx,%eax. However, gas does not require strict AT&T syntax, so the suffix is optional when size can be guessed from register operands, and else defaults to 32-bit (with a warning).

• Immediate operands are marked with a $ prefix, as in addl $5,%eax (add immediate long value 5 to register %eax).

• Missing operand prefix indicates that it is memory-contents; hence movl $foo,%eax puts the address of variable foo into register %eax, but movl foo,%eax puts the contents of variable foo into register %eax.

• Indexing or indirection is done by enclosing the index register or indirection memory cell address in parentheses, as in testb $0x80,17(%ebp) (test the high bit of the byte value at offset 17 from the cell pointed to by %ebp).

Note: There are few programs which may help you to convert source code between AT&T and Intel assembler syntaxes; some of the are capable of performing conversion in both directions.

GAS has comprehensive documentation in TeXinfo format, which comes at least with the source distribution. Browse extracted .info pages with Emacs or whatever. There used to be a file named gas.doc or as.doc around the GAS source package, but it was merged into the TeXinfo docs. Of course, in case of doubt, the ultimate documentation is the sources themselves! A section that will particularly interest you is Machine Dependencies::i386-Dependent::

Again, the sources for Linux (the OS kernel) come in as excellent examples; see under linux/arch/i386/ the following files: kernel/*.S, boot/compressed/*.S, math-emu/*.S.

If you are writing kind of a language, a thread package, etc., you might as well see how other languages ( OCaml, Gforth, etc.), or thread packages (QuickThreads, MIT pthreads, LinuxThreads, etc), or whatever else do it.

Finally, just compiling a C program to assembly might show you the syntax for the kind of instructions you want. See section above.

3.2.3. Intel syntax

Good news are that starting from binutils 2.10 release, GAS supports Intel syntax too. It can be triggered with .intel_syntax directive. Unfortunately this mode is not documented (yet?) in the official binutils manual, so if you want to use it, try to examine http://www.lxhp.in-berlin.de/lhpas86.html, which is an extract from AMD 64bit port of binutils 2.11.

3.2.4. 16-bit mode

Binutils (2.9.1.0.25+) now fully support 16-bit mode (registers and addressing) on i386 PCs. Use .code16 and .code32 to switch between assembly modes.

Also, a neat trick used by several people (including the oskit authors) is to force GCC to produce code for 16-bit real mode, using an inline assembly statement asm(".code16\n"). GCC will still emit only 32-bit addressing modes, but GAS will insert proper 32-bit prefixes for them.

3.2.5. Macro support

GAS has some macro capability included, as detailed in the texinfo docs. Moreover, while GCC recognizes .s files as raw assembly to send to GAS, it also recognizes .S files as files to pipe through CPP before feeding them to GAS. Again and again, see Linux sources for examples.

GAS also has GASP (GAS Preprocessor), which adds all the usual macroassembly tricks to GAS. GASP comes together with GAS in the GNU binutils archive. It works as a filter, like and . I have no idea on details, but it comes with its own texinfo documentation, which you would like to browse (info gasp), print, grok. GAS with GASP looks like a regular macro-assembler to me.

3.3.1. Where to find NASM

http://www.nasm.us, http://sourceforge.net/projects/nasm/

Binary release on your usual metalab mirror in devel/lang/asm/ directory. Should also be available as .rpm or .deb in your usual Linux distribution.

3.3.2. What it does

The syntax is Intel-style. Comprehensive macroprocessing support is integrated.

Supported object file formats are bin, aout, coff, elf, as86, obj (DOS), win32, rdf (their own format).

NASM can be used as a backend for the free LCC compiler (support files included).

Unless you're using BCC as a 16-bit compiler (which is out of scope of this 32-bit HOWTO), you should definitely use NASM instead of say AS86 or MASM, because it runs on all platforms.

NASM comes with a disassembler, NDISASM.

Its hand-written parser makes it much faster than GAS, though of course, it doesn't support three bazillion different architectures. If you like Intel-style syntax, as opposed to GAS syntax, then it should be the assembler of choice...

Note: There are few programs which may help you to convert source code between AT&T and Intel assembler syntaxes; some of the are capable of performing conversion in both directions.

3.4.1. AS86

AS86 is a 80x86 assembler (16-bit and 32-bit) with integrated macro support. It has mostly Intel-syntax, though it differs slightly as for addressing modes. Some time ago it was used in a several projects, including the Linux kernel, but eventually most of those projects have moved to GAS or NASM. AFAIK, only ELKS continues to use it.

AS86 can be found at http://www.debath.co.uk/dev86/, in the bin86 package with linker (ld86), or as separate archive. Documentation is available as the man page and as.doc from the source package. When in doubt, the source code itself is often a good doc: though it is not very well commented, the programming style is straightforward. AS86 is part of a number of BSD and Linux distributions.

AS86 is primarily a 16 bit assembler.

	Using AS86 with BCC
	Here's the GNU Makefile entry for using BCC to transform .s asm into both a.out .o object and .l listing: %.o %.l: %.s bcc -3 -G -c -A-d -A-l -A$*.l -o $*.o $< Remove the %.l, -A-l, and -A$*.l, if you don't want any listing. If you want something else than a.out, you can examine BCC docs about the other supported formats, and/or use the objcopy utility from the GNU binutils package.

3.4.2. YASM

YASM is a complete rewrite of the NASM assembler under the "new" BSD License. It is designed from the ground up to allow for multiple syntaxes to be supported (eg, NASM, TASM, GAS, etc.) in addition to multiple output object formats including COFF, Win32 and Mach-O. Another primary module of the overall design is an optimizer module.

3.4.3. FASM

FASM (flat assembler) is a fast, efficient 80x86 assembler that runs in 'flat real mode'. Unlike many other 80x86 assemblers, FASM only requires the source code to include the information it really needs. It is written in itself and is very small and fast. It runs on DOS/Windows/Linux and can produce flat binary, DOS EXE, Win32 PE, COFF and Linux ELF output. See http://flatassembler.net.

3.4.4. OSIMPA (SHASM)

osimpa is an assembler for Intel 80386 processors and subsequent, written entirely in the GNU Bash command interpreter shell. The predecessor of osimpa was shasm. osimpa is much cleaned up, can create useful Linux ELF executables, and has various HLL-like extensions and programmer convenience commands.

It is (of course) slower than other assemblers. It has its own syntax (and uses its own names for x86 opcodes) Fairly good documentation is included. Check it out: ftp://linux01.gwdg.de/pub/cLIeNUX/interim/ (Access is password controlled). You will probably not use it on regular basis, but at least it deserves your interest as an interesting idea.

3.4.5. AASM

Aasm is an advanced assembler designed to support several target architectures. It has been designed to be easily extended and, should be considered as a good alternative to monolithic assembler development for each new target CPUs and binary file formats.

Aasm should make assembly programming easier for developer, by providing a set of advanced features including symbol scopes, an expressions engine, big integer support, macro capability, numerous and accurate warning messages. Its dynamic modular architecture enables Aasm to extend its set of features with plug-ins by taking advantages of dynamic libraries.

The input module supports Intel syntax (like nasm, tasm, masm, etc.). The x86 assembler module supports all opcodes up to P6 including MMX, SSE and 3DNow! extensions. F-CPU and SPARC assembler modules are under development. Several output modules are available for ELF, COFF, IntelHex, and raw binary formats.

http://savannah.nongnu.org/projects/aasm/

3.4.6. TDASM

The Table Driven Assembler (TDASM) is a free portable cross assembler for any kind of assembly language. It should be possible to use it as a compiler to any target microprocessor using a table that defines the compilation process.

It is available from http://www.penguin.cz/~niki/tdasm/ but is seems it is no longer actively maintained.

3.4.7. HLA

HLA is a High Level Assembly language. It uses a high level language like syntax (similar to Pascal, C/C++, and other HLLs) for variable declarations, procedure declarations, and procedure calls. It uses a modified assembly language syntax for the standard machine instructions. It also provides several high level language style control structures (if, while, repeat..until, etc.) that help you write much more readable code.

HLA is free and comes with source, Linux and Win32 versions available. On Win32 you need MASM and a 32-bit version of MS-link on Win32, on Linux you need GAS, because HLA produces specified assembler code and uses that assembler for final assembling and linking.

3.4.8. TALC

TALC is another free MASM/Win32 based compiler (however it supports ELF output, does it?).

TAL stands for Typed Assembly Language. It extends traditional untyped assembly languages with typing annotations, memory management primitives, and a sound set of typing rules, to guarantee the memory safety, control flow safety,and type safety of TAL programs. Moreover, the typing constructs are expressive enough to encode most source language programming features including records and structures, arrays, higher-order and polymorphic functions, exceptions, abstract data types, subtyping, and modules. Just as importantly, TAL is flexible enough to admit many low-level compiler optimizations. Consequently, TAL is an ideal target platform for type-directed compilers that want to produce verifiably safe code for use in secure mobile code applications or extensible operating system kernels.

3.4.9. Free Pascal

Free Pascal has an internal 32-bit assembler (based on NASM tables) and a switchable output that allows:

• Binary (ELF and coff when crosscompiled .o) output

• NASM

• MASM

• TASM

• AS (aout,coff, elf32)

The MASM and TASM output are not as good debugged as the other two, but can be handy sometimes.

The assembler's look and feel are based on Turbo Pascal's internal BASM, and the IDE supports similar highlighting, and FPC can fully integrate with gcc (on C level, not C++).

Using a dummy RTL, one can even generate pure assembler programs.

3.4.10. Win32Forth assembler

Win32Forth is a free 32-bit ANS FORTH system that successfully runs under Win32s, Win95, Win/NT. It includes a free 32-bit assembler (either prefix or postfix syntax) integrated into the reflective FORTH language. Macro processing is done with the full power of the reflective language FORTH; however, the only supported input and output contexts is Win32For itself (no dumping of .obj file, but you could add that feature yourself, of course). Find it at ftp://ftp.forth.org/pub/Forth/Compilers/native/windows/Win32For/.

3.4.11. Terse

Terse is a programming tool that provides THE most compact assembler syntax for the x86 family! However, it is evil proprietary software. It is said that there was a project for a free clone somewhere, that was abandoned after worthless pretenses that the syntax would be owned by the original author. Thus, if you're looking for a nifty programming project related to assembly hacking, I invite you to develop a terse-syntax frontend to NASM, if you like that syntax.

As an interesting historic remark, on comp.compilers,

1999/07/11 19:36:51, the moderator wrote:

"There's no reason that assemblers have to have awful syntax.  About
30 years ago I used Niklaus Wirth's PL360, which was basically a S/360
assembler with Algol syntax and a a little syntactic sugar like while
loops that turned into the obvious branches.  It really was an
assembler, e.g., you had to write out your expressions with explicit
assignments of values to registers, but it was nice.  Wirth used it to
write Algol W, a small fast Algol subset, which was a predecessor to
Pascal.  As is so often the case, Algol W was a significant
improvement over many of its successors. -John"

3.4.12. Non-free and/or Non-32bit x86 assemblers

You may find more about them, together with the basics of x86 assembly programming, in the Raymond Moon's x86 assembly FAQ.

Note that all DOS-based assemblers should work inside the Linux DOS Emulator, as well as other similar emulators, so that if you already own one, you can still use it inside a real OS. Recent DOS-based assemblers also support COFF and/or other object file formats that are supported by the GNU BFD library, so that you can use them together with your free 32-bit tools, perhaps using GNU objcopy (part of the binutils) as a conversion filter.

4.1.1. CPP

CPP is truly not very expressive, but it's enough for easy things, it's standard, and called transparently by GCC.

As an example of its limitations, you can't declare objects so that destructors are automatically called at the end of the declaring block; you don't have diversions or scoping, etc.

CPP comes with any C compiler. However, considering how mediocre it is, stay away from it if by chance you can make it without C.

4.1.2. M4

M4 gives you the full power of macroprocessing, with a Turing equivalent language, recursion, regular expressions, etc. You can do with it everything that CPP cannot.

See macro4th (this4th) as an example of advanced macroprogramming using m4.

However, its disfunctional quoting and unquoting semantics force you to use explicit continuation-passing tail-recursive macro style if you want to do advanced macro programming (which is remindful of TeX -- BTW, has anyone tried to use TeX as a macroprocessor for anything else than typesetting ?). This is NOT worse than CPP that does not allow quoting and recursion anyway.

The right version of M4 to get is GNU m4 which has the most features and the least bugs or limitations of all. m4 is designed to be slow for anything but the simplest uses, which might still be ok for most assembly programming (you are not writing million-lines assembly programs, are you?).

4.1.3. Macroprocessing with your own filter

You can write your own simple macro-expansion filter with the usual tools: perl, awk, sed, etc. It can be made rather quickly, and you control everything. But, of course, power in macroprocessing implies "the hard way".

4.2.1. Backends from compilers

Compilers like GCC, SML/NJ, Objective CAML, MIT-Scheme, CMUCL, etc, do have their own generic assembler backend, which you might choose to use, if you intend to generate code semi-automatically from the according languages, or from a language you hack: rather than write great assembly code, you may instead modify a compiler so that it dumps great assembly code!

4.2.2. The New-Jersey Machine-Code Toolkit

There is a project, using the programming language Icon (with an experimental ML version), to build a basis for producing assembly-manipulating code. See around http://www.eecs.harvard.edu/~nr/toolkit/

4.2.3. TUNES

The TUNES Project for a Free Reflective Computing System is developing its own assembler as an extension to the Scheme language, as part of its development process. It doesn't run at all yet, though help is welcome.

The assembler manipulates abstract syntax trees, so it could equally serve as the basis for a assembly syntax translator, a disassembler, a common assembler/compiler back-end, etc. Also, the full power of a real language, Scheme, make it unchallenged as for macroprocessing/metaprogramming.

5.1.1. Linking to GCC

This is the preferred way if you are developing mixed C-asm project. Check GCC docs and examples from Linux kernel .S files that go through gas (not those that go through as86).

32-bit arguments are pushed down stack in reverse syntactic order (hence accessed/popped in the right order), above the 32-bit near return address. %ebp, %esi, %edi, %ebx are callee-saved, other registers are caller-saved; %eax is to hold the result, or %edx:%eax for 64-bit results.

FP stack: I'm not sure, but I think result is in st(0), whole stack caller-saved.

Note that GCC has options to modify the calling conventions by reserving registers, having arguments in registers, not assuming the FPU, etc. Check the i386 .info pages.

Beware that you must then declare the cdecl or regparm(0) attribute for a function that will follow standard GCC calling conventions. See C Extensions::Extended Asm:: section from the GCC info pages. See also how Linux defines its asmlinkage macro.

5.1.2. ELF vs a.out problems

Some C compilers prepend an underscore before every symbol, while others do not.

Particularly, Linux a.out GCC does such prepending, while Linux ELF GCC does not.

If you need to cope with both behaviors at once, see how existing packages do. For instance, get an old Linux source tree, the Elk, qthreads, or OCaml.

You can also override the implicit C->asm renaming by inserting statements like

void foo asm("bar") (void);

to be sure that the C function foo() will be called really bar in assembly.

Note that the objcopy utility from the binutils package should allow you to transform your a.out objects into ELF objects, and perhaps the contrary too, in some cases. More generally, it will do lots of file format conversions.

5.1.3. Direct Linux syscalls

Often you will be told that using C library (libc) is the only way, and direct system calls are bad. This is true. To some extent. In general, you must know that libc is not sacred, and in most cases it only does some checks, then calls kernel, and then sets errno. You can easily do this in your program as well (if you need to), and your program will be dozen times smaller, and this will result in improved performance as well, just because you're not using shared libraries (static binaries are faster). Using or not using libc in assembly programming is more a question of taste/belief than something practical. Remember, Linux is aiming to be POSIX compliant, so does libc. This means that syntax of almost all libc "system calls" exactly matches syntax of real kernel system calls (and vice versa). Besides, GNU libc(glibc) becomes slower and slower from version to version, and eats more and more memory; and so, cases of using direct system calls become quite usual. However, the main drawback of throwing libc away is that you will possibly need to implement several libc specific functions (that are not just syscall wrappers) on your own (printf() and Co.), and you are ready for that, aren't you? :-)

Here is summary of direct system calls pros and cons.

Pros:

• the smallest possible size; squeezing the last byte out of the system

• the highest possible speed; squeezing cycles out of your favorite benchmark

• full control: you can adapt your program/library to your specific language or memory requirements or whatever

• no pollution by libc cruft

• no pollution by C calling conventions (if you're developing your own language or environment)

• static binaries make you independent from libc upgrades or crashes, or from dangling #! path to an interpreter (and are faster)

• just for the fun out of it (don't you get a kick out of assembly programming?)

Cons:

• If any other program on your computer uses the libc, then duplicating the libc code will actually wastes memory, not saves it.

• Services redundantly implemented in many static binaries are a waste of memory. But you can make your libc replacement a shared library.

• Size is much better saved by having some kind of bytecode, wordcode, or structure interpreter than by writing everything in assembly. (the interpreter itself could be written either in C or assembly.) The best way to keep multiple binaries small is to not have multiple binaries, but instead to have an interpreter process files with #! prefix. This is how OCaml works when used in wordcode mode (as opposed to optimized native code mode), and it is compatible with using the libc. This is also how Tom Christiansen's Perl PowerTools reimplementation of unix utilities works. Finally, one last way to keep things small, that doesn't depend on an external file with a hardcoded path, be it library or interpreter, is to have only one binary, and have multiply-named hard or soft links to it: the same binary will provide everything you need in an optimal space, with no redundancy of subroutines or useless binary headers; it will dispatch its specific behavior according to its argv[0]; in case it isn't called with a recognized name, it might default to a shell, and be possibly thus also usable as an interpreter!

• You cannot benefit from the many functionalities that libc provides besides mere linux syscalls: that is, functionality described in section 3 of the manual pages, as opposed to section 2, such as malloc, threads, locale, password, high-level network management, etc.

• Therefore, you might have to reimplement large parts of libc, from printf() to malloc() and gethostbyname. It's redundant with the libc effort, and can be quite boring sometimes. Note that some people have already reimplemented "light" replacements for parts of the libc - - check them out! (Redhat's minilibc, Rick Hohensee's libsys, Felix von Leitner's dietlibc, asmutils project is working on pure assembly libc)

• Static libraries prevent you to benefit from libc upgrades as well as from libc add-ons such as the zlibc package, that does on-the-fly transparent decompression of gzip-compressed files.

• The few instructions added by the libc can be a ridiculously small speed overhead as compared to the cost of a system call. If speed is a concern, your main problem is in your usage of system calls, not in their wrapper's implementation.

• Using the standard assembly API for system calls is much slower than using the libc API when running in micro-kernel versions of Linux such as L4Linux, that have their own faster calling convention, and pay high convention-translation overhead when using the standard one (L4Linux comes with libc recompiled with their syscall API; of course, you could recompile your code with their API, too).

• See previous discussion for general speed optimization issue.

• If syscalls are too slow to you, you might want to hack the kernel sources (in C) instead of staying in userland.

If you've pondered the above pros and cons, and still want to use direct syscalls, then here is some advice.

• You can easily define your system calling functions in a portable way in C (as opposed to unportable using assembly), by including asm/unistd.h, and using provided macros.

• Since you're trying to replace it, go get the sources for the libc, and grok them. (And if you think you can do better, then send feedback to the authors!)

• As an example of pure assembly code that does everything you want, examine .

Basically, you issue an int 0x80, with the __NR_syscallname number (from asm/unistd.h) in eax, and parameters (up to six) in ebx, ecx, edx, esi, edi, ebp respectively.

Result is returned in eax, with a negative result being an error, whose opposite is what libc would put into errno. The user-stack is not touched, so you needn't have a valid one when doing a syscall.

Passing sixth parameter in ebp appeared in Linux 2.4, previous Linux versions understand only 5 parameters in registers.

Linux Kernel Internals, and especially How System Calls Are Implemented on i386 Architecture? chapter will give you more robust overview.

As for the invocation arguments passed to a process upon startup, the general principle is that the stack originally contains the number of arguments argc, then the list of pointers that constitute *argv, then a null-terminated sequence of null-terminated variable=value strings for the environment. For more details, do examine , read the sources of C startup code from your libc (crt0.S or crt1.S), or those from the Linux kernel (exec.c and binfmt_*.c in linux/fs/).

5.1.4. Hardware I/O under Linux

If you want to perform direct port I/O under Linux, either it's something very simple that does not need OS arbitration, and you should see the IO-Port-Programming mini-HOWTO; or it needs a kernel device driver, and you should try to learn more about kernel hacking, device driver development, kernel modules, etc, for which there are other excellent HOWTOs and documents from the LDP.

Particularly, if what you want is Graphics programming, then do join one of the GGI or XFree86 projects.

Some people have even done better, writing small and robust XFree86 drivers in an interpreted domain-specific language, GAL, and achieving the efficiency of hand C-written drivers through partial evaluation (drivers not only not in asm, but not even in C!). The problem is that the partial evaluator they used to achieve efficiency is not free software. Any taker for a replacement?

Anyway, in all these cases, you'll be better when using GCC inline assembly with the macros from linux/asm/*.h than writing full assembly source files.

5.1.5. Accessing 16-bit drivers from Linux/i386

Such thing is theoretically possible (proof: see how DOSEMU can selectively grant hardware port access to programs), and I've heard rumors that someone somewhere did actually do it (in the PCI driver? Some VESA access stuff? ISA PnP? dunno). If you have some more precise information on that, you'll be most welcome. Anyway, good places to look for more information are the Linux kernel sources, DOSEMU sources, and sources for various low-level programs under Linux. (perhaps GGI if it supports VESA).

Basically, you must either use 16-bit protected mode or vm86 mode.

The first is simpler to setup, but only works with well-behaved code that won't do any kind of segment arithmetics or absolute segment addressing (particularly addressing segment 0), unless by chance it happens that all segments used can be setup in advance in the LDT.

The later allows for more "compatibility" with vanilla 16-bit environments, but requires more complicated handling.

In both cases, before you can jump to 16-bit code, you must

• mmap any absolute address used in the 16-bit code (such as ROM, video buffers, DMA targets, and memory-mapped I/O) from /dev/mem to your process' address space,

• setup the LDT and/or vm86 mode monitor.

• grab proper I/O permissions from the kernel (see the above section)

Again, carefully read the source for the stuff contributed to the DOSEMU project, particularly these mini-emulators for running ELKS and/or simple .COM programs under Linux/i386.

6.1.1. Tools you need

First of all you need assembler (compiler) -- nasm or gas.

Second, you need a linker -- ld, since assembler produces only object code. Almost all distributions have gas and ld, in the binutils package.

As for nasm, you may have to download and install binary packages for Linux and docs from the nasm site; note that several distributions (Stampede, Debian, SuSe, Mandrake) already have nasm, check first.

If you're going to dig in, you should also install include files for your OS, and if possible, kernel source.

6.2.1. Program layout

Linux is 32-bit, runs in protected mode, has flat memory model, and uses the ELF format for binaries.

A program can be divided into sections: .text for your code (read-only), .data for your data (read-write), .bss for uninitialized data (read-write); there can actually be a few other standard sections, as well as some user-defined sections, but there's rare need to use them and they are out of our interest here. A program must have at least .text section.

Now we will write our first program. Here is sample code:

6.2.2. NASM (hello.asm)

section .text ;section declaration ;we must export the entry point to the ELF linker or global _start ;loader. They conventionally recognize _start as their ;entry point. Use ld -e foo to override the default. _start: ;write our string to stdout mov edx,len ;third argument: message length mov ecx,msg ;second argument: pointer to message to write mov ebx,1 ;first argument: file handle (stdout) mov eax,4 ;system call number (sys_write) int 0x80 ;call kernel ;and exit mov ebx,0 ;first syscall argument: exit code mov eax,1 ;system call number (sys_exit) int 0x80 ;call kernel section .data ;section declaration msg db "Hello, world!",0xa ;our dear string len equ $ - msg ;length of our dear string

6.2.3. GAS (hello.S)

.text # section declaration # we must export the entry point to the ELF linker or .global _start # loader. They conventionally recognize _start as their # entry point. Use ld -e foo to override the default. _start: # write our string to stdout movl $len,%edx # third argument: message length movl $msg,%ecx # second argument: pointer to message to write movl $1,%ebx # first argument: file handle (stdout) movl $4,%eax # system call number (sys_write) int $0x80 # call kernel # and exit movl $0,%ebx # first argument: exit code movl $1,%eax # system call number (sys_exit) int $0x80 # call kernel .data # section declaration msg: .ascii "Hello, world!\n" # our dear string len = . - msg # length of our dear string

6.3.1. Producing object code

First step of building an executable is compiling (or assembling) object file from the source:

For nasm example:

$ nasm -f elf hello.asm

For gas example:

$ as -o hello.o hello.S

This makes hello.o object file.

6.3.2. Producing executable

Second step is producing executable file itself from the object file by invoking linker:

$ ld -s -o hello hello.o

This will finally build hello executable.

Hey, try to run it... Works? That's it. Pretty simple.