Do your job with make!

ArticleCategory: []

Software Development

AuthorImage:[Here comes a small picture of you]

TranslationInfo:[Info concerning writer(s) and translator(s)]

AboutTheAuthor:[a short biography about the author]

Wilbert Berendsen is a professional musician and a enthousiastic Linux user. Once, he intensively hacked assembly for the Z80. Nowadays, he's using Linux for all his production work. Just for fun, he writes introductory articles and he maintains a small website on http://www.xs4all.nl/~wbsoft/. Viva open source!

Abstract:[a small summary/description of this article]

This article over make shows how make works and that it can be used for more things than just software development.

ArticleIllustration:[illustration]

ArticleBody:[The real article: put the text and html-codes here]

Introduction

Almost anyone who uses Linux has applied the make program sometimes. It does its job if a program or kernel is build from the source code, if a package is installed, and so on. 'Make' is an important tool for software development. However, make has much more possibilities!

In this document, we will see that make can be a powerful tool for daily jobs, such as writing articles, books or composing a nice website. During this introduction, many other 'unix tricks' will be handled. At the end of this story, some more tips will be presented on using make. Please note: we are talking about Linux, but in principle it is possible to use make on every operating system.

Example: building a website

We want to build a website that is maintained by different people. Jan takes care of two pages and maintaines them. Piet takes care of the layout.

We need a simple system to separate the layout and content from each other. A powerful solution is: reading the content from a database, everytime the page is requested. For example, PHP and Microsoft Active Server Pages work this way. However, we only have the possibility to store plain HTML (HyperText Markup Language). Furthermore, the content does not change that often to efficiently maintain a database.

Using some simple commands, a website will be constructed.

Layout of our example
Jan has some articles stored in his homedirectory: offer.html and index.html. Both files are located in the directory /home/jan/Docs/website/.
index.html contains some news and a welcome-message. offer.html offers the what-do-I-know potatoes of the week. Piet is responsible for the layout and builds the website. In the end, the pages are published in the directory /home/piet/public_html.
A 'template'
Piet asked Jan to write his pages in plain HTML. He will take care of a suitable layout.

For example, Piet puts the header of the site in header.html and the footer of the site in footer.html. header.html might look like this:

<html><!-- the header -->
<head>
<title>Piet and Jan productions</title>
</head>
<body bgcolor="white">
<table border="0" width="100%"><tr>
<td bgcolor="#c040ff" valign="top">
This is our website<br>
Some rubbish is written down here.<br>
We are very interactive<br>
so this is our telephone number:<br>
<b>0123-456789</b>
</td><td valign="top">
<!-- Put the contents here -->

And this is footer.html:

<!-- the footer -->
</td></tr></table>
</body></html>

For example, the unix commands to build the final page from Jans' index.html are:

cat header.html  /home/jan/Docs/website/index.html
echo -n '<hr>Last modification: '
date '+%A %e %B'
cat footer.html

Please refer to the manual pages of these commands. The final file, as a result of the above commands, is piped to the standard output, which is grabbed to a file:

{
  cat header.html  /home/jan/Docs/website/index.html
  echo -n '<hr>Last modification: '
  date '+%A %e %B'
  cat footer.html
} > /home/piet/public_html/index.html

This procedure can be repeated with the other file, offer.html. In fact, we created a small script that enables the construction of our website.

However, executing this command by hand is not feasible. We can create a shell-script that is executed everytime as Jan has updated his index. However, if Piet decides to change the header or footer, this script should also be executed! On the other hand, if Jan has changed nothing on a day, the script should not be executed. We are using Linux, so we want to use a smart solution (read: automatically)!

At this point, make shows up.

First meeting with make

The info-manual of GNU make is a fantastic document. However, from the beginning it is focussing on a programming environment. For this reason, I am trying to indicate the functions of make in a broader sense:

    make determines whether a set of commands should be executed, 
    based on the time-stamp of the target-file and the time-stamps 
    of the source files.

In other words: if one of the source files, needed to create a target-file, is newer than the target-file, a set of commands will be executed. The purpose of these commands is to update the target-file.

The target-file is the 'target' and the source-files are `prerequisites' (first demands). The commands are executed if one of the `prerequisites' is newer than the target-file (or if the target does not exist). If all prerequisites are older than or equally old as the target, then the commands are not executed and the target is considered as being up-to-date.

In the current working directory, a file should be created that has the name Makefile. This file contains the information that is needed by make to do its job properly. Once we have the Makefile, the only thing that we need to do is: typing 'make' and the commands, needed for creating a new target-file, are executed automatically.

Make is called with the command

make target1 target2 ....

target is optional (if target is left away, the first target in the Makefile is being used). Make is always looking in the current directory for the Makefile. It is possible to supply more than one target.

Makefile syntaxis

The Makefile can be created with an editor and looks like:

# This is an example of a Makefile.
# Comments can be put after a hash (#).

target: prerequisites
	command
        
target: prerequisites
	commando 
        
# and so on and so on.

We start with a target, followed by a colon (:) and the needed prerequisites. In the presence of many prerequisites, it is possible to end the line with a backslash (\) and to continue on the next line.

On the following line(s), one or more commands are presented. Every line is considered as a standalone command. If you want to use multiple lines for one command, you should put backslashes (\\) on the end of the line. Make will connect the commands as if they were written on one line. In this situation, we have to separate the commands with a semicolon (;) in order to prevent mistakes by the executing shell.

Note: The commands should be indented with a TAB, not with 8 spaces!

Make reads the Makefile and determines for every target (starting with the first one) whether the commands should be executed. Every target, together with the prerequisites and rules, is denoted a 'rule'.

If make is executed without arguments, only the first target will be executed.

A Makefile for our example

For our example, the Makefile should look like this:


# This Makefile builds Piets' and Jans' website, the potato-eaters.

all: /home/piet/public_html/index.html /home/piet/public_html/offer.html

/home/piet/public_html/index.html:  header.html footer.html \
                                    /home/jan/Docs/website/index.html
	{ \
          cat header.html  /home/jan/Docs/website/index.html ;\
          echo -n '<hr>Last modification: '                 ;\
          date '+%A %e %B'                                   ;\
          cat footer.html                                    ;\
        } > /home/piet/public_html/index.html

/home/piet/public_html/offer.html:  header.html footer.html \
                                    /home/jan/Docs/website/offer.html
	{ \
          cat header.html  /home/jan/Docs/website/index.html ;\
          echo -n '<hr>Last modification: '                 ;\
          date '+%A %e %B'                                   ;\
          cat footer.html                                    ;\
        } > /home/piet/public_html/offer.html

# the end

Now, we have three targets, 'all' and the files index.html and offer.html from the website. The only function of the target 'all' is to have both others as prerequisites. These are both tested. Because 'all' itself is no filename, the target 'all' will always be executed. (Later, we will introduce a more elegant way of defining targets that are no file).

If the header and footer were modified, both pages will be updated. If Jan modifies one of his pages, only the modified page will be updated. Executing the 'make' command does the job!

Of course, the Makefile has a drawback: it is not easy to oversee. Fortunately, many ways are available of making things more simple!

Streamlining the Makefile

Variables

Thanks to variables, a Makefile can be simplified a lot. Variables are defined as follows:

variable = value

We refer to a variable with the expression $(variabele). If we incorporate this into the Makefile, it looks quite a lot better:

# This Makefile builds Piets' and Jans' website, the potato-eaters.

# Directory where the website is stored:
TARGETDIR = /home/piet/public_html

# Jans' directory:
JANSDIR = /home/jan/Docs/website

# Files needed for the layout:
LAYOUT = header.html footer.html

all: $(TARGETDIR)/index.html $(TARGETDIR)/offer.html

$(TARGETDIR)/index.html:  $(LAYOUT) $(JANSDIR)/index.html
	{ \
          cat header.html $(JANSDIR)/index.html     ;\
          echo -n '<hr>Last modification: '        ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $(TARGETDIR)/index.html

$(TARGETDIR)/offer.html:  $(LAYOUT) $(JANSDIR)/offer.html
	{ \
          cat header.html  $(JANSDIR)/index.html    ;\
          echo -n '<hr>Last modification: '        ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $(TARGETDIR)/offer.html

# the end

It is a good habit to use capital letters for the variables. Now, it is a lot easier to change, e.g., the target-directory.

If you want to, it is possible to define another method for each document that you want to put in the correct layout. What should we do if many documents must be put in the same layout? The Makefile would become very large, while many repetitions are present. This can also be simplified!

Pattern Rules

`Pattern Rules' enables us to use the same set of commands on all kinds of different targets.

If pattern rules are used, the syntaxis of a line changes; an extra pattern field is added:

Multiple targets: pattern : prerequisite prerequisite ...
	command

The pattern is an expression that should be applicable to all targets. A percent-sign is used to incorporate variable-parts of a target-name.

An example:

/home/bla/target1.html /home/bla/target2.html: /home/bla/% : %
	commands

If make reads this, the line is expanded to 2 lines. Here, the pattern determines which part of the target-name is incorporated in the percent-sign.

The percent-sign in the prerequisites-field represents the part that is copied by this percent-sign.

Make expands the above as:

/home/bla/target1.html:	target1.html
	commands
        
/home/bla/target2.html: target2.html
	commands

The percent-sign in the pattern `/home/bla/%' gets with target `/home/bla/target1.html' the value `target1.html', thus expanding the prerequisite `%' to `target1.html'.

For our website, the following rule will be incorporated:

$(TARGETDIR)/index.html $(TARGETDIR)/offer.html: \
                   $(TARGETDIR)/% : $(JANSDIR)/% \
                                         $(LAYOUT)

Now we have one problem left: how to use these variables in the commands? The commands were a little bit different for both targets?

Automatic variables

Fortunately, make defines some variables for itself. Some of these variables are called automated variables. These variables contain, during the execution of the commands (better: just prior to executing these commands), the value of the target and/or prerequisite.

The special variable $< is used to indicate the first prerequisite and the variable $@ expands always to the current target.

Using these variables, it is possible to generalise the complete rule as follows:

$(TARGETDIR)/index.html $(TARGETDIR)/offer.html: $(TARGETDIR)/% : \
                                                     $(JANSDIR)/% \
                                                          $(LAYOUT)
	{ \
          cat header.html  $<                       ;\
          echo -n '<hr>Last modification: '        ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $@

Voilà! This single line functions now for both files!

For completeness, the full Makefile is presented, including some more optimalisations:

# This Makefile builds Piets' and Jans' website, the potato-eaters.

# Directory where the website is published:
TARGETDIR = /home/piet/public_html

# Jans' directory:
JANSDIR = /home/jan/Docs/website

# Files needed for the layout:
LAYOUT = header.html footer.html

# These are the webpages:
DOCS = $(TARGETDIR)/index.html $(TARGETDIR)/offer.html


# Please change nothing below this line;-)
# -------------------------------------------------------------

all: $(DOCS)

$(DOCS): $(TARGETDIR)/% : $(JANSDIR)/% $(LAYOUT)
	{ \
          cat header.html  $<                       ;\
          echo -n '<hr>Last modification: '         ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $@

# the end

This starts to look like how it should be. If more documents are added, it is quite easy to incorporate them in the Makefile, using the DOCS variable, without too much typing.

In the end, the person that maintains the Makefile should easily see how it works, without puzzling on the way how it functions!

Last small optimalisations

We would prefer to mention the documents in DOCS, without including the whole directory. This can be done as follows (we change DOCS in the beginning of the makefile in TEXTS):

TEXTS = index.html  offer.html  yetanotherfile.html

# Please change nothing below this line;-)
# -------------------------------------------------------------
DOCS =  $(addprefix $(TARGETDIR)/,$(TEXTS))

all: $(DOCS)

# and so on

What we see here is a special make function: instead of a variable-name, it is possible to use a complete expression between brackets. This way, it is possible to modify texts in numerous ways.

The special command $(addprefix prefix,list) adds to each element on the list a prefix. In the example, this is the contents of the TARGETDIR variable plus a slash (/).

The listed items are separated with spaces. For this reason, it is not a good idea to process filenames that have spaces with the make command.

To conclude: at the beginning, we already mentioned that the target 'all' won't create a file with the name 'all' (this line does not contain any commands) and as a result, this target is always executed. But how to handle if <accidentally> a file does exist with this name, newer than the other files ...?

There is an easy way to tell make that a particular target always should be executed and that this target does not refer to a file on the hard disk. To achieve this, the target is marked as 'phony' (not real). This is done as follows:

.PHONY: all

Now, the whole Makefile looks like this:

# This Makefile builds Piets' and Jans' website, the potato-eaters.

# Directory where the website is published:
TARGETDIR = /home/piet/public_html

# Jans' directory:
JANSDIR = /home/jan/Docs/website

# Files needed for the layout:
LAYOUT = header.html footer.html

# These are the names of the webpages:
TEXTS = index.html  offer.html  yetanotherfile.html

# Please change nothing below this line;-)
# ------------------------------------------------------
DOCS =  $(addprefix $(TARGETDIR)/,$(TEXTS))
.PHONY: all

all: $(DOCS)

$(DOCS): $(TARGETDIR)/% : $(JANSDIR)/% $(LAYOUT)
	{ \
          cat header.html  $<                       ;\
          echo -n '<hr>Last modification: '         ;\
          date '+%A %e %B'                          ;\
          cat footer.html                           ;\
        } > $@

# the end

Store this file and forget it! From now on, it is possible to maintain your webpages, perhaps by using your crontab, and to properly separate the layout from the content!

Final remarks

Of course, it is possible to modify this example for other situations.

As an example, the simple way in which the document is generated is not error-free: if Jan ends his articles accidentally with </body></html>, most browsers won't display the footer that Piet has made. If we apply grep, perl or tcl, it is possible to put some titles from Jans documents in an clever way in the header of the site.

Of course, Jan can simply write flat text and use the sed command to change all white-lines (carriage returns) in <P>:

sed -e 's/^\s*$/<p>/g'

Furthermore, Jan can write his texts in LyX and use the a program, such as lyx2html, to convert it to HTML. Tremendous possibilities are available!

Another template-construction is also possible.

We haven't considered how possible pictures are transported (scaled, converted, or compressed) to the web directory. It is also possible to automate this process!

In this example, Piet should have read permission in Jans' website directory. The interesting thing of separating these tasks is that they can be applied in very large organisations. It is even possible for Piet to login at the other side of the world, or to mount his homedir over NFS. The examples can also be used for labour that is performed by one user.

Hopefully, it has become clear how the Makefile principles functions and how easy your daily work can become at the moment that you have written a good Makefile!

Tips

Make as user interface
A collection of files can also function as a source for composing an end product or act in different ways.
Using 'phony' targets (.PHONY: target), it is easy to bundle simple functions. An example is configuring the Linux kernel.
Typing make menuconfig starts the configuration with an interactive menu. Typing make xconfig starts the configuration with an Tcl/Tk interface under X.
Both above mentioned targets have nothing to do with the real construction of the kernel. They make up a simple interface to the necessary functions (such as configuring the kernel).
An example
Imagine: you have a large collection of documents that are kept, maintained, and distributed as a whole. You want to make summaries and to print in draft and zoomed-out, to print in high quality, to publish to the web, and so on. All this from within one source file (for example with LaTeX!).
You should create a Makefile with the following PHONY targets:

[help:] Print a simple overview of the functions (targets) that are incorporated in this Makefile.
[print-draft:] Print, e.g., a zoomed-out version of the document (using the ps-utils) to a printer with a low resolution.
[print-full:] Print the complete document.
[ps:] Export the complete document to a PostScript file.
[report:] Summarize title, abstract, and summary of a document and mail that to someone who supervises your work.
[html:] Update the html-version of your work automatically.
and so on, and so on. This way, it is possible to define complex actions in a Makefile. These complex actions can simply be called with an interface that is easy to maintain.
Prerequisites can also be targets.
Files that are prerequisites for a centain target, can become a target themselves in the next step.
This way, it is possible to generate HTML from a text file and to improve the layout of the newly obtained HTML file. An example:
```
TEMPLATE = layout1/Template1.txt

/home/httpd/sales/sales.html: sales.html $(TEMPLATE)
	perl Scripts/BuildPage.pl -template $(TEMPLATE) $< > $@-new
        mv -f $@-new $@

sales.html:	sales.txt
	aptconvert -toc $@ $<
```
Look for how the file would be updated if Template1.txt was changed.
Echo commands, error messages, and tests.
If a command is preceeded with a '@', it is not displayed by the make command:
```
target: prerequisite
	@cc -o target prerequisite
```
If a command is started with a '-', the make process will not terminate if this command generates an error (for example deleting a not-existing file):
```
.PHONY: clean

clean:
	-rm -r $(tempdir)
```
If you want to see what a certain make command does, e.g. make install, but you want to prevent that the command is really executed, use the -n option at the prompt:
```
wilbert@nutnix:~ > make -n install
install -m 755 program /usr/local/bin
install -m 644 program.1 /usr/local/man/man1
wilbert@nutnix:~ >
```

Prevent the substitution of make variables.

If you need the dolar sign ($) as a part of, e.g., a filename or a shell command, use it double ($$):

# A Makefile
# Don't try this at home! :-)
source = menu.txt help.txt

target: $(source)
	for i in $(source)             ;\
        do                              \
          if [ "$$i" = "menu.txt" ]    ;\
          then                          \
            doThis $$i                 ;\
          else                          \
            doThat $$i                 ;\
          fi                           ;\
        done > target

Make will, prior to sending the command to the shell for execution, replace its own variables and change the double dollar-signs in single ones.

More information

More information on how make works and all other possibilities can be found in the `GNU Make Manual'. You can read this manual on your Linux system with the following command:

info make

Of course, it is possible to read the GNU Make Manual with the GNOME and KDE help browsers or the handy tkinfo program.

Links to more information about make:

A (Dutch!) review from Armijn Hemel about the book entitled `Managing your projects with make'.
A Google search on the term tutorial'.

Have Fun!