<!--  vim: set sw=2 sts=2 et ft=docbk:

  Part of the A-A-P recipe executive: Tutorial - Pipe commands

  Copyright (C) 2002-2003 Stichting NLnet Labs
  Permission to copy and use this file is specified in the file COPYING.
  If this file is missing you can find it here: http://www.a-a-p.org/COPYING

-->

<para>
A selection of commands can be connected together with a pipe.
This means the output of one command is the input for the next command.
It is useful for filtering text from a variable or file and writing the result
in a variable or file.
</para>

<bridgehead>Changing a timestamp</bridgehead>

<para>
This example shows how you can change the timestamp in a file.
It is done in-place.
</para>

<programlisting>
    all:
        :print Setting date in foobar.txt.
        :cat foobar.txt
              | :eval re.sub('Last Change: .*\n', 'Last Change: ' + DATESTR + '\n', stdin)
              &gt;! foobar.txt
</programlisting>

<para>
Lets see how this works:
</para>

<literallayout>    % <userinput>cat foobar.txt</userinput>
    This is example text for the A-A-P tutorial.
    Last Change: 2002 Feb 29
    The useful contents would start here.
    % <userinput>aap</userinput>
    Setting date in foobar.txt.
    % <userinput>cat foobar.txt</userinput>
    This is example text for the A-A-P tutorial.
    Last Change: 2002 Oct 21
    The useful contents would start here.
    %
</literallayout>

<para>
The last command in the example consists of three parts.  First comes the
<computeroutput>:cat</computeroutput> command.  It reads the "foobar.txt" file and passes it
throught the pipe to the next command.  "cat" is short for "concatenate".
This is one of the good-old Unix commands that actually does much more than
the name suggests.  In this example nothing is concatenated.  Below you will
see examples where it does.
</para>

<para>
The second part of the example is the <computeroutput>:eval</computeroutput> command.
This is used to read the text coming in through the pipe and modify it with a
Python expression.  In this case the expression is a "re.sub()" function call.
This Python function takes three arguments: A pattern, a replacement string
and the text to operate on.  All occurences of the pattern in the text are
changed to the replacement string.  The pattern "Last Change: .*\n" matches a
line with the date that was inserted previously.  The replacement string
contains <literal>DATESTR</literal>, which is an &Aap; variable that contains today's
date as a string, e.g., "2002 Oct 19".  The text to operate on is
<literal>stdin</literal>.  This is the variable that holds the text that is coming in
through the pipe.
</para>

<para>
The third and last part <computeroutput>&gt;! foobar.txt</computeroutput> redirects the
output of the <computeroutput>:eval</computeroutput> command back to the file "foobar.txt".
Using just "&gt;" would cause an error, since the file already exists.
</para>

<para>
Note that in a Unix shell command this pipe
would not work: The "foobar.txt" would be overwritten before it was read.  In
&Aap; this does not happen, the commands in the pipe are executed one by one.
That makes it easier to use, but it does mean the text is kept in memory.
Don't use pipes for a file that is bigger than half the memory you have
available.
</para>

<para>
Changing a file in-place has the disadvantage that the normal dependencies
don't work, since there is no separate source and target file.  Often it is
better to use a file "foobar.txt.in" as source, change it like in the example
above and write it as a new file.  The recipe would be:
</para>

<programlisting>
    foobar.txt: foobar.txt.in
        :print Setting date in $target.
        :cat $source
                | :eval re.sub('Last Change: .*\n', 'Last Change: ' + DATESTR + '\n', stdin)
                &gt;! $target
</programlisting>


<bridgehead>Creating a file from pieces</bridgehead>

<para>
Sometimes you need to generate a file from several pieces.  Here is an example
that concatenates two files and puts a generated text line in between.
</para>

<programlisting>
    manual.html: body.html footer.html
        @import time
        :eval time.strftime("%A %d %B %Y", time.localtime(time.time()))
                 | :print $(lt)BR$(gt)Last updated: $stdin$BR
                 | :cat body.html - footer.html &gt;! $target
</programlisting>

<para>
There are quite a few items here that need to be explained.  First of all, the
"@import time" line.  This is a Python command to load the "time" module.  So
far we used modules that &Aap; has already loaded for you.  This one isn't,
and since we use the "time" module in the next <computeroutput>:eval</computeroutput>
command it needs to be loaded explicitly.
</para>

<para>
The Python function "strftime()" formats the date and time in a specified
format.  See the Python documentation for the details.  In this case the
resulting string looks like "Monday 21 October 2002".
</para>

<para>
The output of the <computeroutput>:eval</computeroutput> command is piped into a
<computeroutput>:print</computeroutput> command.  The variable <literal>stdin</literal> contains
the output of the previous command.  Note that "$(lt)" is used instead of
"$lt".  The meaning is exactly the same: the value of the <literal>lt</literal>
variable.  Without the extra parenthesis it would read "$ltBR", which would be
the value of the "ltBR" variable.
</para>

<para>
The resulting text is:
</para>

<literallayout>        &lt;BR&gt;Last updated: Monday 21 October 2002\n
</literallayout>

<para>
Note that the first "BR" is the HTML code for a line break, while the "$Br" at
the end is the &Aap; variable that contains a line break (here displayed as
"\n").
</para>

<para>
Finally, the <computeroutput>:cat</computeroutput> command concatenates the file
"body.html", the output of the <computeroutput>:print</computeroutput> command and the file
"footer.html".  Thus the "-" stands for where the pipe input is used.
The result is redirected to <literal>target</literal>, which is "manual.html".
</para>


<bridgehead>Pipe output in a variable</bridgehead>

<para>
The generated date in the previous example could be used elsewhere in the
recipe.  Since we don't want to repeat a complicated expression the result of
the <computeroutput>:eval</computeroutput> command should be redirected to a variable, like
this:
</para>

<programlisting>
    @import time
    :eval time.strftime("%A %d %B %Y", time.localtime(time.time()))
                | :assign Datestamp

    manual.html: body.html footer.html
           :print $(lt)BR$(gt)Last updated: $Datestamp$Br
                 | :cat body.html - footer.html &gt;! $target
</programlisting>

<para>
The <computeroutput>:assign</computeroutput> command takes the input from the pipe and puts
it in the variable mentioned as its argument, which is "Datestamp" here.
Actually, the same can be done with a normal assignment and a Python
expression in backticks, but we intentionally wanted to show using a pipe
here.
</para>


<bridgehead>Creating a file from scratch</bridgehead>

<para>
It is also possible to completely generate a file from scratch.  Here is an
example that generates a C header file:
</para>

<programlisting>
1    :include config.aap
2    pathdef.c: config.aap
3       :print Creating $target
4       :print &gt;! $target /* pathdef.c */
5       :print &gt;&gt; $target /* This file is automatically created by main.aap */
6       :print &gt;&gt; $target /* DO NOT EDIT!  Change main.aap only. */
7       :print &gt;&gt; $target $#include "vim.h"
8       :print &gt;&gt; $target char_u *default_vim_dir = (char_u *)"$VIMRCLOC";
9       :print &gt;&gt; $target char_u *all_cflags = (char_u *)"$CC -c -I$srcdir $CFLAGS";
</programlisting>

<para>
The first <computeroutput>:print</computeroutput> command displays a message, so that it's
clear "pathdef.c" is being generated.  The next line contains "&gt;!" to
overwrite an existing file.  It doesn't matter if the file already existed or
not, it now only contains the line "/* pathdef.c */".  The third and following
lines contain "&gt;&gt;".  This will cause each line to be appended to
"pathdef.c".
</para>

<para>
In the example the <literal>VIMRCLOC</literal> and <literal>srcdir</literal> variables are
defined in the recipe "config.aap".  That is why this file is used as a source
in the dependency.  Also note the use of "$#" in line 7.  Since "#" normally
starts a comment it cannot be used directly here. "$#" is a special item that
results in a "#" in the <computeroutput>:print</computeroutput> output.  This is the
resulting file:
</para>

<literallayout>       /* pathdef.c */
       /* This file is automatically created by main.aap */
       /* DO NOT EDIT!  Change main.aap only. */
       #include "vim.h"
       char_u *default_vim_dir = (char_u *)"/usr/local/share/vim61";
       char_u *all_cflags = (char_u *)"cc -c -I. -g -O2";
</literallayout>

<para>
The list of "&gt;&gt;" redirections is quite verbose.  Fortunately there is a
shorter way:
</para>

<programlisting>
1    :include config.aap
2    pathdef.c: config.aap
3        :print Creating $target
4        text &lt;&lt; EOF
5            /* pathdef.c */
6            /* This file is automatically created by main.aap */
7            /* DO NOT EDIT!  Change main.aap only. */
8            $#include "vim.h"
9            char_u *default_vim_dir = (char_u *)"$VIMRCLOC";
10           char_u *all_cflags = (char_u *)"$CC -c -I$srcdir $CFLAGS";
11               EOF
12       :print $text >! $target
</programlisting>


<para>
In line 4 "text &lt;&lt; EOF" is used.  This is called a block assignment.
The following lines, up to the matching "EOF" line, are assigned to the
variable <literal>text</literal>.  You can use something else than "EOF" if you want
to.  It must be a word that does not appear inside of the text as a line on
its own.  White space before and after the word is ignored.
</para>

<para>
The indent of the text in the block assignment is removed.  The indent of the
first line is used, the same amount of indent is removed from the following
lines.  Thus if the second line has two more spaces worth of indent than the
first line, it will have an indent of two spaces in the result.  Half a tab is
replace with four spaces when necessary (a tab always counts for up to eight
spaces).
</para>
