February 23, 2012

DEVELOPER’S GUIDE

THE DEVELOPER’S GUIDE

NOTICE

The entire SENTINIX specification is being slightly modified. sxpkg, SENTINIX’ own packaging tool, although not finished yet, will lay the foundation for the new revision. If you are interested in making a package contribution, read what’s below first, then post an e-mail to the mailing list before you begin.

CONTENTS

1.1 – THE SENTINIX SPECIFICATION

SENTINIX is a compilation of source code organized in such a way that it may be called a specification under which further development and maintenance of SENTINIX should obey. This specification is simple and logical in it’s structure and developers of SENTINIX should follow it.

The specification is described in detail in the next section (1.2) and section 1.4.

1.2 – ABOUT sentinix-fullsource-X.XX.tar.gz

sentinix-fullsource-X.XX.tar.gz contains a statically compiled development framework (gcc, glibc, etc.) that will create (compile) SENTINIX from scratch, even produce the ISO image and burn CDs. The framework is based on the LFS project although the SENTINIX specification takes inspiration from Slackware.

If you unpack sentinix-fullsource-X.XX.tar.gz you will get a directory named sentinix. Inside it there’s a script named schroot, a bash script that will enter a chroot environment from where compilation of SENTINIX will be possible. Example:

root(~) tar -xzf sentinix-fullsource-0.53.tar.gz
root(~) cd sentinix
root(~/sentinix) ls
README  boot/  etc/   isotope/  mnt/  proc/  sbin/     static/  usr/
bin/    dev/   home/  lib/      opt/  root/  schroot*  tmp/     var/
root(~/sentinix)

The isotope directory contains the source code for the ramdisk installation system and the kernel sources for SENTINIX.

The root of the SENTINIX specification is everything found under usr/src in the sentinix-fullsource package. Each software has it’s own directory, called a package in the SENTINIX specification:

root(~/sentinix/usr/src) ls
README        ed/         libpng/        nasm/             rexima/
ansirem/      eject/      libtermcap/    ncurses/          samba/
apache/       fblogo/     libtool/       nessus/           sed/
autoconf/     fetch/      lilo/          net-snmp/         sendmail/
automake/     file/       links/         netcat/           sh-utils/
bash/         fileutils/  linux/         nfs-utils/        shadow/
bc/           findutils/  linuxmandocs/  ngrep/            sharkwrapper/
bchunk/       flex/       logrotate/     nmap/             slocate/
biew/         fping/      lynx/          ntp/              snort/
bin86/        freetype/   m4/            oggvorbis/        snortcenter/
bind/         gawk/       mad/           openldap/         sox/
binutils/     gcal/       mailscanner/   openmosix-tools/  sysklogd/
bison/        gcc/        mailx/         openssh/          sysstat/
bmon/         gd/         make/          openssl/          sysvinit/
bsd_db/       gdbm/       man/           parted/           syswideconfig/
build-all*    gettext/    man-pages/     patch/            tar/
bzip2/        glib/       mc/            perl/             tcpdump/
cacti/        glibc/      mhash/         pine/             tcpip/
calcpercent/  gmp/        minicom/       popt/             tcptrace/
cdp/          gnupg/      mm/            portmap/          texinfo/
cdparanoia/   gpm/        modutils/      postfix/          textutils/
cdrtools/     grep/       mop00/         procinfo/         traceroute/
clean-all*    groff/      mop01/         procmail/         usleep/
cron/         gzip/       mop02/         procps/           util-linux/
curl/         hdparm/     mop03/         psmisc/           vim/
cyrus-sasl/   infozip/    mop04/         pure-ftpd/        wget/
debianutils/  ipaudit/    mpg321/        python/           which/
dhcp/         iptables/   msyslog/       qstat/            whois/
dhcpcd/       kbd/        mysql/         radiusclient/     zlib/
dialog/       less/       nagios/        raidtools/        zoneinfo/
diffutils/    lftp/       nail/          readline/
e2fsprogs/    libjpeg/    nano/          replimenu/
root(~/sentinix/usr/src)

The build-all script enters all directories (in the correct order, which is listed inside the build-all script) and executes a build script. The clean-all script enters all directories (in the correct order, which is listed inside the clean-all script) and executes a clean script. Let’s look inside the postfix directory:

root(~/sentinix/usr/src) cd postfix
root(~/sentinix/usr/src/postfix) ls -l
total 1352
-rwxr-xr-x    1 root     root          443 Oct 23 11:50 build*
-rwxr-xr-x    1 root     root           48 Aug  9 01:45 clean*
-rw-r--r--    1 root     root      1350854 Jul 28 10:03 postfix-2.0.13.tar.gz
-rw-r--r--    1 root     root          152 Jul 28 11:42 postfix-2.0.13.tar.gz.sig
-rw-r--r--    1 root     root          531 Jun 29 12:24 postfix-install.patch
-rw-r--r--    1 root     root           15 Aug  7 18:57 sxconfig
-rw-r--r--    1 root     root         5258 Jul 28 10:03 wietse.pgp
root(~/sentinix/usr/src/postfix)

Each source package directory has a build script, a clean script, an sxconfig file and the tarball or tarballs for the software. build and clean should be chmodded 0755, sxconfig and the rest should be chmodded 0644.

Most software get away with having only the two scripts, an sxconfig file and the tarball, but that isn’t the case with Postfix. Postfix’ make install forces user interaction during compilation, something we don’t want! Interaction (forced user input) should be avoided in source packages! Only in very special cases should interaction be allowed, but never when compiling SENTINIX from scratch (using the sentinix-fullsource package). postfix-install.patch is a diff-patch that removes user interaction from the Postfix installation process, it is applied from within the build script (see below).

The build script:
#!/bin/sh
CWD=`pwd`

. sxconfig &&
./clean &&
tar -xzf postfix-$VERSION.tar.gz &&
cd postfix-$VERSION &&
chown -R root.root . &&
cat $CWD/postfix-install.patch | patch -N &&

make makefiles CCARGS="-DFD_SETSIZE=2048 -DUSE_SASL_AUTH -I/usr/include/sasl" AUXLIBS="-lsasl2" &&
make &&
strip bin/* libexec/* &&
make install &&
echo "pwcheck_method: pwcheck" > /usr/lib/sasl2/smtpd.conf &&
ln -sf ../sbin/sendmail /usr/lib/sendmail

First, the CWD variable is defined, then the sxconfig file is included (see it’s contents below). The commands in the script are AND‘ed together (the &&) in order for the script to exit with an errorlevel not equal to 0 if any command fails. If a command exits with an errorlevel greater than 0 the AND chain is broken and the script will continue after the chain. In the case with the build script, there should only be one chain (in most cases anyway), this way we can easily find out if any one single command in the script has failed.

The build script untars the Postfix tarball and applies the diff-patch that will remove forced user interaction. It compiles the software, installs it and does some post-installation.

The clean script:
#!/bin/sh
. sxconfig &&
rm -rf postfix-$VERSION
The sxconfig file:
VERSION=2.0.13

1.3 – COMPILING SENTINIX FROM SCRATCH

The following instructions may be found in the README files in the sentinix-fullsource package aswell.

  1. Become root (very important!).
  2. Unpack sentinix-fullsource-X.XX.tar.gz on a partition with at least 4 GB of free space.
  3. To enter the automatic build system you must chroot into it, do this by typing…

    $ cd sentinix/
    $ ./schroot

    …as the root user on your regular Linux system.

  4. You should now be in the chroot environment ready to build SENTINIX and you should see this prompt:

    I have no name!:/#

    "I have no name" means that bash was unable to resolve the username for user id 0 since the glibc passwd stuff isn’t in the build system. Once glibc is installed and bash is reloaded, it will say root instead.

  5. Go to /usr/src and run the build-all script:

    I have no name!:/# cd /usr/src/
    I have no name!:/usr/src# ./build-all

    Compilation now begins. It takes approximately 2 hours and 40 mintues on an AMD 1800+ (1.5GHz) with an ATA-100 hard disk to compile SENTINIX. The build system is not designed for parallel compilation (no make -j anywhere). glibc won’t compile correctly with make -j.

    Please note! glibc is compiled twice, once in the beginning and once at the end. The second compilation is done with the target system’s own compiler, not the static compiler that comes with the build system. This is considered cleaner and more stable.

  6. When compilation is done you won’t be able to run ls or most other basic programs (unless you enter the full path to it) since the /static dir has been renamed to /staticdumphole. Exit the chroot environment (e.g., Ctrl+D). Before you enter again, edit the schroot script and add “ --login” after “/bin/bash“, e.g.:

    chroot $CWD /bin/bash to chroot $CWD/bin/bash --login

  7. Type ./schroot again and you’re inside the full SENTINIX distro. You may delete /staticdumphole if you wish.
  8. Enter /isotope and edit the config file, e.g.:

    $ cd /isotope
    $ nano config

  9. Change the following variables to suit your system:
    # Below, the speed of your CD recorder (speed to record in):
    CDRSPEED=16
    # Speed to record CDRW discs:
    CDRWSPEED=4
    # SCSI (or scsi-generic) device to use:
    CDRDEV=0,0,0
    

    Leave all other variables untouched.

  10. Run build.sources, i.e.:

    root@sandbox:/isotope# ./build.sources

  11. When finished OK, run build.iso, i.e.:

    root@sandbox:/isotope# ./build.iso

  12. If all is well, insert a blank CD-R or CD-RW and run mkrealcd or mkcdrw, i.e.:

    To create a CD-R, type…

    root@sandbox:/isotope# ./mkrealcd

    To create a CD-RW, type…

    root@sandbox:/isotope# ./mkcdrw

  13. If the recording process went fine, try the CD out by booting from it. Avoid to run the installation process, as you might overwrite your regular Linux system during partitioning. A separate development box for SENTINIX is recommended.

1.4 – HOW TO ADD NEW SOFTWARE PACKAGES TO SENTINIX

If you’ve installed SENTINIX from the ISO there is no source tree as described in section 1.2, there’s just an empty /usr/src directory. SENTINIX does not have separate source/binary packages, only source packages. Users (non-developers) are advised to run a script (insrc) on new packages or updates. The script works like Slackware’s installpkg except that it extracts a source package, compiles and installs the software automatically, without (in most cases) user interaction.

Read section 1.2 above if you haven’t already, then come back here and read the specifics on how to create a new software package.

STEP BY STEP

  1. There are many things to keep in mind while creating the package, these are:
    • The package must work in sentinix-fullsource from-scratch-compilation.
    • The package must work as an update to a previous version of the package.
    • Forced user interaction (halted compilation) is only to be allowed during an update phase, never in sentinix-fullsource (never during the automatic compilation).
  2. Create a directory under /usr/src to hold the software, e.g. ngrep. It should be chmodded 0755 and chowned to root.root.
  3. Put the software tarball(s) and other things needed (diff patches, etc.) into that directory, e.g.:
    root@sandbox:/usr/src/ngrep# ls -l
    total 508
    -rw-r--r--    1 root     root       520209 Oct 30 21:39 ngrep-1.41.tar.bz2
    root@sandbox:/usr/src/ngrep#
    

    The tarball(s) and any diff patches should be chmodded to 0644 and chowned to root.root.

  4. Create a sxconfig file, e.g.:

    root@sandbox:/usr/src/ngrep# nano sxconfig

    The sxconfig file should only contain version variables, for example (in the case of ngrep):

    VERSION=1.41

    The $VERSION variable is then used in the build script. The sxconfig file is used to simplify future maintenance of new versions of the package. The file should be chmodded 0644 and chowned root.root.

  5. Create a build script, e.g.:

    root@sandbox:/usr/src/ngrep# nano build

    This is an example of it’s content:

    #!/bin/sh
    
    . sxconfig &&
    ./clean &&
    /bin/tar -xjvf ngrep-$VERSION.tar.bz2 &&
    cd ngrep-$VERSION &&
    /bin/chown -R root.root . &&
    ./configure --prefix=/usr --build=i386-sentinix-linux &&
    make &&
    make install
    

    The build script should preferrably be small and easy to read. It must exit with an exit code above 0 if an error occured and equal to 0 if no error occured, which is easily done by ANDing the commands together (&& at the end of each command).

  6. Create a clean script. The clean script is used to clean-up the package’s directory from any previously unpacked tarball(s), it can (in a way) be compared to make distclean. E.g.:

    root@sandbox:/usr/src/ngrep# nano clean

    In the case of ngrep, this is the clean script’s contents:

    #!/bin/sh
    . sxconfig &&
    /bin/rm -rf ngrep-$VERSION
    

    Both the build script and the clean script must be chmodded +x (chmod +x build clean) and should be chowned root.root.

  7. You may now create a CONTRIBUTOR file containing a couple of lines stating who made the package, for example, the ngrep CONTRIBUTOR file:
    root@sandbox:/usr/src/ngrep# cat CONTRIBUTOR
    root@sandbox:/usr/src/ngrep#
    

    You may include e-mail address, workplace, homepage, smilies, signatures, etc.

  8. Change directory to .. and compress the package, i.e.:
    root@sandbox:/usr/src/ngrep# cd ..
    root@sandbox:/usr/src# tar -czf ngrep-1.41-1-sx.tar.gz ngrep
    

    In most cases the package’s name should start with what the main tarball starts with, which is ngrep-1.41 in this example. To that, “-1” should be added if it’s the first release of this package, etc. Then, finally, “-sx.tar.gz” should be added. Currently, we’ll only use gzipped tarballs, not bzipped (this may change in the future but for now we want consistency).

  9. E-mail Michel Blomgren the package (or provide a link in the e-mail).

May the source be with you!