Feb 052008
 

It took to me quite a while before I found a good, well documented, idiot-proof guide to this… that’s why I decided to save it in my own library.

All credits go to Dave Vehrs, and original article with all comments can be found here.

Also, one person nicknamed bombocat made a script to automate the whole process: Script to make the thing automagically…

This howto will explain how to install the base Debian GNU/Linux system onto a USB flash thumb drive.

So open your favorite root login shell and follow these 12 simple steps!

I. Load required kernel modules (if necessary)

Load any/all needed kernel modules (this is a partial list, actual list depends on your configuration):

root@hostname# modprobe ehci_hcd
root@hostname# modprobe ohci_hcd 
root@hostname# modprobe usbhid
root@hostname# modprobe usb_storage

II. Install required applications

Install the necessary applications on the build system:

root@hostname# apt-get install parted debootstrap

III. Identifying your media

The first thing we need to do is identify what our system identifies our media as. To do this, simply stick the usb thumbdrive into one of the usb ports and then run the tail command:

root@hostname# tail -n 14 /var/log/messages
Jan  1 12:00:00 hostname kernel: ohci_hcd 0000:00:02.1: wakeup
Jan  1 12:00:00 hostname kernel: usb 2-3: new full speed USB device using ohci_hcd 
                                 and address 2
Jan  1 12:00:00 hostname kernel: Initializing USB Mass Storage driver...
Jan  1 12:00:00 hostname kernel: scsi2 : SCSI emulation for USB Mass Storage devices
Jan  1 12:00:00 hostname kernel: usbcore: registered new driver usb-storage
Jan  1 12:00:00 hostname kernel: USB Mass Storage support registered.
Jan  1 12:00:00 hostname kernel:   Vendor:           Model: TS256MJFLASHA     Rev: 1.00
Jan  1 12:00:00 hostname kernel:   Type:   Direct-Access                      ANSI SCSI 
                                   revision: 02
Jan  1 12:00:00 hostname kernel: SCSI device sda: 506400 512-byte hdwr sectors (259 MB)
Jan  1 12:00:00 hostname kernel: sda: Write Protect is off
Jan  1 12:00:00 hostname kernel: SCSI device sda: 506400 512-byte hdwr sectors (259 MB)
Jan  1 12:00:00 hostname kernel: sda: Write Protect is off
Jan  1 12:00:00 hostname kernel:  sda: sda1 sda2
Jan  1 12:00:00 hostname kernel: sd 2:0:0:0: Attached scsi removable disk sda
root@hostname#

As we can see from this output, the device was detected and assigned to /dev/sda.

IV. Wipe the disk

The first thing we want to do is remove any old data from the drive. To do this, we’ll use the shred tool which overwrites the media with progressive cycles of random and nonrandom data to make recovery of any old data near impossible. As a final step, shred will overwrite everything with zeros.

root@hostname# shred -n 1 -z -v /dev/sda
shred: /dev/sda: pass 1/2 (random)...
shred: /dev/sda: pass 1/2 (random)...1.1MiB/248MiB 0%
shred: /dev/sda: pass 1/2 (random)...2.4MiB/248MiB 0%
shred: /dev/sda: pass 1/2 (random)...3.7MiB/248MiB 1%
<SNIP>
shred: /dev/sda: pass 2/2 (000000)...246MiB/248MiB 99%
shred: /dev/sda: pass 2/2 (000000)...248MiB/248MiB 100%
root@hostname#

For this example, shred wil run in verbose mode, with one overwrite pass of random data (-n 1) and then overwrite with zeros (-z).

V. Partition, format and mount the media

Next we need to partition the media. For a flash media installation, we will have a single partition and no swap.

For simple formating operations like this, I prefer to use GNU Parted because its a simple command.

For small media (<=512mb)
root@hostname# parted /dev/sda "mklabel msdos mkpartfs primary ext2 0.0 -0 
                                set 1 boot on"
Information: Don't forget to update /etc/fstab, if necessary.
root@hostname#

Now that we have our partition, we need to create a temporary mount point and mount our partition to it so we can perform our install.

root@hostname# mkdir /mnt/buildroot
root@hostname# mount -t ext2 /dev/sda1 /mnt/buildroot
root@hostname#
For large media (>512mb)
Some BIOS will have difficulting seeing larger partition sizes on flash media, but the kernel does not have this limitation. So we can get around the problem by using a small /boot partition.

root@hostname# parted /dev/sda "mklabel msdos mkpartfs primary ext2 0.0 21.0 
                                mkpartfs primary ext2 21.0 -0 set 1 boot on"

Then mount like so:

mkdir /mnt/buildroot
mount /dev/sda2 /mnt/buildroot
mkdir /mnt/buildroot/boot
mount /dev/sda1 /mnt/buildroot/boot

VI. Install base packages

Now that we have our partition mounted, we can install the base Debian system onto it.

root@hostname# debootstrap --arch i386 sid /mnt/buildroot
I: Retrieving Release
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Found additional base dependencies: libdb4.2 libgnutls12 libreadline5
   libsigc++-2.0-0c2a openbsd-inetd readline-common
I: Checking component main on http://ftp.debian.org/debian...
I: Retrieving adduser
<SNIP>
I: Configuring gnupg...
I: Configuring sysklogd...
I: Configuring klogd...
I: Configuring netbase...
I: Configuring openbsd-inetd...
I: Base system installed successfully.
root@hostname#

VII. Chroot Jail

root@hostname# chroot /mnt/buildroot /bin/su -
hostname:~#

VIII. System Configuration

FILE: /etc/fstab

Use vi to create the /etc/fstab file and add these contents to it:

For small media (<=512mb)
#/etc/fstab: static file system information.
#
/dev/sda1       /              ext2    defaults,errors=remount-ro,noatime 0 1
proc            /proc          proc    defaults                           0 0
tmpfs           /etc/network/run tmpfs defaults,noatime                   0 0
tmpfs           /tmp           tmpfs   defaults,noatime                   0 0
tmpfs           /var/lock      tmpfs   defaults,noatime                   0 0
tmpfs           /var/log       tmpfs   defaults,noatime                   0 0
tmpfs           /var/run       tmpfs   defaults,noatime                   0 0
tmpfs           /var/tmp       tmpfs   defaults,noatime                   0 0
For large media (>512mb)
#/etc/fstab: static file system information.
#
/dev/sda2       /              ext2    defaults,errors=remount-ro,noatime 0 1
/dev/sda1       /boot          ext2    defaults,noatime                   0 1
proc            /proc          proc    defaults                           0 0
tmpfs           /etc/network/run tmpfs defaults,noatime                   0 0
tmpfs           /tmp           tmpfs   defaults,noatime                   0 0
tmpfs           /var/lock      tmpfs   defaults,noatime                   0 0
tmpfs           /var/log       tmpfs   defaults,noatime                   0 0
tmpfs           /var/run       tmpfs   defaults,noatime                   0 0
tmpfs           /var/tmp       tmpfs   defaults,noatime                   0 0

Then mount all the filesystems:

hostname:~# mount -a
hostname:~#

Note: There maybe errors here that /dev/sda1 and /dev/sda2 can’t be found. It is safe to ignore them and continue.
Set Hostname

Set the hostname by editing /etc/hostname, and then add the base configuration to /etc/hosts:

127.0.0.1 localhost.localdoman localhost <hostname>

FILE: /etc/apt/sources.list

Next we have to add some sources to the Apt configuration.

deb http://ftp.debian.org/debian sid main non-free contrib
deb-src http://ftp.debian.org/debian sid main non-free contrib
deb http://mirrors.kernel.org/debian/ sid main non-free contrib
deb-src http://mirrors.kernel.org/debian/ sid main non-free contrib

IX. Install additional packages and kernel

Start by updating the apt databases.

root@hostname# apt-get update
Get:1 http://mirrors.kernel.org sid Release.gpg [189B]
Get:2 http://mirrors.kernel.org sid Release [38.3kB]
Get:3 http://mirrors.kernel.org sid/main Packages [4079kB]
Get:4 http://ftp.debian.org sid Release.gpg [189B]
Hit http://ftp.debian.org sid Release
Hit http://ftp.debian.org sid/main Packages
Get:5 http://ftp.debian.org sid/non-free Packages [74.6kB]
Get:6 http://ftp.debian.org sid/contrib Packages [57.1kB]
Get:7 http://ftp.debian.org sid/main Sources [1559kB]
Get:8 http://ftp.debian.org sid/non-free Sources [30.3kB]
Get:9 http://ftp.debian.org sid/contrib Sources [24.3kB]
Get:10 http://mirrors.kernel.org sid/non-free Packages [74.6kB]
Get:11 http://mirrors.kernel.org sid/contrib Packages [57.1kB]
Get:12 http://mirrors.kernel.org sid/main Sources [1559kB]
Get:13 http://mirrors.kernel.org sid/non-free Sources [30.3kB]
Get:14 http://mirrors.kernel.org sid/contrib Sources [24.3kB]
Fetched 7608kB in 48s (158kB/s)
Reading package lists... Done
root@hostname#

Install Localepurge

The first thing we’re going to install is localepurge to help keep the installation size down by removing all documentation in languages other than those you speak. When you install localepurge, it will ask you what locales you would like to keep. As an american english speaker, I select the following locales: en, en_us, and en_us.UTF8. Be careful not to remove too many locales or you may lose some functionality.

root@hostname$ apt-get install localepurge
Reading package lists... Done
Building dependency tree... Done
Suggested packages:
  debfoster deborphan
The following NEW packages will be installed
  localepurge
0 upgraded, 1 newly installed, 0 to remove and 30 not upgraded.
Need to get 35.2kB of archives.
After unpacking 87.0kB of additional disk space will be used.
Get: 1 http://ftp.debian.org sid/main localepurge 0.4.1 [35.2kB]
Fetched 35.2kB in 9s (3780B/s)
Preconfiguring packages ...
Configuring localepurge
-----------------------

localepurge will remove all locale files from your system but the ones for the
language codes you select now. Usually two character locales like "de" or "pt"
rather than "de_DE" or "pt_BR" contain the major portion of localizations. So
please select both for best support of your national language settings.  The
entries from /etc/locale.gen will be preselected if no prior configuration has
been successfully completed.

  1. aa            92. en_SG          183. ja_JP.UTF-8   274. se
  2. aa_DJ         93. en_US          184. ka            275. se_NO
  3. aa_ER         94. en_US.UTF-8    185. ka_GE         276. si
  4. aa_ER@saaho   95. en_ZA          186. kk            277. si_LK
  5. aa_ET         96. en_ZW          187. kk_KZ         278. sk
  <SNIP> 
  77. en           168. hy_AM         259. pt_BR         350. zh_CN
  78. en_AU        169. ia            260. pt_PT         351. zh_CN.GB18030
  79. en@boldquot  170. id            261. pt_PT@euro    352. zh_CN.GB2312
  80. en_BW        171. id_ID         262. rm            353. zh_CN.GBK
  81. en_CA        172. is            263. ro            354. zh_CN.UTF-8
  82. en_DK        173. is_IS         264. ro_RO         355. zh_HK
  83. en_GB        174. it            265. ru            356. zh_HK.UTF-8
  84. en_GB.UTF-8  175. it_CH         266. ru_RU         357. zh_SG
  85. en_HK        176. it_IT         267. ru_RU.KOI8-R  358. zh_TW
  86. en_IE        177. it_IT@euro    268. ru_RU.UTF-8   359. zh_TW.Big5
  87. en_IE@euro   178. iw            269. ru_UA         360. zh_TW.EUC-TW
  88. en_IN        179. iw_IL         270. rw            361. zh_TW.UTF-8
  89. en_NZ        180. ja            271. rw_RW         362. zu
  90. en_PH        181. ja_JP         272. sa            363. zu_ZA
  91. en@quot      182. ja_JP.EUC-JP  273. sa_IN

(Enter the items you want to select, separated by spaces.)

Selecting locale files 77 93 94

localepurge failed to preconfigure, with exit status 10
Selecting previously deselected package localepurge.
(Reading database ... 79926 files and directories currently installed.)
Unpacking localepurge (from .../localepurge_0.4.1_all.deb) ...
Setting up localepurge (0.4.1) ...
Configuring localepurge
-----------------------

Based on the same locale information you chose above, localepurge can also
delete superfluous localized man pages.

Also delete localized man pages? yes

If you are content with the selection of locales you chose to keep and don't
want to care about whether to delete or keep newly found locales, just deselect
this option to automatically remove new locales you probably wouldn't care about
anyway. If you select this option, you will be given the opportunity to decide
whether to keep or delete newly introduced locales.

Inform about new locales? yes
root@hostname#

After this, everytime you run apt-get to install or upgrade, it will post-install run localepurge to remove all unwanted documentation.

For now, we have to force it to run for the first time.

hostname:~# localepurge
localepurge: Disk space freed in /usr/share/locale: 25396K
hostname:~#

As space is limited, get in the habit of removing apt’s cached files frequently.

root@hostname# apt-get clean

Install Kernel

Before we can install the kernel, we need to set its configuration. Edit /etc/kernel-img.conf so that it looks like this:

do_symlinks = yes
relative_links = yes
do_bootloader = no
do_bootfloppy = no
do_initrd = yes        ## <--- Verify this line
link_in_boot = yes
postinst_hook = /sbin/update-grub
postrm_hook   = /sbin/update-grub

Next we install the kernel.

hostname:~# apt-get install linux-image-2.6.16-1-686
Reading package lists... Done
Building dependency tree... Done
Suggested packages:
  linux-doc-2.6.16 linux-source-2.6.16 grub lilo
Recommended packages:
  libc6-i686
The following NEW packages will be installed:
  linux-image-2.6.16-1-686
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
5 not fully installed or removed.
Need to get 0B/15.7MB of archives.
After unpacking 46.8MB of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 7204 files and directories currently installed.)
Unpacking linux-image-2.6.16-1-686 (from
  .../linux-image-2.6.16-1-686_2.6.16-5_i386.deb) ...
Done.
Setting up busybox (1.01-4) ...
Setting up libklibc (1.3.1-1) ...
Setting up klibc-utils (1.3.1-1) ...
Setting up udev (0.088-2) ...
A chroot environment has been detected, udev not started.

Setting up initramfs-tools (0.59b) ...
wc: /proc/swaps: No such file or directory
tail: cannot open `/proc/swaps' for reading: No such file or directory

Setting up linux-image-2.6.16-1-686 (2.6.16-5) ...

 Hmm. The package shipped with a symbolic link /lib/modules/2.6.16-1-686/source
 However, I can not read it: No such file or directory
 Therefore, I am deleting /lib/modules/2.6.16-1-686/source

Running depmod.
Finding valid ramdisk creators.
Using mkinitramfs-kpkg to build the ramdisk.

Error, do this: mount -t proc none /proc
hostname:~# apt-get clean
hostname:~#

Install Bootloader (Grub or Lilo)
Either:

install the grub binaries:

root@hostname# apt-get install grub

Or:

install the lilo binaries:

root@hostname# apt-get install lilo

X. Exit the Chroot Jail

At this time, we need to exit the chroot

hostname:~# exit
logout
root@hostname #

XI. Install bootloader

Either: GRUB

To install grub into the bootsector

root@hostname# grub-install --recheck --root-directory=/mnt/buildroot /dev/sda 
Probing devices to guess BIOS drives. This may take a long time.
Installation finished. No error reported.
This is the contents of the device map /mnt/buildroot//boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.

(fd0)   /dev/fd0
(hd0)   /dev/hda
(hd1)   /dev/sda
root@hostname#

Next we need to open /mnt/buildroot/boot/grub/menu.lst and add this configuration.:

For small media (<=512mb)
# default num
default         0

# timeout sec
timeout         5

# pretty colours
color green/black black/green

title   Debian GNU/Linux-2.6.16-1-686
root    (hd0,0)
kernel  /vmlinuz-2.6.16-1-686 root=/dev/sda1 init=/sbin/init
initrd  /initrd.img-2.6.16-1-686
savedefault
boot

title   Debian GNU/Linux-2.6.16-1-686 (Rescue/Single)
root    (hd0,0)
kernel  /vmlinuz-2.6.16-1-686 root=/dev/sda1 init=/sbin/init single
initrd  /initrd.img-2.6.16-1-686
boot
For large media (>512mb)
# default num
default         0

# timeout sec
timeout         5

# pretty colours
color green/black black/green

title   Debian GNU/Linux-2.6.16-1-686
root    (hd0,0)
kernel  /vmlinuz-2.6.16-1-686 root=/dev/sda2 init=/sbin/init
initrd  /initrd.img-2.6.16-1-686
savedefault
boot

title   Debian GNU/Linux-2.6.16-1-686 (Rescue/Single)
root    (hd0,0)
kernel  /vmlinuz-2.6.16-1-686 root=/dev/sda2 init=/sbin/init single
initrd  /initrd.img-2.6.16-1-686
boot

Then we need to run grub to link it all together

root@hostname# grub
Probing devices to guess BIOS drives. This may take a long time.

    GNU GRUB  version 0.97  (640K lower / 3072K upper memory)

       [ Minimal BASH-like line editing is supported.   For
         the   first   word,  TAB  lists  possible  command
         completions.  Anywhere else TAB lists the possible
         completions of a device/filename. ]
grub> root (hd1,0)
root (hd1,0)
 Filesystem type is ext2fs, partition type 0x83
grub> setup (hd1)
setup (hd1)
 Checking if "/boot/grub/stage1" exists... yes
 Checking if "/boot/grub/stage2" exists... yes
 Checking if "/boot/grub/e2fs_stage1_5" exists... yes
 Running "embed /boot/grub/e2fs_stage1_5 (hd1)"... 
          failed (this is not fatal)
 Running "embed /boot/grub/e2fs_stage1_5 (hd1,0)"... 
          failed (this is not fatal)
 Running "install /boot/grub/stage1 (hd1) /boot/grub/stage2 
          p /boot/grub/menu.lst "... succeeded
Done.
grub> quit

Or: LILO

Edit /mnt/buildroot/etc/lilo.conf so that it looks similar to this:

For small media (<=512mb)
boot=/dev/sda
root=/dev/sda1
compact 

bitmap=/boot/sid.bmp
bmp-colors=1,,0,2,,0
bmp-table=120p,173p,1,15,17
bmp-timer=254p,432p,1,0,0
install=bmp

default=sid

# install=menu
map=/boot/map
vga=normal
delay=20
image=/boot/vmlinuz-2.6.16-1-686
label=sid
root=/dev/sda1
read-only
initrd=/boot/initrd.img-2.6.16-1-686
For large media (>512mb)
boot=/dev/sda
root=/dev/sda2
compact 

bitmap=/boot/sid.bmp
bmp-colors=1,,0,2,,0
bmp-table=120p,173p,1,15,17
bmp-timer=254p,432p,1,0,0
install=bmp

default=sid

# install=menu
map=/boot/map
vga=normal
delay=20
image=/boot/vmlinuz-2.6.16-1-686
label=sid
root=/dev/sda2
read-only
initrd=/boot/initrd.img-2.6.16-1-686

Then load the configuration into the master boot record

root@hostname# lilo -M /dev/sda # install MBR
root@hostname# lilo -b /dev/sda # install lilo 
root@hostname#

Thanks to Rick Bronson for submitting the Lilo configuration on my original howto.

XII. Add user accounts

Either:

Copy an existing /etc/group, /etc/passwd, and /etc/shadow file over from another system (this has to be done from outside the chroot directory).

root@hostname# cp /etc/passwd /etc/group /etc/shadow /mnt/buildroot/etc/
root@hostname#

Then chroot in and create their homedirectories

root@hostname# chroot /mnt/buildroot /bin/su -
hostname:~# mkdir /home/<username>
hostname:~# chown <username>.<username> /home/<username>
<Repeat as necessary>
hostname:~# exit
root@hostname#

Or:

Set root password and add users in the chroot

root@hostname# chroot /mnt/buildroot /bin/su -
hostname:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
hostname:~#adduser test
Adding user `test'...
Adding new group `test' (1001).
Adding new user `test' (1001) with group `test'.
Creating home directory `/home/test'.
Copying files from `/etc/skel'
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for test
Enter the new value, or press ENTER for the default
        Full Name []: test
        Room Number []:
        Work Phone []:
        Home Phone []:
        Other []:
Is the information correct? [y/N] y
hostname:~#
<Repeat as necessary for more users>
hostname:~# exit
root@hostname#

And thats it. time to reboot and test.

Feb 042008
 

SMS Server Boot ScreenshotExcellent project, run by a Greek guy (very responsive in Project Forum) to build a no-graphic server quickly with live Cd.

Site is http://sms.it-ccs.com/ and server ISO (minimal, only 233 MB) is available here.

Perfect way to build up a Home Headless Server, since the bundled software includes all of the most common systems (e.g. https, ftp, even torrentflux) and it can be administered via Webmin.

List of included stuff:

– Servers:
Apache version 2.2.4
BIND version 9.4.1
ISC DHCPd version 3.0.5
Fetchmail version 6.3.8+SSL+NLS
HylaFAX Server version 5.1.6
MySQL version 5.0.37
OpenSLP version 1.2.1
OpenVPN version 2.0_rc16, OpenSSL version 0.9.7e
ProFTPd version 1.30
OpenSSH_4.6
Samba version 3.025
Sendmail version 8.14.1, config V10/Berkeley
SpamAssassin version 3.2.1
Sarg version 2.2.3.1
Squid version 2.6
VsFTPd version 2.0.5
PPPD Dialing Server version 2.4.4
CUPS Print Server version 1.2.11
UW-IMAP Server

– Clients:
Open LDAP version 2.3.32
BitTorrent PHP Client TorrentFlux version 2.3
ADSL Client RP-PPPoE version 3.8
PPP Dialing Client WvDial version 1.56

– Security:
OpenSSL version 0.9.7e
ClamAV version 0.91.1
STunnel version 4.17

– Other stuff:
Perl version 5.8.8
PHP Version 5.2.3
Python 2.5.1
Dynamic DNS Provider
Squid Analysis Report Generator
hplip (HP print/scan/fax support)
netatalk (Appletalk file and print server)
WebERP Web Based Accounting/ERP System
IlohaMail Webmail

Being based on Slackware 12, it works using modules.

Surely worth a try…