BootTorrent

The project

The thesis (available in Italian only) project developed a new procedure based on BitTorrent to improve the performance of the usual network boot based on PXE-DHCP-TFTP. The project is almost completed and at the moment, the next version will offer personalized boot parameters.

The instructions available on this page will be updated as needed in the future; for any suggestion please contact me by email (davide[dot]bruschi[at]studenti[dot]unimi[dot]it ) or compile the form at the end of this page.

The project is GPL licensed as the tools it is based on (SliTaz, BitTorrent, Aria2).

Note: for a complete description of the project, see the thesis (in italian) at the beginning of this page or read this paper.

If you find this project useful or interesting and if you decide to use it, please leave a comment at the end of this page. :)

Davide Bruschi

Short description of the new procedure:

The new boot procedure works as described below:

  • PCs boots with wake-on-lan
  • Every pc obtain his IP configuration through DHCP
  • Now every pc downloads the kernel and the image of Slitaz (both the files are available below) through TFTP (they're about 13MB so is not a big problem also if we use TFTP)
  • SliTaz boots up and automatically starts the download of the files (kernel + compressed image) of the final system, using BitTorrent
  • At the end of the download, we launch a kexec that allow us to boot the final system instead of the temporary system used for the download through BitTorrent

Il server DHCP/TFTP richiede le seguenti modifiche: The DHCP/TFTP server needs the following changes:

  • We need to put Slitaz's files in the default TFTP directory
  • We need to update the syslinux.cfg file on the TFTP server with the correct configurations
  • We need to install the following packages available through the repository: bittorrent, aria2
  • The bittorrent package is needed to activate the tracker (the available commands after the installation could be different depending on the linux distribution of the server… use dpkg -L bittorrent to view the available commands after the installation)
  • The aria2 package is the bittorrent client we use to distribute the final system to the clients
  • After the server's changes you have just to create a directory where to put the final system's files and the script to activate the tracker and the client for the seeding (aria2)

Here we provide a general version of the scripts and files for the tool use:

File syslinux.cfg (to use in the TFTP server)

default base

label base
	kernel ./bzImage
	append initrd=./rootfs4.lzma lang=it_IT kmap=it rw root=/dev/null ip=dhcp vga=normal autologin

Script for the server

#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# This script use the bittorrent and aria2 packages available on the repository +
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

#Activating tracker in background on port 8000 (obviously you can set the port that you prefer)

echo ">>> Starting Tracker http://indirizzodelserver:8000/announce"
/usr/bin/bittorrent-tracker --port 8000 --dfile dfile > log_tracker.txt &

#Creating and adding torrents to the tracker (needed every time because
#if the file changes it has a different hash in the metafile)
#(you can set ip and port as you prefer)

echo ">>> Creating and adding torrents to the tracker"

/usr/bin/maketorrent-console http://indirizzodelserver:8000/announce SILab 

#Start the client with all the torrent fiels in the current directory

echo ">>> Starting torrents condivision"


#setting seed-ratio at 999.0 so it will be never reached
#so aria2 will stay active until you will close it by hand

aria2c --enable-dht=false --disable-ipv6=true --bt-seed-unverified=true --seed-ratio=999.0 --listen-port=6882 SILab.torrent

#In the comment: the version with RPC activated
#aria2c --enable-rpc --rpc-listen-all --disable-ipv6=true --bt-seed-unverified=true --seed-ratio=999.0 *.torrent


#Kill the tracker at the end of the job (you can do it by hand)

echo ">>> killing tracker"
kill $(pidof -x bittorrent-tracker)

Script used inside Slitaz's image (btboot.sh)

#!/bin/ash

#Download through torrent (version without RPC)

aria2c --enable-dht=false --disable-ipv6=true --seed-time=0 --file-allocation=none -j5 /torrent/SILab.torrent


#Prepare ed run kexec (example with our image)

kexec -l SILab/2.6.39-gentoo-r3 --append="ramdisk_size=1228800 root=/dev/ram rw ip=dhcp udev nodevfs" --initrd=SILab/diskless.img.gz

kexec -e

SliTaz image and kernel

NOTE: the boot script inside the images below is called by “etc/init.d/local.sh” instead of being specified in “etc/rcS.conf”; anyway the result is the same

NOTE2: the images are posted in inverted order so the first one is always the last one

Version updated at 07/03/2014 (with parameters):

NEWS:

  • Also this one is based on Slitaz rolling released the 16/02/2014 link
  • In this version we installed also the bash interpreter needed for scripting (see details below), anyway the size is almost the same (before 9.8MB, after 10.0MB)
  • See below for the parameters use and personalization.

Immagine SliTaz

Kernel SliTaz (kernel v3.2.53)

Version updated at 04/03/2014 (still without parameters):

NEWS:

  • This version is based on Slitaz rolling released the 16/02/2014 link
  • In this version the discover (for DHCP configuration) is done without modification for compatibility reasons

Immagine SliTaz

Kernel SliTaz (kernel v3.2.53)

First version used in the thesis for the tests (still without parameters):

(At the moment the dhcp's discover is optimized for the SiLab lab. at the University of the Study of Milan; soon we'll release e new general version)

Immagine SliTaz

Kernel SliTaz

How to modify Slitaz image for your needs

Version with parameters

In this case is not necessary to edit the image directly. You just have to tell Slitaz where to find the script and the file .torrent needed.

To do that, you have to do the following things:

  • Create a tar.gz file containing the script btboot.sh and the .torrent file. (You can name the tar.gz file as you prefer but the script name has to be btboot.sh; you can change the .torrent file name as you like, you just have to write the correct name in the script before use it)
  • On the server you have to activate an HTTP service (if not already active); we'll use it for transfer the tar.gz to the client…
  • Now you have to put the tar.gz on your HTTP server home directory (or in a subdirectory)
  • The last thing to do is to edit the file syslinux.cfg in the TFTP server adding the parameter boottorrent in APPEND specifying the path of the tar.gz as shown below:
default base

label base
	kernel ./bzImage
	append initrd=./rootfs4.lzma lang=it_IT kmap=it rw root=/dev/null ip=dhcp vga=normal autologin boottorrent=http://indirizzo_del_server/percorso/file.tar.gz

Doing this, when Slitaz starts, as first thing, will start the download of the tar.gz with the correct script and .torrent files and then will perform the boot using bittorrent for downloading the final system. If the final system changes, you just have to update the .torrent file in the tar.gz.

NOTE1: it's important that the file is a tar.gz, otherwise, slitaz will not work properly

NOTE2: in case you want to edit the image by hand, the boottorrent parameter is read from /etc/init.d/bootopts.sh and the download is done in the same place; the decompression of the tar.gz archive and the execution of the script are done from /etc/init.d/local.sh (this script needs the bash interpreter for working properly, so we installed it and changed the script's head from #!/bin/sh to #!/bin/bash)

Version without parameters

With the non parameterized version you have to edit the already existing scripts by hand as shown below:

  • The image is a CPIO archive compressed with LZMA so, firt of all, we need to decompress it with “unlzma filename.lzma”
  • Now we need to extract the content of the CPIO in a temporary dicrectory. We just need to execute those commands as sudo (needed for the 3° command):
> mkdir tmp
> cd tmp/
> cpio -id < ../initrd
  • Now is possible to edit the script in /root/ (btboot.sh) and the .torrent file in /torrent/
  • Now we need to recreate the CPIO archive and recompress it with LZMA:
> find . | cpio --create --format=’newc’ > ../newinitrd
> cd ../
> lzma newinitrd

NOTE: in case of problems with the file's permissions we can get all the rights with the (very drastic) “sudo chmod -R 777 *”; this command doesn't create security problems because the image will be used in a controlled situation and just for the time needed for the download of the final system…

Results

The details and the results of the tests are available in the thesis at the beginning of this page; below we report some short results for give you an idea of the improvements obtained:

Video

Test configuration

124 PC; LAN 100Mbps Connection LAN ←→ Server = 4x100Mbps (LACP)

Final Operating System = about 420MB

Old procedure = PXE boot with transfer based on TFTP protocol

Performance graphs

Similar works

Discussion

bellard, 2014/06/27 15:05, 2014/06/27 15:09
Should be a feature of http://ipxe.org/
And may be inserted into http://hg.slitaz.org/slitaz-arm/file/tip/rpi/piboot#l1
You could leave a comment if you were logged in.
tesi/boottorrent_en.txt · Last modified: 2018/06/13 09:21 by atrent
CC Attribution-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0