Difference between revisions of "Installation/Live Images/USB-Stick for OFW11"

From OpenFOAMWiki
(the mount should now point to OFW11, instead of being random)
(Cheat sheet for copying installation to USB-Stick: Added section for the automount)
Line 139: Line 139:
 
rsync -a site lubuntu-3.2 /mnt/casper/upper/home/lubuntu/foam/
 
rsync -a site lubuntu-3.2 /mnt/casper/upper/home/lubuntu/foam/
 
</bash>
 
</bash>
 +
 +
 +
=== Making the compressed filesystems mounted at start ===
 +
 +
Edit the {{tt|rc.local}} file:
 +
<bash>sudo nano /mnt/casper/upper/etc/rc.local</bash>
 +
 +
Place the following inside the file:
 +
<bash>#!/bin/sh -e
 +
#
 +
# rc.local
 +
#
 +
# This script is executed at the end of each multiuser runlevel.
 +
# Make sure that the script will "exit 0" on success or any other
 +
# value on error.
 +
#
 +
# In order to enable or disable this script just change the execution
 +
# bits.
 +
#
 +
# By default this script does nothing.
 +
 +
mount /cdrom/OpenFOAM-3.0.x.sqfs /home/lubuntu/OpenFOAM/OpenFOAM-3.0.x -t squashfs -o loop,ro
 +
mount /cdrom/ThirdParty-3.0.x.sqfs /home/lubuntu/OpenFOAM/ThirdParty-3.0.x -t squashfs -o loop,ro
 +
mount /cdrom/foam-extend-3.2.x.sqfs /home/lubuntu/OpenFOAM/foam-extend-3.2 -t squashfs -o loop,ro
 +
 +
exit 0
 +
</bash>
 +
 +
Save and close, by using the key combination {{tt|Ctrl+X}} and then hit the {{tt|Y}} key separately.
 +
 +
Finally, make the script executable:
 +
<bash>sudo chmod +x /mnt/casper/upper/etc/rc.local</bash>
  
 
Now you can continue where you left off at section [[#Transferring the local installation to the persistent partition file in the drive|Transferring the local installation to the persistent partition file in the drive]].
 
Now you can continue where you left off at section [[#Transferring the local installation to the persistent partition file in the drive|Transferring the local installation to the persistent partition file in the drive]].

Revision as of 11:07, 31 January 2016

WARNING: These instructions are currently being written and under testing!

1 Creating the development machine

1.1 Installation details

In a nutshell:

  1. Install Lubuntu 15.10 x86_64 in a virtual machine.
  2. Install OpenFOAM 3.0.x, as instructed here: Installation/Linux/OpenFOAM-3.0.x/Ubuntu#Ubuntu_15.10
  3. Install foam-extend 3.2.x, based on the instructions from here: Installation/Linux/foam-extend-3.2/Ubuntu#Ubuntu_15.10
  4. Install swak4Foam, PyFoam and cfMesh for OpenFOAM 3.0.x, since these are already in foam-extend 3.2.x.


1.1.1 Install Lubuntu 15.10 x86_64 in a virtual machine

TODO


1.1.2 Packages that need to be installed in the development VM

TODO

Part of the software stack:

 
sudo apt-get install squashfs-tools

1.1.3 Install OpenFOAM 3.0.x

TODO


1.1.3.1 Install additional contributions for OpenFOAM 3.0.x

TODO swak4Foam, PyFoam and cfMesh


1.1.4 Install foam-extend 3.2.x

TODO


1.2 Preparations before copying to final image

This is a cheat sheet on the commands needed for cleaning up each installation before copying to the USB-Stick:

1.2.1 Clean up OpenFOAM 3.0.x et al

of30x
foam3rdParty
rm -r build
rm log.*
rm *.log
cd ..
wclean all swak4Foam
wclean all cfMesh
rm -r PyFoam/build/


1.2.2 Clean up foam-extend 3.2.x et al

fe32
rm log.make


1.3 Cheat sheet for copying installation to USB-Stick

Notes:

First of all, create our sandbox folder:

mkdir ~/sandbox

1.3.1 Copying OpenFOAM 3.0.x et al

In a new terminal, run:

of30x
cd ~/OpenFOAM
TARGET=~/sandbox/OpenFOAM
mkdir $TARGET/
rsync -a swak4Foam cfMesh PyFoam lubuntu-3.0.x $TARGET/
 
rsync -a OpenFOAM-3.0.x/  $TARGET/OpenFOAM-3.0.x/
rsync -a ThirdParty-3.0.x/ $TARGET/ThirdParty-3.0.x/
 
cd $TARGET/OpenFOAM-3.0.x/
rm -r platforms/linux64GccDPInt32OptSYSTEMOPENMPI/
rm -r platforms/linux64GccDPInt32Opt/{src,applications}
 
cd $TARGET/ThirdParty-3.0.x/
rm -r CGAL-4.7/ ParaView-4.4.0/ scotch_6.0.3/


1.3.2 Copying foam-extend 3.2.x et al

In a new terminal, run:

fe32
cd ~/foam
TARGET=~/sandbox/foam
mkdir $TARGET/
rsync -a site $TARGET/
 
mkdir $TARGET/foam-extend-3.2
cd foam-extend-3.2/
rsync -a ./ $TARGET/foam-extend-3.2/
 
cd $TARGET/foam-extend-3.2/
find applications src -name "$WM_OPTIONS" | xargs rm -r
rm -r ThirdParty/rpmBuild/{BUILD*,TGZS,RPMS,rpmDB}/*
find applications src -name "*.dep" | xargs rm


1.3.3 Packing the sandbox to a compressed image

Given that the final installation is over 2 GiB, therefore we need to save up space by relying on squashfs to get the whole thing trimmed down to something more manageable. To do so, run:

mksquashfs ~/sandbox/OpenFOAM/OpenFOAM-3.0.x ~/sandbox/OpenFOAM-3.0.x.sqfs
mksquashfs ~/sandbox/OpenFOAM/ThirdParty-3.0.x ~/sandbox/ThirdParty-3.0.x.sqfs
mksquashfs ~/sandbox/foam/foam-extend-3.2 ~/sandbox/foam-extend-3.2.x.sqfs

Check the final size of the files, just in case:

ls -l -h ~/sandbox/*.sqfs


1.3.4 Copying the compressed images to the USB-stick

It's simply this:

rsync -a -v --progress ~/sandbox/*.sqfs /media/$USER/OFW11/


1.3.5 Making sure the compressed images are properly used

In a new terminal, run:

cd /mnt/casper/upper/home/lubuntu/

Create the place-holder folders and symbolic links which will link into the original folders:

mkdir -p OpenFOAM/OpenFOAM-3.0.x
mkdir -p OpenFOAM/ThirdParty-3.0.x
mkdir -p foam/foam-extend-3.2


1.3.6 Copying the folders that were not compressed

Simply run:

cd ~/sandbox/OpenFOAM/
rsync -a swak4Foam cfMesh PyFoam lubuntu-3.0.x /mnt/casper/upper/home/lubuntu/OpenFOAM/
 
cd ~/sandbox/foam/
rsync -a site lubuntu-3.2 /mnt/casper/upper/home/lubuntu/foam/


1.3.7 Making the compressed filesystems mounted at start

Edit the rc.local file:

sudo nano /mnt/casper/upper/etc/rc.local

Place the following inside the file:

#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
 
mount /cdrom/OpenFOAM-3.0.x.sqfs /home/lubuntu/OpenFOAM/OpenFOAM-3.0.x -t squashfs -o loop,ro
mount /cdrom/ThirdParty-3.0.x.sqfs /home/lubuntu/OpenFOAM/ThirdParty-3.0.x -t squashfs -o loop,ro
mount /cdrom/foam-extend-3.2.x.sqfs /home/lubuntu/OpenFOAM/foam-extend-3.2 -t squashfs -o loop,ro
 
exit 0

Save and close, by using the key combination Ctrl+X and then hit the Y key separately.

Finally, make the script executable:

sudo chmod +x /mnt/casper/upper/etc/rc.local

Now you can continue where you left off at section Transferring the local installation to the persistent partition file in the drive.

2 Creating the USB-Stick

2.1 Reminders

  • sdX refers to the device itself for the USB drive.
  • sdX1 refers to the first partition.
  • You can find the actual device identification if you use the auto-mount feature that Linux Distributions provide to explore the drive's contents. Then run:
    mount

    and the last entry is likely the one related to the USB drive, assuming it's the latest one mounted in the system. For example, it may show this:

    /dev/sdf1 on /media/ofuser/F361-E869 type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)

    which means that:

    • /dev/sdf is the drive's device identification path;
    • /dev/sdf1 is the drive's first partition path.


2.2 Commands for formatting USB-Stick drive

These steps are dedicated for creating a stick drive with a partition size that should fit all 8GB USB drives, including those that lie about their total size, such as those that really only have around 7,750,000,000 bytes. The following instructions aim for a primary partition of 7,654,604,800 bytes.

Steps:

  1. Switch to root or keep in mind to do the command with sudo.
  2. First of all, unmount the partition that it probably auto-mounted:
    umount /dev/sdX1
  3. For creating the partition:
    fdisk /dev/sdX
    d  #delete
    o  #create DOC partition table
    c  #toggle DOS compatibility
     
    #should show:
    #  DOS Compatibility flag is set (DEPRECATED!)
     
    n  #new partition
    p  #new partition
    1  #partition 1
    62    #first sector
    +7300M  #end sector, as size in MB
     
    t  #change type
    c  #change to W95 FAT32 (LBA)
     
    a  #make it bootable
     
    p  #prints current table on screen make a note of the shown values, because you're going to need the End sector number
     
    w  #write partition table
    Note
    Make sure you have the units in sectors:
    u  # use as many times as possible until it shows
     
    #Changing display/entry units to sectors
  4. Optional - Full clean up of partition:
    dd if=/dev/zero of=/dev/sdX1 bs=512
    1. In another terminal (also as root), tun:
      while killall -USR1 dd; do sleep 30; done
    2. Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
    3. It will probably end with the message:
      dd: error writing '/dev/sdX1': No space left on device

      because we didn't define the total number of bytes/blocks to write.

    4. Once completed, make sure it's fully written by running:
      sync

      wait until it returns control back to you.

  5. Once complete, then for formatting the partition properly:
    mkfs -t vfat -n OFW11 /dev/sdX1

2.3 Creating the live USB

Tested with Lubuntu 15.10 x86_64.

Need to have usb-creator-gtk installed and we will be using Lubuntu 64-bit as the basis for our stick drives. In addition, it's advisable to do these steps from a working installation of the desired Lubuntu version, even if it's within a virtual machine.

Steps:

  1. As root (or with sudo), run:
    usb-creator-gtk
  2. Select the ISO for the Lubuntu Live CD/DVD.
  3. Select the stick drive you want as the target for the Live USB.
    • Note: Do not use the erase function, otherwise you will loose the correct booting mechanism that was created with fdisk and mkfs.
  4. Select the largest possible "Stored in reserved extra space", at least taking into account how much will be left over for the read-anywhere files, such as presentations and VM installers.
    • For example:
      • 690 MiB for the cloned Live CD
      • 1.0 GiB for the read-anywhere data.
      • 7.1 GiB - 1.0 GiB - 690 MiB = 5.42 GiB = 5556.4 MiB... which is larger than 4GiB, the file size limit for FAT32 partitions.
  5. Finally, click on the "Make Startup Disk" button and wait a while for it to run...


2.3.1 First boot up of the new USB-Stick

After booting up the USB-Stick in a real machine or virtual machine, install the necessary packages, with the following commands:

sudo apt-get update
sudo apt-get install virtualbox-guest-utils virtualbox-guest-dkms

which will install the necessary guest additions for Virtualbox to be usable in case the USB-stick doesn't boot directly in the real machine.

Then continue to install the necessary packages for OpenFOAM 3.0.x and foam-extend 3.2.x:

sudo apt-get install git-core build-essential cmake flex bison zlib1g-dev qt4-dev-tools libqt4-dev libqtwebkit-dev gnuplot gnuplot-x11 \
libreadline-dev libncurses-dev libxt-dev libopenmpi-dev openmpi-bin libboost-system-dev libboost-thread-dev libgmp-dev \
libmpfr-dev python python-dev binutils-dev libiberty-dev rpm mercurial graphviz

Then change the desktop image, by right clicking on the desktop and choosing Desktop Preferences. Make sure you download the image from the Workshop website and place it in the folder /home/lubuntu/Images.


2.3.2 Transferring the local installation to the persistent partition file in the drive

Steps:

  1. Mount the stick with the interactive mount that Lubuntu has got from the file manager.
  2. Copy the image file to the home folder. This file will contain the persistent installation on the USB-stick, which is why it's best to first copy to your system, so that's quicker in storing several thousands of files inside it:
    rsync -a -v --progress /media/$USER/OFW11/casper-rw ~/casper-rw
  3. As root (or with sudo), run:
    mkdir /mnt/casper
    mount -t auto ~/casper-rw /mnt/casper/ -o loop
  4. The persistent home folder will now be located at /mnt/casper/upper/home/lubuntu/:
    cd /mnt/casper/upper/home/lubuntu/

    If it doesn't exist, then create it and change ownership:

    mkdir -p /mnt/casper/upper/home/lubuntu/
    chown -R $USER:$USER /mnt/casper/upper/home/lubuntu/
  5. Copy the files you have on the sandbox folder into the designated folder, as explained in section Cheat sheet for copying installation to USB-Stick.
  6. When you're done copying files to the image, make sure the persistent home folder has the correct owner permissions:
    chown -R 999:999 /mnt/casper/upper/home/lubuntu/
  7. Then unmount it:
    umount /mnt/casper
  8. Then copy the casper-rw file back to the USB-stick:
    rsync -a -v --progress ~/casper-rw /media/$USER/OFW11/
  9. Finally, unmount the stick:
    umount /dev/sdX1

2.3.3 Creating the bootable ISO for the virtual machine

References:

  1. Install the package genisoimage in your Lubuntu development system:
    sudo apt-get install genisoimage
  2. Mount the USB-stick.
  3. Create a folder where you will place all files that will go into the ISO file. For example:
    mkdir ISOroot
  4. Copy the folders syslinux and install from the USB-stick, into the folder ISOroot, so that you get this folder structure:
    ISOroot
    |-- install
    |-- syslinux
  5. Copy the boot files into the folder syslinux, for example:
    cp /media/$USER/OFW11/casper/{vmlinuz.efi,initrd.lz} ISOroot/syslinux/
  6. Edit the file ISOroot/syslinux/txt.cfg and change it to this:
    default live
    label live
    menu label ^Try Lubuntu without installing
    kernel /syslinux/vmlinuz.efi
    append noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/lubuntu.seed boot=casper initrd=/syslinux/initrd.lz quiet splash ---
    label live-install
    menu label ^Install Lubuntu
    kernel /syslinux/vmlinuz.efi
    append noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/lubuntu.seed boot=casper only-ubiquity initrd=/syslinux/initrd.lz quiet splash ---
    label check
    menu label ^Check disc for defects
    kernel /syslinux/vmlinuz.efi
    append noprompt boot=casper integrity-check initrd=/syslinux/initrd.lz quiet splash ---
    label memtest
    menu label Test ^memory
    kernel /install/mt86plus
    label hd
    menu label ^Boot from first hard disk
    localboot 0x80

    Essentially you will be changing the paths that state /casper/ to /syslinux/.

  7. Create the ISO file by running the following command from the parent folder where ISOroot is located:
    mkisofs -o VMbootUSB.iso    -b syslinux/isolinux.bin -c syslinux/boot.cat    -no-emul-boot -boot-load-size 4 -boot-info-table    ISOroot
  8. The final image file VMbootUSB.iso is what we will need to boot the USB-stick inside the virtual machine.


2.3.4 Creating the virtual machine in Virtualbox

Keep in mind that these steps are only needed for creating the virtual machine to be placed inside the USB-stick. Nonetheless, it might be useful in case the exported OVA file is incompatible.

References:

Side notes:

  • On Ubuntu, had to install the necessary packages for getting the USB2/3 drivers for Virtualbox on the host side (the real machine):
    sudo apt-get install virtualbox-ext-pack

    And then run the following command for adding my user to the group vboxusers:

    sudo usermod -aG vboxusers $USER

    Then I had to log out and then log back in, otherwise it's not possible to use a real USB drive in a virtual machine within Virtualbox.

Steps:

  1. Simply create a new Virtual Machine in Virtualbox, meant for a Ubuntu 64-bit machine and without any disks, because we will boot straight from the USB drive.
  2. After the virtual machine is created, edit its settings and:
    1. increase the RAM to at least 2048MB;
    2. change from 1 to 2 virtual cores;
    3. choose for the optical drive to use the VMbootUSB.iso;
    4. probably best to turn off audio capabilities;
    5. in the USB section, choose USB 3.0 or 2.0 controller, depending on what kind of USB-stick you're using;
    6. still in the USB section, click on the button that adds a new USB device and choose your USB-stick.
  3. Boot up the VM and test it.
  4. If it works fine, do a standard shutdown within the virtual machine.
  5. Then use on the main menu for the Virtualbox the item File -> Export Appliance... to export the virtual machine to a settings file to be placed inside the USB-stick, preferably with the file name OFW11USBVM.ova.
  6. Best to place both the OFW11USBVM.ova and VMbootUSB.iso files inside the folder VM in the real USB-stick partition (i.e. not inside the booted up Live Lubuntu).


2.4 Cloning the original stick to a new image

Steps:

  1. Make sure you don't have the stick mounted! Example to umount:
    umount /dev/sdX1
  2. As root (or with sudo), run:
    dd if=/dev/sdX of=OFW11-2016_v2.img bs=512 count=14950463
    • The value "14950463" is the End sector count 14950462 plus 1.
    • 14950463 sector * 512 B/sector = 7654636544 B = 7654636544 ~= 7300 MiB
    1. In another terminal (also as root), tun:
    while killall -USR1 dd; do sleep 30; done
    1. Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
  3. When it's done cloning, make sure the file system is synced:
    sync

    wait until it returns control back to you.

  4. Now you can mount the stick again, if you want. Or simply eject it.


2.5 Cloning the image to a new stick

It's almost the reverse of the previous steps:

  1. Make sure you don't have the stick mounted! Example to umount:
    umount /dev/sdX1
  2. As root (or with sudo), run:
    dd if=OFW11-2016_v2.img of=/dev/sdX bs=512
    1. In another terminal (also as root):
    ps -a
    1. In another terminal (also as root), tun:
    while killall -USR1 dd; do sleep 30; done
    1. Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
  3. When it's done cloning, make sure the file system is synced:
    sync

    wait until it returns control back to you.

  4. Now you can mount the stick, if you want. Or simply eject it.