Installation/Live Images/USB-Stick for OFW11

From OpenFOAMWiki
< Installation‎ | Live Images
Revision as of 20:01, 30 January 2016 by Wyldckat (Talk | contribs)

WARNING: These instructions are currently being written and under testing!

1 Creating the development machine

In a nutshell:

  1. Install Lubuntu 15.10 x86_64 in a virtual machine.
  2. Install OpenFOAM 3.0.x, as instructed here: Installation/Linux/OpenFOAM-3.0.x/Ubuntu#Ubuntu_15.10
  3. Install foam-extend 3.2.x, based on the instructions from here: Installation/Linux/foam-extend-3.2/Ubuntu#Ubuntu_15.10
  4. Install swak4Foam, PyFoam and cfMesh for OpenFOAM 3.0.x, since these are already in foam-extend 3.2.x.

1.1 Installation details

TODO


1.2 Preparations before copying to final image

This is a cheat sheet on the commands needed for cleaning up each installation before copying to the USB-Stick:

1.2.1 Clean up OpenFOAM 3.0.x et al

of30x
foam3rdParty
rm -r build
rm log.*
rm *.log
cd ..
wclean all swak4Foam
wclean all cfMesh
rm -r PyFoam/build/


1.2.2 Clean up foam-extend 3.2.x et al

fe32
rm log.make


1.3 Cheat sheet for copying installation to USB-Stick

Notes:

TODO This is to be done after the sandbox is full. --- First we need to change ownership of folders, to make it easier to copy things to the USB-stick:

sudo chown -R $USER:$USER /mnt/casper/upper/home/lubuntu/

Create our sandbox folder:

mkdir ~/sandbox


1.3.1 Copying OpenFOAM 3.0.x et al

In a new terminal, run:

of30x
cd ~/OpenFOAM
TARGET=~/sandbox/OpenFOAM
mkdir $TARGET/
rsync -a swak4Foam cfMesh PyFoam lubuntu-3.0.x $TARGET/
 
rsync -a OpenFOAM-3.0.x/  $TARGET/OpenFOAM-3.0.x/
rsync -a ThirdParty-3.0.x/ $TARGET/ThirdParty-3.0.x/
 
cd $TARGET/OpenFOAM-3.0.x/
rm -r platforms/linux64GccDPInt32OptSYSTEMOPENMPI/
rm -r platforms/linux64GccDPInt32Opt/{src,applications}
 
cd $TARGET/ThirdParty-3.0.x/
rm -r CGAL-4.7/ ParaView-4.4.0/ scotch_6.0.3/


1.3.2 Copying foam-extend 3.2.x et al

In a new terminal, run:

fe32
cd ~/foam
TARGET=~/sandbox/foam
mkdir $TARGET/
rsync -a site $TARGET/
 
mkdir $TARGET/foam-extend-3.2
cd foam-extend-3.2/
rsync -a ./ $TARGET/foam-extend-3.2/
 
cd $TARGET/foam-extend-3.2/
find applications src -name "$WM_OPTIONS" | xargs rm -r
rm -r ThirdParty/rpmBuild/{BUILD*,TGZS,RPMS,rpmDB}/*
find applications src -name "*.dep" | xargs rm

2 Creating the USB-Stick

2.1 Reminders

  • sdX refers to the device itself for the USB drive.
  • sdX1 refers to the first partition.
  • You can find the actual device identification if you use the auto-mount feature that Linux Distributions provide to explore the drive's contents. Then run:
    mount

    and the last entry is likely the one related to the USB drive, assuming it's the latest one mounted in the system. For example, it may show this:

    /dev/sdf1 on /media/ofuser/F361-E869 type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)

    which means that:

    • /dev/sdf is the drive's device identification path;
    • /dev/sdf1 is the drive's first partition path.


2.2 Commands for formatting USB-Stick drive

These steps are dedicated for creating a stick drive with a partition size that should fit all 8GB USB drives, including those that lie about their total size, such as those that really only have around 7,750,000,000 bytes. The following instructions aim for a primary partition of 7,654,604,800 bytes.

Steps:

  1. Switch to root or keep in mind to do the command with sudo.
  2. First of all, unmount the partition that it probably auto-mounted:
    umount /dev/sdX1
  3. For creating the partition:
    fdisk /dev/sdX
    d  #delete
    o  #create DOC partition table
    c  #toggle DOS compatibility
     
    #should show:
    #  DOS Compatibility flag is set (DEPRECATED!)
     
    n  #new partition
    p  #new partition
    1  #partition 1
    62    #first sector
    +7300M  #end sector, as size in MB
     
    t  #change type
    c  #change to W95 FAT32 (LBA)
     
    a  #make it bootable
     
    p  #prints current table on screen make a note of the shown values, because you're going to need the End sector number
     
    w  #write partition table
    Note
    Make sure you have the units in sectors:
    u  # use as many times as possible until it shows
     
    #Changing display/entry units to sectors
  4. Optional - Full clean up of partition:
    dd if=/dev/zero of=/dev/sdX1 bs=512
    1. In another terminal (also as root), tun:
      while killall -USR1 dd; do sleep 30; done
    2. Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
    3. It will probably end with the message:
      dd: error writing '/dev/sdX1': No space left on device

      because we didn't define the total number of bytes/blocks to write.

    4. Once completed, make sure it's fully written by running:
      sync

      wait until it returns control back to you.

  5. Once complete, then for formatting the partition properly:
    mkfs -t vfat /dev/sdX1


2.3 Creating the live USB

Tested with Lubuntu 15.10 x86_64.

Need to have usb-creator-gtk installed and we will be using Lubuntu 64-bit as the basis for our stick drives. In addition, it's advisable to do these steps from a working installation of the desired Lubuntu version, even if it's within a virtual machine.

Steps:

  1. As root (or with sudo), run:
    usb-creator-gtk
  2. Select the ISO for the Lubuntu Live CD/DVD.
  3. Select the stick drive you want as the target for the Live USB.
    • Note: Do not use the erase function, otherwise you will loose the correct booting mechanism that was created with fdisk and mkfs.
  4. Select the largest possible "Stored in reserved extra space", at least taking into account how much will be left over for the read-anywhere files, such as presentations and VM installers.
    • For example:
      • 690 MiB for the cloned Live CD
      • 1.0 GiB for the read-anywhere data.
      • 7.1 GiB - 1.0 GiB - 690 MiB = 5.42 GiB = 5556.4 MiB... which is larger than 4GiB, the file size limit for FAT32 partitions.
  5. Finally, click on the "Make Startup Disk" button and wait a while for it to run...


2.3.1 First boot up of the new USB-Stick

After booting up the USB-Stick in a real machine or virtual machine, install the necessary packages, with the following commands:

sudo apt-get update
sudo apt-get install virtualbox-guest-utils virtualbox-guest-dkms

which will install the necessary guest additions for Virtualbox to be usable in case the USB-stick doesn't boot directly in the real machine.

Then continue to install the necessary packages for OpenFOAM 3.0.x and foam-extend 3.2.x:

sudo apt-get install git-core build-essential cmake flex bison zlib1g-dev qt4-dev-tools libqt4-dev libqtwebkit-dev gnuplot gnuplot-x11 \
libreadline-dev libncurses-dev libxt-dev libopenmpi-dev openmpi-bin libboost-system-dev libboost-thread-dev libgmp-dev \
libmpfr-dev python python-dev binutils-dev libiberty-dev rpm mercurial graphviz

Then change the desktop image, by right clicking on the desktop and choosing Desktop Preferences. Make sure you download the image from the Workshop website and place it in the folder /home/lubuntu/Images.


2.3.2 Transferring the local installation to the persistent partition file in the drive

Note: These instructions will mount and transfer the files directly to the persistent file in the USB drive, so you might want to not operate directly on this file.

  • If so, copy the file casper-rw to a local disk and roughly follow the steps accordingly with the new path for the file.

Steps:

  1. Mount the stick with the interactive mount that Lubuntu has got from the file manager.
  2. As root (or with sudo), run:
    mkdir /mnt/casper
    mount -t auto /media/$USER/*/casper-rw /mnt/casper/ -o loop
  3. The persistent home folder will now be located at /mnt/casper/upper/home/lubuntu/:
    cd /mnt/casper/upper/home/lubuntu/

    If it doesn't exist, then create it and change owership:

    mkdir -p /mnt/casper/upper/home/lubuntu/
    chown -R 999:999 /mnt/casper/upper/home/lubuntu/
  4. When you're done copying files to the image, make sure the persistent home folder has the correct owner permissions:
    chown -R 999:999 /mnt/casper/upper/home/lubuntu/
  5. Then unmount it:
    umount /mnt/casper
  6. Also unmount the stick:
    umount /dev/sdX1


2.3.3 Creating the bootable ISO for the virtual machine

References:

  1. Install the package genisoimage in your Lubuntu development system:
    sudo apt-get install genisoimage
  2. Mount the USB-stick.
  3. Create a folder where you will place all files that will go into the ISO file. For example:
    mkdir ISOroot
  4. Copy the folders syslinux and install from the USB-stick, into the folder ISOroot, so that you get this folder structure:
    ISOroot
    |-- install
    |-- syslinux
  5. Copy the boot files into the folder syslinux, for example:
    cp /media/$USER/*/casper/{vmlinuz.efi,initrd.lz} ISOroot/syslinux/
  6. Edit the file ISOroot/syslinux/txt.cfg and change it to this:
    default live
    label live
    menu label ^Try Lubuntu without installing
    kernel /syslinux/vmlinuz.efi
    append noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/lubuntu.seed boot=casper initrd=/syslinux/initrd.lz quiet splash ---
    label live-install
    menu label ^Install Lubuntu
    kernel /syslinux/vmlinuz.efi
    append noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/lubuntu.seed boot=casper only-ubiquity initrd=/syslinux/initrd.lz quiet splash ---
    label check
    menu label ^Check disc for defects
    kernel /syslinux/vmlinuz.efi
    append noprompt boot=casper integrity-check initrd=/syslinux/initrd.lz quiet splash ---
    label memtest
    menu label Test ^memory
    kernel /install/mt86plus
    label hd
    menu label ^Boot from first hard disk
    localboot 0x80

    Essentially you will be changing the paths that state /casper/ to /syslinux/.

  7. Create the ISO file by running the following command from the parent folder where ISOroot is located:
    mkisofs -o VMbootUSB.iso    -b syslinux/isolinux.bin -c syslinux/boot.cat    -no-emul-boot -boot-load-size 4 -boot-info-table    ISOroot
  8. The final image file VMbootUSB.iso is what we will need to boot the USB-stick inside the virtual machine.


2.3.4 Creating the virtual machine in Virtualbox

Keep in mind that these steps are only needed for creating the virtual machine to be placed inside the USB-stick. Nonetheless, it might be useful in case the exported OVA file is incompatible.

References:

Side notes:

  • On Ubuntu, had to install the necessary packages for getting the USB2/3 drivers for Virtualbox on the host side (the real machine):
    sudo apt-get install virtualbox-ext-pack

    And then run the following command for adding my user to the group vboxusers:

    sudo usermod -aG vboxusers $USER

    Then I had to log out and then log back in, otherwise it's not possible to use a real USB drive in a virtual machine within Virtualbox.

Steps:

  1. Simply create a new Virtual Machine in Virtualbox, meant for a Ubuntu 64-bit machine and without any disks, because we will boot straight from the USB drive.
  2. After the virtual machine is created, edit its settings and:
    1. increase the RAM to at least 2048MB;
    2. change from 1 to 2 virtual cores;
    3. choose for the optical drive to use the VMbootUSB.iso;
    4. probably best to turn off audio capabilities;
    5. in the USB section, choose USB 3.0 or 2.0 controller, depending on what kind of USB-stick you're using;
    6. still in the USB section, click on the button that adds a new USB device and choose your USB-stick.
  3. Boot up the VM and test it.
  4. If it works fine, do a standard shutdown within the virtual machine.
  5. Then use on the main menu for the Virtualbox the item File -> Export Appliance... to export the virtual machine to a settings file to be placed inside the USB-stick, preferably with the file name OFW11USBVM.ova.
  6. Best to place both the OFW11USBVM.ova and VMbootUSB.iso files inside the folder VM in the real USB-stick partition (i.e. not inside the booted up Live Lubuntu).


2.4 Cloning the original stick to a new image

Steps:

  1. Make sure you don't have the stick mounted! Example to umount:
    umount /dev/sdX1
  2. As root (or with sudo), run:
    dd if=/dev/sdX of=OFW11-2016_v2.img bs=512 count=14950463
    • The value "14950463" is the End sector count 14950462 plus 1.
    • 14950463 sector * 512 B/sector = 7654636544 B = 7654636544 ~= 7300 MiB
    1. In another terminal (also as root), tun:
    while killall -USR1 dd; do sleep 30; done
    1. Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
  3. When it's done cloning, make sure the file system is synced:
    sync

    wait until it returns control back to you.

  4. Now you can mount the stick again, if you want. Or simply eject it.


2.5 Cloning the image to a new stick

It's almost the reverse of the previous steps:

  1. Make sure you don't have the stick mounted! Example to umount:
    umount /dev/sdX1
  2. As root (or with sudo), run:
    dd if=OFW11-2016_v2.img of=/dev/sdX bs=512
    1. In another terminal (also as root):
    ps -a
    1. In another terminal (also as root), tun:
    while killall -USR1 dd; do sleep 30; done
    1. Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
  3. When it's done cloning, make sure the file system is synced:
    sync

    wait until it returns control back to you.

  4. Now you can mount the stick, if you want. Or simply eject it.