Installation/Live Images/USB-Stick for OFW11
WARNING: These instructions are currently being written and under testing!
Contents
- 1 Creating the development machine
- 1.1 Installation details
- 1.2 Preparations before copying to final image
- 1.3 Cheat sheet for copying installation to USB-Stick
- 1.3.1 Copying OpenFOAM 3.0.x et al
- 1.3.2 Copying foam-extend 3.2.x et al
- 1.3.3 Packing the sandbox to a compressed image
- 1.3.4 Copying the compressed images to the USB-stick
- 1.3.5 Making sure the compressed images are properly used
- 1.3.6 Copying the folders that were not compressed
- 1.3.7 Making the compressed filesystems mounted at start
- 1.3.8 Update the aliases file
- 1.3.9 Next steps - go back to transferring from-to the USB-stick
- 2 Creating the USB-Stick
1 Creating the development machine
1.1 Installation details
In a nutshell:
- Install Lubuntu 15.10 x86_64 in a virtual machine.
- Install OpenFOAM 3.0.x, as instructed here: Installation/Linux/OpenFOAM-3.0.x/Ubuntu#Ubuntu_15.10
- Install foam-extend 3.2.x, based on the instructions from here: Installation/Linux/foam-extend-3.2/Ubuntu#Ubuntu_15.10
- Install swak4Foam, PyFoam and cfMesh for OpenFOAM 3.0.x, since these are already in foam-extend 3.2.x.
1.1.1 Install Lubuntu 15.10 x86_64 in a virtual machine
TODO
1.1.2 Packages that need to be installed in the development VM
TODO
Part of the software stack:
sudo apt-get install squashfs-tools
1.1.3 Install OpenFOAM 3.0.x
TODO
1.1.3.1 Install additional contributions for OpenFOAM 3.0.x
TODO swak4Foam, PyFoam and cfMesh
1.1.4 Install foam-extend 3.2.x
TODO
1.2 Preparations before copying to final image
This is a cheat sheet on the commands needed for cleaning up each installation before copying to the USB-Stick:
1.2.1 Clean up OpenFOAM 3.0.x et al
of30x
foam3rdParty
rm -r build
rm log.*
rm *.log
cd ..
wclean all swak4Foam
wclean all cfMesh
rm -r PyFoam/build/
1.2.2 Clean up foam-extend 3.2.x et al
fe32 rm log.make
1.3 Cheat sheet for copying installation to USB-Stick
Notes:
- This is meant to be used mid-way of the instructions from sub-section Transferring the local installation to the persistent partition file in the drive.
First of all, create our sandbox folder:
mkdir ~/sandbox
And if this isn't the first time copying, run:
sudo chown -R $USER:$USER ~/sandbox
1.3.1 Copying OpenFOAM 3.0.x et al
In a new terminal, run:
of30x cd ~/OpenFOAM TARGET=~/sandbox/OpenFOAM mkdir $TARGET/ rsync -a swak4Foam cfMesh PyFoam lubuntu-3.0.x $TARGET/ rsync -a OpenFOAM-3.0.x/ $TARGET/OpenFOAM-3.0.x/ rsync -a ThirdParty-3.0.x/ $TARGET/ThirdParty-3.0.x/ cd $TARGET/OpenFOAM-3.0.x/ rm -r platforms/linux64GccDPInt32OptSYSTEMOPENMPI/ rm -r platforms/linux64GccDPInt32Opt/{src,applications} cd $TARGET/ThirdParty-3.0.x/ rm -r CGAL-4.7/ ParaView-4.4.0/ scotch_6.0.3/
1.3.2 Copying foam-extend 3.2.x et al
In a new terminal, run:
fe32 cd ~/foam TARGET=~/sandbox/foam mkdir $TARGET/ rsync -a site $TARGET/ mkdir $TARGET/foam-extend-3.2 cd foam-extend-3.2/ rsync -a ./ $TARGET/foam-extend-3.2/ cd $TARGET/foam-extend-3.2/ find applications/{solvers,utilities} src -name "$WM_OPTIONS" | xargs rm -r rm -r ThirdParty/rpmBuild/{BUILD*,TGZS,RPMS,rpmDB}/* find applications src -name "*.dep" | xargs rm
1.3.3 Packing the sandbox to a compressed image
Given that the final installation is over 2 GiB, therefore we need to save up space by relying on squashfs to get the whole thing trimmed down to something more manageable (for more details, see this page).
But first, we must change the ownership of the contents:
sudo chown -R 999:999 ~/sandbox/OpenFOAM/OpenFOAM-3.0.x sudo chown -R 999:999 ~/sandbox/OpenFOAM/ThirdParty-3.0.x sudo chown -R 999:999 ~/sandbox/foam/foam-extend-3.2
To do so, run:
sudo mksquashfs ~/sandbox/OpenFOAM/OpenFOAM-3.0.x ~/sandbox/OpenFOAM-3.0.x.sqfs sudo mksquashfs ~/sandbox/OpenFOAM/ThirdParty-3.0.x ~/sandbox/ThirdParty-3.0.x.sqfs sudo mksquashfs ~/sandbox/foam/foam-extend-3.2 ~/sandbox/foam-extend-3.2.x.sqfs
- Note: If this isn't the first time you are running these commands, then run before the commands above:
sudo rm ~/sandbox/*.sqfs
Check the final size of the files, just in case:
ls -l -h ~/sandbox/*.sqfs
1.3.4 Copying the compressed images to the USB-stick
It's simply this:
rsync -a -v --progress ~/sandbox/*.sqfs /media/$USER/OFW11/
1.3.5 Making sure the compressed images are properly used
In a new terminal, run:
cd /mnt/casper/upper/home/lubuntu/
Create the place-holder folders and symbolic links which will link into the original folders:
mkdir -p OpenFOAM/OpenFOAM-3.0.x mkdir -p OpenFOAM/ThirdParty-3.0.x mkdir -p foam/foam-extend-3.2
1.3.6 Copying the folders that were not compressed
Simply run:
cd ~/sandbox/OpenFOAM/ rsync -a swak4Foam cfMesh PyFoam lubuntu-3.0.x /mnt/casper/upper/home/lubuntu/OpenFOAM/ cd ~/sandbox/foam/ rsync -a site lubuntu-3.2 /mnt/casper/upper/home/lubuntu/foam/
1.3.7 Making the compressed filesystems mounted at start
Edit the rc.local file:
sudo nano /mnt/casper/upper/etc/rc.local
Place the following inside the file:
#!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will "exit 0" on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. mount /cdrom/OpenFOAM-3.0.x.sqfs /home/lubuntu/OpenFOAM/OpenFOAM-3.0.x -t squashfs -o loop,ro mount /cdrom/ThirdParty-3.0.x.sqfs /home/lubuntu/OpenFOAM/ThirdParty-3.0.x -t squashfs -o loop,ro mount /cdrom/foam-extend-3.2.x.sqfs /home/lubuntu/foam/foam-extend-3.2 -t squashfs -o loop,ro exit 0
Save and close, by using the key combination Ctrl+X and then hit the Y key separately.
Finally, make the script executable:
sudo chmod +x /mnt/casper/upper/etc/rc.local
1.3.8 Update the aliases file
Edit the .bash_aliases on the persistent mounted folder:
nano /mnt/casper/upper/home/lubuntu/.bash_aliases
And place the following code inside it:
alias of30x='source $HOME/OpenFOAM/OpenFOAM-3.0.x/etc/bashrc' alias fe32='source $HOME/foam/foam-extend-3.2/etc/bashrc'
Save and close, by using the key combination Ctrl+X and then hit the Y key separately.
1.3.9 Next steps - go back to transferring from-to the USB-stick
Now you can continue where you left off at section Transferring the local installation to the persistent partition file in the drive.
2 Creating the USB-Stick
2.1 Reminders
- sdX refers to the device itself for the USB drive.
- sdX1 refers to the first partition.
- You can find the actual device identification if you use the auto-mount feature that Linux Distributions provide to explore the drive's contents. Then run:
mount
and the last entry is likely the one related to the USB drive, assuming it's the latest one mounted in the system. For example, it may show this:
/dev/sdf1 on /media/ofuser/F361-E869 type vfat (rw,nosuid,nodev,relatime,uid=1000,gid=1000,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,showexec,utf8,flush,errors=remount-ro,uhelper=udisks2)
which means that:
- /dev/sdf is the drive's device identification path;
- /dev/sdf1 is the drive's first partition path.
- Although this might seem counter-intuitive, if you have already done several trials in formatting and using dd on the USB drive, Windows 10 might stop recognizing it and Windows 7 will complain about some issues, because the partition table is not sane enough for conventional use. The solution is to:
- Format the USB drive (stick) with gparted into FAT32 or NTFS.
- Then go into Windows to check if it's recognizable again.
- If it's recognizable again, you might as well try saving files in the USB stick on Windows, just so you can make sure that it's working as intended.
- Finally, you can go into Lubuntu again and get back into this tutorial for creating a live USB stick.
2.2 Commands for formatting USB-Stick drive
These steps are dedicated for creating a stick drive with a partition size that should fit all 8GB USB drives, including those that lie about their total size, such as those that really only have around 7,750,000,000 bytes. The following instructions aim for a primary partition of 7,654,604,800 bytes.
Steps:
- Switch to root or keep in mind to do the command with sudo.
- First of all, unmount the partition that it probably auto-mounted:
umount /dev/sdX1
- For creating the partition:
fdisk /dev/sdX d #delete o #create DOC partition table c #toggle DOS compatibility #should show: # DOS Compatibility flag is set (DEPRECATED!) n #new partition p #new partition 1 #partition 1 62 #first sector +7300M #end sector, as size in MB t #change type c #change to W95 FAT32 (LBA) a #make it bootable p #prints current table on screen make a note of the shown values, because you're going to need the End sector number w #write partition table
- Note
- Make sure you have the units in sectors:
u # use as many times as possible until it shows #Changing display/entry units to sectors
- Optional - Full clean up of partition:
dd if=/dev/zero of=/dev/sdX1 bs=512
- In another terminal (also as root), tun:
while killall -USR1 dd; do sleep 30; done
- Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
- It will probably end with the message:
dd: error writing '/dev/sdX1': No space left on device
because we didn't define the total number of bytes/blocks to write.
- Once completed, make sure it's fully written by running:
sync
wait until it returns control back to you.
- In another terminal (also as root), tun:
- Once complete, then for formatting the partition properly:
mkfs -t vfat -n OFW11 /dev/sdX1
2.3 Creating the live USB
Tested with Lubuntu 15.10 x86_64.
Need to have usb-creator-gtk installed and we will be using Lubuntu 64-bit as the basis for our stick drives. In addition, it's advisable to do these steps from a working installation of the desired Lubuntu version, even if it's within a virtual machine.
- Warning
- There is a critical bug in syslinux that is provided in Ubuntu 15.10, as reported here. The work around is to run the following commands:
cd wget http://launchpadlibrarian.net/200593067/syslinux_6.03+dfsg-5ubuntu1_amd64.deb wget http://launchpadlibrarian.net/200593072/syslinux-common_6.03+dfsg-5ubuntu1_all.deb sudo dpkg -i syslinux-common_6.03+dfsg-5ubuntu1_all.deb syslinux_6.03+dfsg-5ubuntu1_amd64.deb
- The Deb packages came from here and here.
- The following hack is optional, but advisable, if you don't want to have to do the last fix on the steps below every time you need to do another live USB:
sudo sed -i -e 's=\(if len(target_mounts) > 0:\)=if False: #\1=' /usr/share/usb-creator/usb-creator-helper
As this will disable the auto-mount-and-get-damaged-syslinux-from-casper.
Steps:
- As root (or with sudo), run:
usb-creator-gtk
- Select the ISO for the Lubuntu Live CD/DVD.
- Select the stick drive you want as the target for the Live USB.
- Note: Do not use the erase function, otherwise you will loose the correct booting mechanism that was created with fdisk and mkfs.
- Select the largest possible "Stored in reserved extra space", at least taking into account how much will be left over for the read-anywhere files, such as presentations and VM installers.
- For example:
- 690 MiB for the cloned Live CD
- 1.0 GiB for the read-anywhere data.
- 7.1 GiB - 1.0 GiB - 690 MiB = 5.42 GiB = 5556.4 MiB... which is larger than 4GiB, the file size limit for FAT32 partitions.
Finally, click on the "Make Startup Disk" button and wait a while for it to run...
When it's finally completed, we need to do some additional fixing:
- For example:
- Unmount the stick:
umount /dev/sdX1
- If you didn't do the hack for /usr/share/usb-creator/usb-creator-helper, then you'll need to redeploy the syslinux and MBR:
sudo syslinux -f /dev/sdX1 sudo dd if=/usr/lib/syslinux/mbr/mbr.bin of=/dev/sdX bs=446 count=1 conv=sync sudo sync
- Now interactively mount the USB-stick again and then you can finally unmount+eject interactively.
2.3.1 First boot up of the new USB-Stick
After booting up the USB-Stick in a real machine or virtual machine, install the necessary packages, with the following commands:
sudo apt-get update sudo apt-get install virtualbox-guest-utils virtualbox-guest-dkms
which will install the necessary guest additions for Virtualbox to be usable in case the USB-stick doesn't boot directly in the real machine.
Then continue to install the necessary packages for OpenFOAM 3.0.x and foam-extend 3.2.x:
sudo apt-get install git-core build-essential cmake flex bison zlib1g-dev qt4-dev-tools libqt4-dev libqtwebkit-dev gnuplot gnuplot-x11 \ libreadline-dev libncurses-dev libxt-dev libopenmpi-dev openmpi-bin libboost-system-dev libboost-thread-dev libgmp-dev \ libmpfr-dev python python-dev binutils-dev libiberty-dev rpm mercurial graphviz
Now proceed with cleaning up the downloaded Deb files:
sudo apt-get clean sudo apt-get autoclean
Then change the desktop image, by right clicking on the desktop and choosing Desktop Preferences. Make sure you download the image from the Workshop website and place it in the folder /home/lubuntu/Images.
For adding a link to the LX terminal (as shown in the image to the right):
- Right click on the tool-bar at bottom and choose Add / Remove Panel Items.
- Click on the first occurrence of Application Launch Bar.
- Click on the button Preferences.
- In the new window Application Launch Bar, in the Installed Applications, browse into System Tools and pick LXTerminal.
- Click on the + Add button.
- Finally, click on the Close buttons for each one of the opened windows.
You can also add the LXTerminal icon to the desktop environment, by right clicking in the System Tools -> LXTerminal icon at the LXDE main menu (start menu).
2.3.2 Transferring the local installation to the persistent partition file in the drive
Steps:
- Mount the stick with the interactive mount that Lubuntu has got from the file manager.
- Copy the image file to the home folder. This file will contain the persistent installation on the USB-stick, which is why it's best to first copy to your system, so that's quicker in storing several thousands of files inside it:
rsync -a -v --progress /media/$USER/OFW11/casper-rw ~/sandbox/casper-rw
- As root (or with sudo), run:
mkdir /mnt/casper mount -t auto ~/sandbox/casper-rw /mnt/casper/ -o loop
- The persistent home folder will now be located at /mnt/casper/upper/home/lubuntu/:
cd /mnt/casper/upper/home/lubuntu/
If it doesn't exist, then create it and change ownership:
mkdir -p /mnt/casper/upper/home/lubuntu/ chown -R $USER:$USER /mnt/casper/upper/home/lubuntu/
- Copy the files you have on the sandbox folder into the designated folder, as explained in section Cheat sheet for copying installation to USB-Stick.
- When you're done copying files to the image, make sure the persistent home folder has the correct owner permissions:
chown -R 999:999 /mnt/casper/upper/home/lubuntu/
- Then unmount it:
umount /mnt/casper
- Then copy the casper-rw file back to the USB-stick:
rm /media/$USER/OFW11/casper-rw rsync -a -v --progress ~/sandbox/casper-rw /media/$USER/OFW11/
- Finally, unmount the stick:
umount /dev/sdX1
2.3.3 Creating the bootable ISO for the virtual machine
References:
- Install the package genisoimage in your Lubuntu development system:
sudo apt-get install genisoimage
- Mount the USB-stick.
- Create a folder where you will place all files that will go into the ISO file. For example:
mkdir ISOroot
- Copy the folders syslinux and install from the USB-stick, into the folder ISOroot, so that you get this folder structure:
ISOroot |-- install |-- syslinux
- Copy the boot files into the folder syslinux, for example:
cp /media/$USER/OFW11/casper/{vmlinuz.efi,initrd.lz} ISOroot/syslinux/
- Edit the file ISOroot/syslinux/txt.cfg and change it to this:
default live label live menu label ^Try Lubuntu without installing kernel /syslinux/vmlinuz.efi append noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/lubuntu.seed boot=casper initrd=/syslinux/initrd.lz quiet splash --- label live-install menu label ^Install Lubuntu kernel /syslinux/vmlinuz.efi append noprompt cdrom-detect/try-usb=true persistent file=/cdrom/preseed/lubuntu.seed boot=casper only-ubiquity initrd=/syslinux/initrd.lz quiet splash --- label check menu label ^Check disc for defects kernel /syslinux/vmlinuz.efi append noprompt boot=casper integrity-check initrd=/syslinux/initrd.lz quiet splash --- label memtest menu label Test ^memory kernel /install/mt86plus label hd menu label ^Boot from first hard disk localboot 0x80
Essentially you will be changing the paths that state /casper/ to /syslinux/.
- Create the ISO file by running the following command from the parent folder where ISOroot is located:
mkisofs -o VMbootUSB.iso -b syslinux/isolinux.bin -c syslinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table ISOroot
- The final image file VMbootUSB.iso is what we will need to boot the USB-stick inside the virtual machine.
2.3.4 Creating the virtual machine in Virtualbox
Keep in mind that these steps are only needed for creating the virtual machine to be placed inside the USB-stick. Nonetheless, it might be useful in case the exported OVA file is incompatible.
References:
Side notes:
- On Ubuntu, had to install the necessary packages for getting the USB2/3 drivers for Virtualbox on the host side (the real machine):
sudo apt-get install virtualbox-ext-pack
And then run the following command for adding my user to the group vboxusers:
sudo usermod -aG vboxusers $USER
Then I had to log out and then log back in, otherwise it's not possible to use a real USB drive in a virtual machine within Virtualbox.
Steps:
- Simply create a new Virtual Machine in Virtualbox, meant for a Ubuntu 64-bit machine and without any disks, because we will boot straight from the USB drive.
- After the virtual machine is created, edit its settings and:
- increase the RAM to at least 2048MB;
- change from 1 to 2 virtual cores;
- choose for the optical drive to use the VMbootUSB.iso;
- probably best to turn off audio capabilities;
- in the USB section, choose USB 3.0 or 2.0 controller, depending on what kind of USB-stick you're using;
- still in the USB section, click on the button that adds a new USB device and choose your USB-stick.
2.4 Cloning the original stick to a new image
Steps:
- Make sure you don't have the stick mounted! Example to umount:
umount /dev/sdX1
- As root (or with sudo), run:
dd if=/dev/sdX of=OFW11-2016_v2.img bs=512 count=14950463
- The value "14950463" is the End sector count 14950462 plus 1.
- 14950463 sector * 512 B/sector = 7654636544 B = 7654636544 ~= 7300 MiB
In the meantime, while it's copying:
- In another terminal (also as root), run:
while killall -USR1 dd; do sleep 30; done
- Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
- When it's done cloning, make sure the file system is synced:
sync
wait until it returns control back to you.
- Now you can mount the stick again, if you want. Or simply eject it.
2.5 Cloning the image to a new stick
It's almost the reverse of the previous steps:
- Make sure you don't have the stick mounted! Example to umount:
umount /dev/sdX1
- As root (or with sudo), run:
dd if=OFW11-2016_v2.img of=/dev/sdX bs=512
- In another terminal (also as root):
ps -a
- In another terminal (also as root), run:
while killall -USR1 dd; do sleep 30; done
- Then look at the previous terminal window and every 30 seconds it should update on the current write status. Given how this command works, if you have more than one dd running, it will update for all running dd applications.
- In another terminal (also as root):
- When it's done cloning, make sure the file system is synced:
sync
wait until it returns control back to you.
- Now you can mount the stick, if you want. Or simply eject it.
2.6 Converting the USB-stick image to a VDI disk
The objective here is for people who cannot write the image to a USB drive, but that would still like to run the image in a VirtualBox. For these people, we can easily create a VDI disk like this:
VBoxManage convertdd --format VDI OFW11-2016_v2.img OFW11-2016_v2.vdi
Side note: Got the conversion idea from here, but the correct command is from here.