Skip to end of metadata
Go to start of metadata

The Goal

To migrate my main OpenIndiana box to Joyent's SmartOS.

The Motivation

My reason? I want to run an illumos distribution with clear focus on a server OS with active development.
Also the people at Joyent went with NetBSDs pkgsrc(7) for building and maintaining third-party software instead of IPS.
Having also run FreeBSD for many years, SmartOS just felt like a perfect match for my needs and interests :)


When I decided to run OpenIndiana as my storage box a few years ago, SmartOS wasn't around.
Quickly it became evident that the development of OpenIndiana became slow and the IPS package repositories didn't give me the packages I was looking for.

The Excercise (a.k.a. getting busy)

Used smartos-20130222T000747Z
All the steps described below are performed using the smartos-20130222T000747Z DVD ISO

This weekend I finally started the migration from OpenIndiana to SmartOS.
I was running OI on a single vdev mirrored rpool of 2 disks.
I also use a single vdev 3-disk raidz zpool for storage and a single vdev single disk backup pool (sata drive in a usb enclosure).

After making the necessary backups to my backup zpool it was finally time to start my journey :)
Under OI my zpools used the following controller and targets:

c3t1d0s0 and c3t0d0s0 was my mirrored rpool (rpool)
c3t2d0, c3t3d0 and c3t4d0 was my raidz pool (pool1)
c4t0d0p0 was my external backup pool (backup)

I went with the latest SmartOS image available which at this time is smartos-20130222T000747Z and can be downloaded from
My system has problems booting from usb so I got the DVD ISO, burned it and booted from it.
I chose the no-install option from the SmartOS menu to verify on which controller and targets SmartOS would find my disks.

zpool import showed the following:

rpool using c1t0d0s0 and c1t1d0s0
pool1 using c1t2d0 , c1t3d0 and c1t4d0
backup using c0t0d0p0

I find it interesting why SmartOS found the disks to use different controller ids than OI!

Since I wanted a path back to OI in case SmartOS on my box wouldn't properly install or run I decided to split my OI rpool.
So rebooting back into OI, then it was time to detach disk 2 of the rpool mirror leaving disk 1 intact:

# zpool detach rpool c3t0d0s0
Check and double check which disk you are detaching to not wipe out the wrong disk and thus your OI install by giving this disk to the SmartOS install
If SmartOS finds a zpool on a disk it might have difficulty or simply won't allow this disk to be installed onto.

Before rebooting back to SmartOS I decided to write out some zeroes to the beginning of the disk so the disk would be clear to install to.

# dd if=/dev/zero of=/dev/rdsk/c3t0d0s0 bs=1024 count=5

Now we've taken care of that is was time to reboot back into SmartOS again to choose the install option.
Now to provide my choices for the very few questions SmartOS asks which is super simplistic and great :)

- assigning a static IP to the admin nic
- install to c1t0d0
- enter a password for root

It's trivial to attach a 2nd disk turning the SmartOS zones pool into a mirror so I installed SmartOS to just 1 disk for now.

After a short while everything was setup and it was time to reboot into the SmartOS live image.
No problems booting and soon I was presented with the console login of the live image.

Once I was logged into the global zone, a zpool list showed that all of the zpools had been imported, including the now single disk OI rpool
I decided to destroy the rpool of the OI install and attach that disk to the zones pool ASAP.

[root@00-1b-21-98-51-c1 ~]# zpool destroy rpool

With that gone, we can turn the zones pool into a mirror.

This IMHO is imperative for a type 1 hypervisor such as SmartOS since it's running entirely off of a ramdisk!

[root@00-1b-21-98-51-c1 ~]# zpool attach zones c1t0d0 c1t1d0

After the resilver of disk 2 was complete here's the status of the zones pool:

[root@00-1b-21-98-51-c1 ~]# zpool status zones
  pool: zones
 state: ONLINE
  scan: resilvered 1.36G in 0h0m with 0 errors on Sat Mar  2 16:16:15 2013

        NAME        STATE     READ WRITE CKSUM
        zones       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            c1t0d0  ONLINE       0     0     0
            c1t1d0  ONLINE       0     0     0

Resilvering the newly attached disk completes quite quickly at this point, since there is very little data on it.

I went for a reboot to verify if the system would come up properly.
Reboot completed without any hitch, here's the complete listing of all zpools on the system:

[root@00-1b-21-98-51-c1 ~]# zpool list
backup  1.36T   112G  1.25T         -     8%  1.00x  ONLINE  -
pool1   5.44T  2.62T  2.81T         -    48%  1.00x  ONLINE  -
zones    232G  1.40G   231G         -     0%  1.00x  ONLINE  -

I noticed that, despite having zpool upgraded my zpools on the latest development release of OI (oi_151a7) to use zpool-features(5) , running zpool upgrade on SmartOS still showed that pools pool1 and backup could be upgraded!
Recently some excellent work has gone into illumos implementing lz4 compression and filesystem_limits into ZFS(1M) which were implemented by Joyent to their SmartOS builds.
This just warranted my decision from migrating from OI to SmartOS even more.

The verdict

I feel the migration was a success!
Since the migration I've setup a couple of zones for mail and irc and will be installing several more.

All zones are using the 12.4.1 multiarch ( repository provided by Joyent's @jperkin (

oi oi Delete
smartos smartos Delete
migrate migrate Delete
pkgsrc pkgsrc Delete
multiarch multiarch Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.