Skip to end of metadata
Go to start of metadata
This page documents building an older way to build packages. It has been superseded by: https://pkgsrc.joyent.com/docs/building/

The PK Framework

The pk framework helps manage daily tasks related to pkgsrc binary packages. The config files included with the pk framework plug into the pkgsrc configuration to aid in building on SmartOS when using standard pkgsrc workflows. However, you can also wrap them around pkgsrc tools and use them to repeatedly manage predefined sets of packages.

The pk framework does currently support full bulk builds. In other words, build the complete pkgsrc tree.

Setting Up the PK Framework

The best way to build pkgsrc packages on SmartOS and/or the Joyent Public Cloud, is to start with a machine provisioned using a 'base' or 'base64' image (these very previously known as 'smartos' and 'smartos64'). Other images will also work but will include extra packages you may or may not need (as far as pkgsrc is concerned).

Note: if you use the previous (smartos/smartos64) images, the GCC compiler available there cannot automatically build 32bit or 64bit objects as needed, and need to build something manually (outside of pkgsrc control), make sure to pass -m64 to GCC in order to get 64bit objects. The base64 image uses GCC 4.7, which finally supports a default 64bit target, so you don't need to do that. (Also, building through pkgsrc automatically adjusts the flags as needed too, regardless of the GCC version.)

For these instructions, the parent path is arbitrary. However, you will need to adjust a config file if you do not plan to use the default path of /opt/pkgsrc.


  1. Ensure git is installed and install the GNU Compiler Collection (gcc) from the repository.

  2. Clone the main pkgsrc tree from github.
    This will take several minutes to complete.



  3. Install pkgsrc modules. 

    This is optional but recommended. Joyent uses both repositories as submodules when building out the standard package set for SmartOS. If you only care about packages available in the main pkgsrc repository, you can safely ignore these submodules.
  4. Clone the pk framework from github.

  5. Add the pk framework path to your PATH environment variable.

  6. Open /opt/local/etc/mk.conf and ensure all values are set correctly. You do not need to change anything unless you install pk framework in a path other than the default. If you install pk framework in a different directory, adjust the OVERLAY path accordingly.
    This ensures the pk configuration files are sourced properly by pkgsrc.


You can test the install by running this command:


If successful, you will see something similar to the following:

Output of pk info

Configuring the pk Framework

You can define many of the variables internally used in pk through a simple config file. This allows you to avoid passing in an unwieldy number of options on the command line if you are running the pk framework in a non-standard way. Use the configuration file found at config/pkrc.example as a template. You can create a similar file at /.pkrc or copy the example file as @/.pkrc@ and change as needed. 

Building a Single Package

Here is an example of building a simple package with no dependencies.

Example of building a single package


For now, you can safely ignore the warning about the availability of ZFS rollback. See below for more information.

The proper way to pass packages to the pk build command is to use full PKGPATH notation. In other words, the relative category/package_name path under the pkgsrc tree. Look under /opt/pkgsrc.

In the above example, the first thing pkgsrc does is to determine that gmake is needed to build this particular package. At that point, it builds the package and installs it, resulting in a binary for each package. Packages are stored in a PACKAGES directory, which is a variable based on the current pkgsrc release and the compiler chosen. In this example, packages are stored in /opt/pk/packages/2012Q1/i386/All/.

You can see the PACKAGES path by using pk info. See the above pk info example.

Using Binary Packages

Once the build completes, nothing is installed except package dependencies. You can use packages that you build by adding the path to your packages in your pkgin setup. For example:


If more than one version of a package exists across all registered repositories, pkgin will default to the latest version.

If you need to install an earlier version of a package, specify the full version binary or use the "greater than" (>) or "less than" (<) symbols to install an earlier version than what you specify. The following commands will install PHP 5.2 before the current version of PHP 5.3:

You can also use pkg_add with an absolute path to the package binary files or set PKG_PATH to the repository path before you call pkg_add. The following lines perform the same action:


If ZFS delegation (TBD) is not available, anything installed explicitly or as a dependency will stay in place until you remove it yourself.

Serving Binary Packages

The packages built by pkgsrc are just files on disk. So, you can use the web server of your choice to serve up the packages you build. The pk script can rsync the binary packages over to a different system.

Joyent Cloud Specific Notes

When using Joyent Public Cloud use the base[64] 1.7.x image. You can upgrade previous smartos[64] JoyentCloud machines to 2012Q1 by installing the 'smtools' package from the 2012Q1 package set, and calling the 'sm-rebuild-pkgsrc' script contained in the package. Pick the package set that matches the architecture you need (i.e. 'i386' for the 32bit smartos/base image, and 'x86_64' for the 64bit smartos64/base64 image):

or

then

and follow the instructions.

Labels:
pkgsrc pkgsrc Delete
package package Delete
binary binary Delete
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.
  1. Apr 29, 2012

    I tried following the instructions and when building archivers/unzip (or any other package) it abruptly ends with an error like this:

    And for the sake of documentation:

    Suggestions?

    1. May 11, 2012

      Suggested by Filip:

      "can you checkout the 'pkgsrc_2011Q4' branch of the 'pk' repository instead of master? I'm making some compiler related changes in pk as I work on 2012Q1, and it's my guess those are the reason the build is getting rejected. Btw. if you look under /opt/pk/logs you should fine more clues as to why the build failed. If a package fails, pk leaves the respective log file around (likely something like /opt/pk/logs/<date-time-stamp>/common/archivers_unzip.log)."

  2. May 14, 2012

    I needed to add two packages.  digest and nbpatch ie

    This goes a little further it uncovers the issue that it believe gcc is not installed and is trying to bootstrap gcc.   

    Log Output - pk build archivers/unzip

    This fails due to not being able to build some of the dependencies.  Considering gcc is installed (using pkgin in) I'm not sure it should be doing this.  It then builds the first few dependencies for gcc46 being gtextinfo and libtoo-base but fails on mpfr due to libgmp due to a different ABI. 

    1. May 15, 2012

      Angus, I'm confused. I just repeated the steps on this page on a clean SmartMachine, and I haven't hit the issue. Can you make sure you're on the 'pkgsrc_2011Q4' branch on the *pk* repository please? I had started making some changes earlier in preparation for 2012Q1, and forgot to make the 'pkgsrc_2011Q4' branch the default on Github, so you might have gotten 'master' instead.

  3. May 25, 2012

    Okay, it looks like these instructions only work on a smartos[64] 1.6.x image, which (as of this writing) only appear to be available in eu-ams-1 and (as of very recently) us-west-1. 1.5.4, the most widely deployed, is on 2011Q3, which looks to be be why we've been struggling

    1. May 25, 2012

      Uh. You're right in that there seems to be an inconsistency amongst image versions available across the data centers. I'll see to bringing them into sync. Meanwhile, you could use the update.sh script from the 2011Q4 package set URL to update your SmartMachine to 2011Q4.

      1. May 25, 2012

        oh, I hadn't really appreciated that update.sh was there. thanks for the pointer!

        and I just took it for granted that the image versions were inconsistent across DCs. Had I known that other people didn't see it, I would've brought it up earlier!

  4. Oct 16, 2012

    I suspect that the default value for tmpfs is not enough in a base64 image (60a3b1fa-0674-11e2-abf5-cb82934a8e24) to build gettext-tools.  I tried using a base64 image, having not set anything special when I created the VM.  I'm trying to build things for a mail system.  I started with clamav and it stopped in gettext-tools.  Checking the build log it said not enough space in /tmp/.... I then changed in the GZ, tmpfs (512), and max_physical_memory (1GB).  restarted zone and gettext-tools now builds.

    My bare metal server is a Sun X2100M2 with 4GB of RAM.  Not speedy by today's standards but it runs SmartOS just fine.

  5. Nov 30, 2012

    It appears that if you want SMF support built with your package, you have to use http://github.com/mamash/pkgsrc instead of http://github.com/joyent/pkgsrc .

    Unfortunately mamash/pkgsrc is now 5 months out of date (forked from 2011Q4). Will mamash/pkgsrc be merged back into joyent/pkgsrc anytime soon?

  6. Jul 27, 2013

    A number of things here are a little tough to figure out.  I have a fork of pkgsrc, particularly wip, with an updated version of one package.  I can build it happily with bmake.  I'm coming to this from lots of experience with Gentoo Portage, so my expectations may be a little skewed.

    It seemed sane to use a custom prefix, so I added one to ~/.pkrc (BTW, the scripts in pk appear to look for ~/.pkgsrc.conf - I created a symlink for it, but is this doc out of date?).  But using the custom prefix fails, since it attempts to find binaries of all a project's dependencies under that new (empty) prefix.  I really just want to package the one modified package - linking against existing binaries is fine for the rest.  But it appears that if I want that one package, I also need to compile all of its prerequisites from source - ?  Or is there a way to instruct pk "build this thing to this prefix, but use this other prefix for resolving dependencies (copying them if necessary)"

    It's not clear how one should set up zfs delegation for this purpose (I did read the vmadm page about it - but it's not obvious how having a mount called /data would solve anything) - this page notes that, for example, clean will not work without it (and indeed it doesn't).

    Also not obvious that running 'pk build' by itself will try to build the universe.  I stopped this after it had been running for a day, after reading the note above that says "The pk framework does currently support full bulk builds" which suggests this would have just gone on for a long time before dying eventually.  Since it appears that that's what you have to do if you want to use a custom prefix, *and* pk will fail at that, how *do* you do it?