Use pkgsrc on ARM

What is this page?

This page describes how to use pkgsrc on ARM architecture with EABI support. I bought TeraStation Live, which is called "TeraStation Living" in Japan, and wanted to use it as a home server. The original system in TeraStation Live, however, contains the very minimum software, and does not have development environment. If you want to use some new software, you have to prepare an additional system to install it. There are several ways, for example, crosstool, Debian ARM EABI Port, and other package systems...

From the multiple choices, I decided to use pkgsrc. The main reason is that I can change prefix (location to install) when bootstrap. If prefix is /usr and it's not customizable, the package system may break the original system. Among well-known package systems, pkgsrc seems to be the only one whose prefix can be changed.

I think pkgsrc is a good choice to use on embedded systems like TeraStation, but there are not many pages which answer my purpose. I'm happy if this page is of reference to someone.

Download

• arm-2005q3-2_pkg20080127.tar.gz (61MB, pkg_info_20080127.txt)
• arm-2005q3-2_pkg20080210.tar.gz (62MB, pkg_info_20080210.txt)

This binary was built on "CodeSourcery ARM 2005q3-2". You can check whether your system is the same or not by running /lib/libc.so.6.

 

# /lib/libc.so.6GNU C Library stable release version 2.3.6, by Roland McGrath et al.

Copyright (C) 2005 Free Software Foundation, Inc.

This is free software; see the source for copying conditions.

There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A

PARTICULAR PURPOSE.

Compiled by GNU CC version 3.4.4 (release) (CodeSourcery ARM 2005q3-2).

Compiled on a Linux 2.6.12-rc3 system on 2006-01-26.

...(snip)

If your environment is different from mine, and you want to know how to make this tarball, see Bootstrapping pkgsrc section.

上述大意是：上面两个软件包中的软件是基于“CodeSourcery ARM 2005q3-2”编译产生的，不一定可以在当前系统下直接使用。如果执行：

# /lib/libc.so.6

会显示上述橙色底纹代码，那么上述软件就可以直接在当前系统下使用了。否则，这些软件就无法在当前系统中直接使用，需要使用的话就要重新进行编译。而编译这些软件需要先建立pkgsrc编译环境，Bootstrapping pkgsrc这一节就给出了如何建立pkgsrc编译环境的详细步骤。

Setting up pkgsrc

I assume you can login to TeraStation as root, and your main HDD partition is mounted under /mnt/array1.

# mkdir /mnt/array1/system
# cd /mnt/array1/system
# tar zxf /path/to/arm-2005q3-2_pkg2008MMDD.tar.gz

Make links to /usr.

# ln -s /mnt/array1/system/pkg /usr
# ln -s /mnt/array1/system/pkgsrc /usr

由于/usr是一个已经存在的目录，所以上述两条命令的结果是在/usr目录下分别创建名为pkg和pkgsrc这两个符号链接

Set PATH.

# echo 'export PATH=/usr/pkg/bin:/usr/pkg/sbin:/usr/pkg/gcc34/bin:${PATH}' >> ~/.profile
# source ~/.profile
# hash -r

source命令是在当前bash环境下读取并执行FileName中的命令。source命令与shell scripts的区别是：source是在当前bash环境下执行命令，而scripts是启动一个子shell来执行命令。因此，如果把环境变量的设置命令（或alias等等）写进scripts中，就只会影响子shell,无法改变当前的BASH；如果想通过文件（包含命令列）设置环境变量时，就要用source命令。注意，ehco命令中的 ' 是单引号，不是反单引号。

hash命令：显示、添加或清除哈希表。linux系统下会有一个hash表，当你刚开机时这个hash表为空，每当你执行过一条命令时，hash表会记录下这条命令的路径，就相当于缓存一样。第一次执行命令，shell解释器默认会从PATH路径下寻找该命令的路径，当你第二次使用该命令时，shell解释器首先会查看hash表，没有该命令才会去PATH路径下寻找。hash -r 人作用是清除hash表。

Set ld's library path and run ldconfig.

# mv /etc/ld.so.conf /etc/ld.so.conf.bak
# echo /usr/pkg/lib >> /etc/ld.so.conf
# cat /etc/ld.so.conf.bak >> /etc/ld.so.conf
# ldconfig

Checkout pkgsrc from CVS.

# cd /mnt/array1/system
# CVS_RSH=ssh cvs -d anoncvs@anoncvs.NetBSD.org:/cvsroot checkout pkgsrc

Then, /usr/pkg and /usr/pkgsrc are now ready to use.

Setting up glibc and linux headers

To compile something, headers of glibc and linux are required. Get arm-2005q3-2-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2 and unpack it. There is glibc in arm-none-linux-gnueabi.

# mkdir /mnt/array1/system/arm-2005q3
# cd /mnt/array1/system/arm-2005q3
# wget http://downloads.nas-central.org/ALL_ARM9/DevelopmentTools/CrossToolchains/CodeSourcery/arm-2005q3-2-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2
# bzcat arm-2005q3-2-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2 | tar xf -
# mv arm-none-linux-gnueabi/libc /mnt/array1/system

命令bzcat，他可以查看bz2文件的内容。“tar xf -”中的“-”指的是前一命令“bzcat arm-2005q3-2-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2 ”列出来的文件，事实上就是进行解压。

Make links.

# ln -s /mnt/array1/system/libc/usr/include /usr
# ln -s /mnt/array1/system/libc/etc/rpc /etc

/lib/usr/include这个目录是编译器内定的头文件搜索目录，/libc/etc/rpc是一个文件。

由于我的系统中已经存在/usr/include这个目录，所以第一条命令是无法完成的，会提示文件已经存在。这一命令的目的是指定编译时所需头文件的搜索目录，由于头文件的搜索目录也可以通过设置环境变量C_INCLUDE_PATH来达到，因此，我执行下面的命令来代替上述第一条命令：

# echo 'export C_INCLUDE_PATH=/mnt/disk1/system/libc/usr/include:${C_INCLUDE_PATH} >> /.profile

Set PATH.

# echo 'export PATH=${PATH}:/mnt/array1/system/libc/usr/bin:/mnt/array1/system/libc/sbin' >> ~/.profile
# source ~/.profile
# hash -r

That's it. You can now build software using pkgsrc. Enjoy!

Some Q & A

First, how can I get a root privilege in TeraStation?

I don't write the details. There are several ways, and using firmware update seems to be one of the popular way for it. I could get root by not the firmware update, but a pretty classic way. This is the time to show your stuff. Good luck!

pkgsrc cannot find ncurses.h or libncurses.so when building some software.

Add the following line to the pkgsrc's Makefile.

.include "../../mk/curses.buildlink3.mk"

Notice that this line should be before the line .include "../../mk/bsd.pkg.mk". For example, see /usr/pkgsrc/devel/gtexinfo/Makefile.

pkgsrc reports an error related to iconv when building some software.

That may be because there are two headers of the same name, /usr/include/iconv.h and /usr/pkg/include/iconv.h. Temporarily, you may bypass the error by renaming the one in /usr/include.

# mv /usr/include/iconv.h /usr/include/iconv.h.bak

Where has devel/cvs gone?

It's been moved to devel/scmcvs. See also: HEADS UP: devel/cvs -> devel/scmcvs

I don't want to permit any change of /mnt/array1/system through samba!

Edit /etc/samba/smb.conf and restart samba. Or, stopping samba service is also a good idea if you don't use samba.

I set LANG=ja_JP.UTF-8, but it does not have effect. Why?

That may be because glibc doesn't know the locale. To check effective locales, run locale -a.

# locale -a
C
POSIX

In this case, only C and POSIX locale are available. To make Japanese UTF-8 locale, for example, do:

# mkdir /mnt/array1/system/libc/usr/lib/locale
# ln -s /mnt/array1/system/libc/usr/lib/locale /usr/lib
# ln -s /mnt/array1/system/libc/usr/share/i18n /usr/share
# localedef -i ja_JP -c -f UTF-8 ja_JP.UTF-8

I can ssh from TeraStation to another machine if I'm root. But if I'm not root, I cannot ssh with the error "Host key verification failed". Why?

That may be because the permission of /dev/tty is wrong. To check the permission, run ls -l /dev/tty.

# ls -l /dev/tty
crw-r--r-- 1 root root 5, 0 Dec 28 01:27 /dev/tty

In this case, only root can ssh. To ssh even if not root, add write permission to /dev/tty.

# chmod 666 /dev/tty

Bootstrapping pkgsrc

This section is my memo which explains how to bootstrap pkgsrc. First, make a buildbox and copy the original system.

Reference: chroot Arm Build Environment

# mkdir /mnt/array1/buildbox
# cd /
# for d in `ls |grep -v dev |grep -v proc|grep -v mnt`; do tar cf /mnt/array1/buildbox/$d.tar $d; done
# cd /mnt/array1/buildbox
# for f in *.tar; do tar xf $f; done

其中,d为变量，$d表示取出变量d的值；f为变量，$f表示取出变量f人值。

-v表示取反的意思；是将dev、proc、mnt这3个目录排除的原因是

对于dev目录，

对于proc目录，这里挂载着类型为proc的文件系统

对于mnt目录，这里的子目录挂载着硬盘分区，会存放着大量的用户数据。我们要迁移的只是系统的目录结构，不需要用户数据

分号 ;  的作用相当于换行

特别注意的是：for语句中的两个 ` 不是单引号 ；而是反单引号，切记!

Get native development tools arm-tools-0_16-8.tgz and unpack it.

# cd /mnt/array1/buildbox
# tar zxf /path/to/arm-tools-0_16-8.tgz

arm-tools-0_16-8.tgz是native chaintool。由于要在LSPro上编译自身使用的软件，需要用到native编译工具链，而不是cross编译工具链。

Mount proc and dev to chroot environment.

# mount -t proc none /mnt/array1/buildbox/proc
# mount -o bind /dev /mnt/array1/buildbox/dev

chroot into the buildbox.

# chroot /mnt/array1/buildbox /bin/sh

该命令的作用是：在经过 chroot 后，执行/mnt/array1/buildbox（此时变为根目录’/’)下的/bin/sh.

If /tmp is a symbolic link (in my case, symlink to /mnt/ram), remove it and make new tmp dir.

# rm /tmp
# mkdir /tmp

符号链接/tmp文件已经不需要了，即使保留它也不能作为temp目录使用。

Get pkgsrc and unpack it.

# cd /tmp
# wget ftp://ftp.NetBSD.org/pub/pkgsrc/current/pkgsrc.tar.gz
# cd /usr
# tar zxf /tmp/pkgsrc.tar.gz

Get some additional software from Precompiled Binaries and install to /opt directory.

# cd /tmp
# wget http://downloads.buffalo.nas-central.org/LSPro_ARM9/Distributions/Genlink/Binaries/armv5tejl-softfloat-linux-gnueabi/coreutils-6.10-r1.tbz2
# wget http://downloads.buffalo.nas-central.org/LSPro_ARM9/Distributions/Genlink/Binaries/armv5tejl-softfloat-linux-gnueabi/gawk-3.1.5-r5.tbz2

# wget http://downloads.buffalo.nas-central.org/LSPro_ARM9/Distributions/Genlink/Binaries/armv5tejl-softfloat-linux-gnueabi/grep-2.5.1a-r1.tbz2

# wget http://downloads.buffalo.nas-central.org/LSPro_ARM9/Distributions/Genlink/Binaries/armv5tejl-softfloat-linux-gnueabi/patch-2.5.9-r1.tbz2# mkdir /opt
# cd /opt
# bzcat /tmp/coreutils-6.7-r1.tbz2 | tar xf -
# bzcat /tmp/gawk-3.1.5-r2.tbz2 | tar xf -
# bzcat /tmp/grep-2.5.1a-r1.tbz2 | tar xf -
# bzcat /tmp/patch-2.5.9-r1.tbz2 | tar xf -

要解压bz2压缩包，可以使用tar结合bzcat进行解压缩。其中，coreutils-6.7-1.tbz2这个软件包中包括了许多基本命令，比如ls、mv、rm等等，而系统本来就存在这些命令，这样一来，上述基本命令在系统中就很有可能存在两个版本（事实上几乎就是这样）。这样一来，使用新版本的命令就容易出现问题，因为新命令所依赖的库与原系统中的同名命令是不同的。

coreutils这个软件包包含了软件使用时所需要的目录结构：/usr、/bin、/lib、/etc。

Set PATH.

# export PATH=/opt/bin:/opt/usr/bin:${PATH}
# hash -r

这一步设置之后，当执行命令时，最先在/opt/bin中搜索相应的命令然后执行，接着是/opt/usr/bin，最后才搜索最初PATH变量中的路径。这样一来，执行ls等基本命令时，就会执行/opt/bin或/opt/usr/bin中的相应命令，而不是执行原系统中的相应命令。上面已经提及这样一样可能出现程序无法运行的问题，实践证明也是如何，当执行这一步之后，ls、mv等命令无法运行，错误提示说缺少libacl.so.6等库文件

Rename a linux header which causes a build error.

# mv /usr/include/linux/ext2_fs.h /usr/include/linux/ext2_fs.h.bak

Apply a patch to bootstrap, and finally run bootstrap!

# cd /tmp
# wget http://second.ddo.jp/~okayama/memo/pkgsrc-arm/bootstrap-20080104.patch
# cd /usr/pkgsrc/bootstrap
# patch -p0 < /tmp/bootstrap-20080104.patch
# TR=/opt/bin/tr AWK=/opt/bin/awk GREP=/opt/bin/grep TOUCH=/opt/bin/touch ./bootstrap --prefix=/usr/pkg --pkgdbdir=/usr/pkg/var/db/pkg --varbase=/usr/pkg/var

After bootstrap, you can install software using pkgsrc in the chroot environment. Building GCC is a little complicated task, so I create a separate section to explain it.

按照最后一条命令：

TR=/opt/bin/tr AWK=/opt/bin/awk GREP=/opt/bin/grep TOUCH=/opt/bin/touch ./bootstrap --prefix=/usr/pkg --pkgdbdir=/usr/pkg/var/db/pkg --varbase=/usr/pkg/var

进行编译，无法编译通过，遇到以下问题：

首先，遇到的第一个问题是：编译过程要调用rm命令，但rm命令无法执行，编译中止。

究其原因是：由于先前执行了 export PATH=/opt/bin:/opt/usr/bin:${PATH}这一命令，因此，调用的rm命令事实上就是/opt/bin/rm，我手动执行rm命令时也出现与编译一样的错误提示，于是问题找到了。因此，我恢复PATH变量，做法是：

先执行：

sh-2.05b# echo $PATH
/opt/bin:/opt/usr/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

再复制上述输出结果的“红色字体”部分，这部分就是原来PATH变量的值，然后执行:

sh-2.05b# export PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

sh-2.05b# hash -r

这样一来,PATH变量就恢复原来的值，当执行rm命令时不会去执行/opt/bin/rm，而是执行/bin/rm。

再次执行上述编译命令：

TR=/opt/bin/tr AWK=/opt/bin/awk GREP=/opt/bin/grep TOUCH=/opt/bin/touch ./bootstrap --prefix=/usr/pkg --pkgdbdir=/usr/pkg/var/db/pkg --varbase=/usr/pkg/var

遇到第二个问题：

编译过程要调用touch 命令，但touch命令无法执行，提示的错误信息与前面的出错信息相同，编译中止

于是，我把命令中的“TOUCH=/opt/bin/touch”去掉，即不指定touch命令，即执行：

TR=/opt/bin/tr AWK=/opt/bin/awk GREP=/opt/bin/grep ./bootstrap --prefix=/usr/pkg --pkgdbdir=/usr/pkg/var/db/pkg --varbase=/usr/pkg/var

这样一来，编译中要调用touch命令时，就会执行/bin/touch，而不是/opt/bin/touch。编译过程还是出现问题：

提示touch命令不支持-f选项，编译中止

我手动执行:

sh-2.05b# touch --help

发现/bin/touch只支持-c选项，很明显，/bin/touch这个命令的功能阉割了。怎么办？最先想通过ipkg install touch来安装touch，但发现并没有这个package；于是只能重新编译touch，但是google了很久都找不到touch的源代码。最后在以下的帖子：

http://stackoverflow.com/questions/11528267/where-can-i-find-source-code-for-linux-core-commands

发现我犯了一个认识性错误：

touch是一个独立的软件包，这是错误的。

事实上，touch并不是一个独立的软件包，它是包含在coreutils这个软件包中的一个小工具而已，于是执行ipkg list coreutils，发现能找到软件包，于是ipkg install coreutils（之前我已经安装了ipkg软件系统）。coreutils的软件都安装在/opt/bin上，通过ls命令发现，coreutils软件包中的所有工具名都以coreutils-为前缀，后面再接工具名，比如coreutils-touch；这样的名字显然不方便使用，因此，在安装时同时会建立一个符号链接指向每一个工具，符号链接的名字就是工具的名字，比如对于coreutils-touch这个工具，就有一个指向它的符号链接touch，如下图：

于是执行:

# mv /opt/bin/coreutils-touch /mnt/array1/buildbox/bin/touch

再次执行编译命令：

TR=/opt/bin/tr AWK=/opt/bin/awk GREP=/opt/bin/grep ./bootstrap --prefix=/usr/pkg --pkgdbdir=/usr/pkg/var/db/pkg --varbase=/usr/pkg/var

这一次顺利，编译成功。

How to build GCC for ARM with EABI support

GCC starts to support EABI feature from version 4.1. In pkgsrc, however, there is no lang/gcc41 or lang/gcc42. One solution is to use CodeSourcery's GCC-3.4, to which some patches are applied to support ARM EABI. You can get the source from Download the ARM 2005q3-2 Release.

# cd /tmp
# wget http://www.codesourcery.com/public/gnu_toolchain/arm-none-eabi/arm-2005q3-2-arm-none-eabi.src.tar.bz2
# bzcat arm-2005q3-2-arm-none-eabi.src.tar.bz2 | tar xf -

Unset C_INCLUDE_PATH and LIBRARY_PATH.

# unset C_INCLUDE_PATH
# unset LIBRARY_PATH

Unpack the CodeSourcery's GCC instead of pkgsrc's original gcc34.

# cd /usr/pkgsrc/lang/gcc34
# bmake extract
# cd work
# bzcat /tmp/arm-2005q3-2-arm-none-eabi/gcc-2005q3-2.tar.bz2 | tar xf -
# mv gcc-3.4.6 gcc-3.4.6.orig
# mv gcc-2005q3 gcc-3.4.6
# cd -

Edit Makefile to change CONFIGURE_ARGS. I added following lines:

CONFIGURE_ARGS+= --host=armv5tejl-unknown-linux-gnueabi
CONFIGURE_ARGS+= --target=armv5tejl-unknown-linux-gnueabi
CONFIGURE_ARGS+= --with-float=soft
CONFIGURE_ARGS+= --enable-c99

Then, run bmake. Or, if you want info files, bmake MAKEINFO=/usr/pkg/bin/makeinfo. After the build is finished, run bmake install. In arm-2005q3-2_pkg2008MMDD.tar.gz, this gcc34 is included.

Another solution is to use pkgsrc-wip, which has gcc42.

# cd /usr/pkgsrc
# CVS_RSH=ssh cvs -d anoncvs@anoncvs.netbsd.se:/cvsroot checkout wip

Then you can use wip/gcc42. After gcc42 is installed, set PATH and ld's library path. It's almost the same way in the case of gcc34. In addition, you may have to apply this patch to pkgsrc/mk/compiler/gcc.mk.

Remember to re-install devel/libtool-base after gcc34 or gcc42 is installed.