The Answer Gang

/usr/src/linux symlink considered harmful

some right answers become wrong, eventually.

While it's normally the practice here to state who's asking and who's answering, on this issue, that itself was a hot topic.

While answering "A rather unique query" last month, Mike dispensed some common wisdom... which has, it seems, become unwise, at least unless you are exceedingly careful of the context.

Thanks to Michal Jaegermann from the kernel list for bringing it to more serious attention (can we say flame war here in the land of curmudgeons? knew ya could), everyone from the Gang who hopped in, and, especially, Breen Mullins and Dan Wilder for providing clearer detail into the nature of the problem. And my apologies to anyone who feels a need to get grumpy that I ruined all concept of timeline in this thread, in favor of clarity to the readers.

Distro vendors and anyone who tends to build themselves kernels of different vintages (mixing 2.0 with 2.2, etc) should pay special attention.

[Mike] This is the normal Linux convention. Actually, you can place your build tree anywhere, but you should make /usr/src/linux a symlink to it so that the compiler will find the include files.

[Michal] Actually no, you SHOULDN'T!! Please do not spread an incorrect information in TAG or Linus will come and will haunt you for the rest of your lives.

I'll spare the readership the flame war on his flight into hyperbole. -- Heather

[Mike] (Is this [headers in /usr/src/linux/include/] still required now that glibc has its own kernel headers?)

[Michal] Headers in /usr/include/linux are "private" but these should be those headers which were used in a compilation of your libraries (notably glibc) and hacking around the with a link in /usr/src is a mistake as Linus tried to explain many times - sometimes quite forcibly. Headers used in a kernel compilation are NOT searched for in subdirectories of /usr/src/linux but are specific to a kernel version and can be drastically different between different versions, or at least you do not have any guarantees that they are not. If you happen to have sources to one of 2.2 kernels and one of 2.4 then /usr/src/linux link is supposed to mean what?

Good question... building a kernel vs. building other things, this link does or doesn't exist or is real instead of a link; some other link named "build" in the modules subtree does or doesn't exist, and if it does, what's a good link look like? [hot topic compression algorithm, kinda lossy but hopefully sufficient.]

[Mike] OK tag, what do you think? Is it time to stop linking /usr/src/linux to /usr/src/linux-VERSION ?

[Michal] AFAIK this time was at least like two years ago. Some things just have a big inertia.

[Breen] That does seem to be the official answer.

From the 2.4.0 release, in linux/README:

INSTALLING the kernel:

- If you install the full sources, put the kernel tarball in a
directory where you have permissions (eg. your home directory) and
unpack it:

gzip -cd linux-2.4.XX.tar.gz | tar xvf -

Replace "XX" with the version number of the latest kernel.

Do NOT use the /usr/src/linux area! This area has a (usually
incomplete) set of kernel headers that are used by the library header
files.  They should match the library, and not get messed up by
whatever the kernel-du-jour happens to be.

[Dan] Yes. No. Maybe.

Many userland programs need (or think they need) kernel includes. They usually get these through /usr/include/asm and /usr/include/linux, which are often themselves symlinks:

/usr/include/asm -> /usr/src/linux/include/asm
/usr/include/linux -> /usr/src/linux/include/linux

Perhaps this is wrong, and either

• The userland stuff shouldn't be relying on kernel includes, or
• /usr/include/asm and /usr/include/linux should not be symlinks, but instead should be directories containing stable descriptions that won't change with kernel version, (!?!) of kernel-ish things the userland needs.

In the one case, the application developers are at fault, and should be told to mend their ways. In the other, blame the distributions.

[Michal] Debian does not provide bad links for a long time. Red Hat also recently caught itself on a mistake. I do not know details of other distributions but if they not fixed that already then likely they will soon.

Some source packages indeed search for /usr/src/linux for configuration purposes. If this is not just a default which could, and should, be adjusted then they are simply wrong. Current 2.2 kernels will install 'build' link in its /lib/modules subdirectory to indicate where sources for a given version are/were. This is not a foolproof either but still better than alternatives.

[Dan] In either case the poor end user can't be faulted for tolerating those links into the kernel source. The conscientious user might be praised for complaining to the program maintainer.

Recent application source trees exhibiting things like

#include <linux/...

include (just for starters) autofs, cdrecord, DOSEMU, gnupg, kde, mysql, ntp, pgp, procps, python, samba, util-linux, wu-ftpd.

Perhaps we need a userland API? oh yeah, right, got that, called glibc. Sigh. I think we can grant that procps has to know what /proc is really up to, though.

[Dan] Or maybe we just say there exist, though perhaps there should not, applications that depend on kernel version. And they pick that up through symlinks into the kernel source tree.

[Mike] None of my systems have ever had a /usr/src/linux directory at all. (Otherwise, I would not have been able to make the symlink without erasing stuff first.)

So the thread at the end of this month still carries some questions:

• glibc does or does not have its own set of linux-kernel-like headers?
• Is the new kernel different enough that those are now a trap lying in wait for a newly-built-from-source userland application?
• Is procps (for example) really a userland application, in that context?
• If you lack this mystic version of headers where would you get them, obviously (?) real kernel sources are not the right place?
• If distros are evading /usr/src/linux entirely how will apps like procps build correctly at all if we want to get the source?
• what sort of specialty symptoms happen from screwing this up, so that we of the Answer Gang and our faithful readers can put some spark of sense in our toolkit. "You say it's doing what? Hmmm are you sure you don't have a bogus /usr/src/linux?"

[Dan] What's a poor user to do?

At the very least, folks, while you're building new kernels out there, here's a few safety tips:

• when you put your kernel in place don't forget to put the matching System.map in with it. Don't toast it when you toss the source tree out for some space. That's your real symbol table, /proc/ksyms probably is a poor substitute.
• keep constantly up to date on modutils, it tries to be good across the 2.x.x range of kernels
• when you tweak /etc/lilo.conf don't forget to actually run /sbin/lilo... before you reboot!
• after you boot on the new kernel, run
  depmod -a -F /path/to/correct/System.map
  so you really know that your modules aren't missing any symbols.
• backups are good things. Always keep your old kernel binary and its modules around for a little while.

If anyone has some good checklist points to look out for when compiling userland apps, or a clearer description of what's going on in glibc's tiny brain when it reaches for "headers", let us know!

This page edited and maintained by the Editors of Linux Gazette Copyright © 2001
Published in issue 62 of Linux Gazette February 2001

HTML script maintained by of Starshine Technical Services, http://www.starshine.org/

1   2   3   4   5   6   7