The other day I ran `emerge -uD system`, because I felt I might actually loose contact to the cutting edge. I'm now gifted with a whole set on broken applications and libraries due to links that are not existing anymore. While on a ordinary Gentoo one would run revdep-rebuild, the one on Prefix is broken at the time of this writing due to a rewrite of it that hasn't been ported. So this little article is all about porting of revdep-rebuild to MacOSX.

How revdep-rebuild works 

revdep-rebuild works in 5 steps:

1. Finding obj type files on the system (those are basically binary code files)
2. Determining the paths in which libraries reside
3. Finding broken binary code files
4. Mapping broken files to ebuilds
5. Emerging ebuilds

On Linux these things are straight forward. The command `ldd` gives you the list of libraries needed. The reference to a library is only done by its file name (without any directories), thus what ldd prints out is what the file name of the required library is and where it would currently load it from (i.e. it gives the full pathname). Environment variables like LD_LIBRARY_PATH can influence this decision at runtime.

On MacOSX it's slightly more complicated. The closest equivalent of Linux' `ldd` is `otool -L`. The Mach-O files on MacOSX have the absolute pathname of the library it's linking against. They also store a current and compatibility version along with them. The compatibility version of a used library is available both in the library and in the executable using it. When it differs, linkage will fail. Further, MacOSX uses two ways to deal with namespaces. There is the flat and the two-level namespace.

The flat namespace is what is used in Linux as well: when a library exports some symbol, for example a function, it is available throughout the process space in which the dynamic linker has assembled the libraries and the program. That means in particular that any given symbol name can only be once in the namespace of a process. That allows for example the LD_PRELOAD mechanism of the Gentoo sandbox under Linux to intercept calls done to the c library. You just instruct the linker to load a library which implements all file management functions (open, read, mkdir, etc) before you load anything else. This will place the symbol for these functions in the process space and when they are needed, they are taken from that loaded library, not from the system library. When the real library get's loaded later, these symbols are either ignored or the PRELOADed library hid them somehow - here I'm not sure, actually.

The two-level namespace of the MacOSX is different. When applications gets linked with two-level namespace enabled, then for each symbol it is also noted, from which library it has been taken. That allows now two libraries to be linked into the process namespace that have common symbols without anyone of them being ignored. One particular example I'm aware of where this is used is the way the OpenGL libraries in /usr/X11R6 are made (at least on Tiger). The libGL.dylib in /usr/X11R6/lib links against /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib. Both export the same GL functions, but since they are in a different namespace, they don't conflict and one library can actually use the methods of the other.

So, our port of the main_checks method of revdep-rebuild has to accommodate for that. I believe the majority of us open source people are only loosely aware of specifics of operating systems other than Linux and so it comes that many build scripts made of open source projects are only loosely take care of namespaces and current and compatibity versions. So it happens sometimes, that although a previously available symbol gets removed from a library, but the compatibility version stays the same. It happened to me more than once. And now it's getting nasty, because what revdep-rebuild would have to do now is to the same as the dyld would: it has to look whether the linker could put together the process namespace correctly given the instructions in the load commands (`otool -l`).

Which parts of revdep-rebuild are platform specific

Most of the step mentioned earlier are related to portage. Operating system specific are steps 2 and 3. Step 2, because the location of libraries might differ from one system to the other and step 3, because the way libraries are linked into an executable differ. Normally I would expect step 2 to be already covered somewhere in portage, because it's ultimately portage who's exercising some control over where files get installed (by means of providing defaults to the econf in ebuilds for example).

The revdep-rebuild coming with gentoolkit-0.2.4.1-r00.1 has individual methods dealing with each of these steps and the method main_checks implements step 3 for Linux.

How step 3 is performed for Linux

 For each of the earlier mentioned steps a sort of result file is generated. Each step takes some of the files from the previous steps as its input and generates new files to be used in the next step (some descriptions are taken from revdep-rebuild itself):

 For each $FILE in 2_ldpath.rr, main_checks determines what type of file it is looking at.

• If it's a library, it executes `ldd $FILE`. Any file matching the LD_LIBRARY_MASK variable is filtered out. Of the remaining files, any file not matching the SONAME_SEARCH variable is filtered out. From the remaining files, the ones for which ldd determined a full patch name are apparently existing, while the ones marked not found are not. 
• If it's a .la file, it is looked for non existing references in it

Any file from the above referencing missing files is added to 3_broken.rr. Here is how this file looks:

obj /usr/bin/gkbd-indicator-plugins-capplet

obj /usr/bin/gnome-keyboard-properties

Tools available on MacOSX to do stuff

• `otool -L` lists linked libraries and their compatibility versions. It lists also the target that was given as parameter