Cross-compilation patches, Buildroot and OpenEmbedded
I have been working on getting gobject-introspection (GOI) into Buildroot mainline, and last week the patch set was finally merged!
This patch set took:
- Two and a half years.
- 17 patch revisions.
- Two trips to Europe
Now I would like to give back.
Because GOI must first run and scan a binary when creating .gir files, cross-compilation is quite tricky! Mainly because if the binary is cross-compiled, you would not typically be able to run or scan a cross-compiled binary from the host system.
Because of this significant limitation, both Buildroot and Openembedded have decided to use several wrappers of which runs the native scanner using qemu to create the binaries.
Buildroot and OE provide wrappers that are specific to each project. I am sure it's possible to make them a bit more generic and add them to GOI master, but that isn't the purpose of this report. As such, I would like to first focus on the patches needed to cross-compile and run GOI.
The patch series in Buildroot takes several packages from OpenEmbedded (OE) (with credit), which I have not been able to find here, even though some of them say "Upstream-Status: Pending."
I was hoping to discuss as to whether or not any of the patches provided in Buildroot would be acceptable to the upstream project. The eight patches, what they do, and their respective links are as follows:
Revert an incomplete upstream attempt at cross-compile support This patch is straight forward: Remove the unused GI_CROSS_LAUNCHER option.
This patch adds 4 options to meson_options.txt:
- Use gobject introspection tools installed in the host system. Instead of hardcoding paths to the host system, it's best in a cross-compilation project to use a host path instead.
- Use a wrapper to run gi-compiler and binaries produced by gi-scanner
- Use an ldd wrapper instead of system ldd command in giscanner OE was nice enough to create a prelink-rtld program that is useful for cross-compiling. Instead of hard coding /bin/ld, check if there is an ldd wrapper and use that instead.
- enable-introspection-data (OE specific, although Buildroot may add this option in the future).
- Build introspection data (.gir and .typelib files) in addition to the library and tools. Some packages require just the library and do not require .gir files. In an embedded scenario, it may be beneficial to not build .gir or .typelib files if they aren't necessary.
- pkg-config-sysroot-path (OE specific) Specify a sysroot path to prepend to pkg-config output
- If there is no introspection data on the host, meson throws the error:
"Unknown variable "typelibs." Because tests are not required nor desired when cross-compiling, removing the sub dir tests fixes this issue.Instead of fully disabling tests as this is not going to work for upstream, perhaps this could be changed to a simple option?
- With this option, giscanner uses a wrapper executable to run binaries it's producing, instead of running them directly.
- This option is useful in cross-compile environments where the system's ldd command does not work on binaries built for a different architecture.
- By default LD_LIBRARY_PATH is set to the list of target library paths; this default breaks down in the cross-compilation environment, as we need to run a native emulation wrapper rather than the target binary itself. This patch allows exporting those paths to a different environment variable, which can be picked up and used by the wrapper.
- Some packages (such as gstreamer1's base plugins) don't pass all of the required paths by default to g-ir-scanner. Both OE and Buildroot use the environment variable "GIR_EXTRA_LIBS_PATH" to fix this shortcoming, the g-ir-scanner wrapper passes GIR_EXTRA_LIBS_PATH to the g-ir-scanner utility via the
--lib-dirs-envvarvariable. However, as it currently stands, ccompiler.py does not add these paths via rpath-link, so this patch adds that capability.
- prelink-rtld, which we use instead of ldd returns 127 when it can't find a library. It is not an error per se, but it breaks subprocess.check_output().
Currently, this patch is inappropriate for upstream, which is understandable. Perhaps instead of unconditionally ignoring the return value, a new argument that would allow this behavior is more appropriate?
Let me know what you guys think! I appreciate all the hard work you have put in to make a great piece of software, and I am super excited that Buildroot can finally use GOI to update several packages!