-
-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compiling libgit2 on glibc 2.17 stat error #255
Comments
TL;DR: Zig prior to You can skip the remainder of this comment, relevant details are covered in the follow-up comment. Original response (suspected relevance to glibc 2.32 vs glibc 2.33)If it helps, this reminded me of when I was looking into a glibc version linking behaviour: #232 (comment) At the bottom of that linked comment is a collapsed section "Dynamic linking differences (resolved)", which has some output from another collapsed section "Reproduction example" earlier in the comment. It's a very simple reproduction that shows how # Same output for 2.17:
$ cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.32
$ readelf -W --version-info --dyn-syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
61: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@GLIBC_2.2.5 (2)
62: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@GLIBC_2.2.5 (2) $ cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.33
$ readelf -W --version-info --dyn-syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
25: 0000000000000000 0 FUNC GLOBAL DEFAULT UND fstat64@GLIBC_2.33 (7)
40: 0000000000000000 0 FUNC GLOBAL DEFAULT UND stat64@GLIBC_2.33 (7) Initial investigationFor Inspecting the broken jj 0.18.0 quickinstall build$ cd /tmp
$ curl -fsSL \
https://github.com/cargo-bins/cargo-quickinstall/releases/download/jj-cli-0.18.0/jj-cli-0.18.0-x86_64-unknown-linux-gnu.tar.gz \
| tar -xz -C /tmp
$ file jj
ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.0.0, with debug_info, not stripped
# The broken glibc build doesn't link openssl (static linked?) or interpreter:
$ patchelf --print-needed jj
libm.so.6
libpthread.so.0
libc.so.6
libdl.so.2
$ patchelf --print-interpreter jj
/lib64/ld-linux-x86-64.so.2
$ readelf -W --version-info --syms jj | grep 'Name: GLIBC' | sed -re 's/.*GLIBC_(.+) Flags.*/\1/g' | sort -t . -k1,1n -k2,2n | tail -n 1
2.17
$ readelf -W --version-info --syms jj | grep stat64
27228: 0000000000000000 0 FILE LOCAL DEFAULT ABS stat64.c
27229: 0000000000000000 0 FILE LOCAL DEFAULT ABS fstat64.c
27232: 0000000000000000 0 FILE LOCAL DEFAULT ABS lstat64.c
27323: 0000000001274b90 49 FUNC WEAK DEFAULT 15 fstat64
27408: 0000000001274b70 23 FUNC WEAK DEFAULT 15 stat64
27434: 0000000001274bd0 26 FUNC WEAK DEFAULT 15 lstat64
33015: 0000000001274b70 23 FUNC GLOBAL DEFAULT 15 __stat64
33018: 0000000001274b90 49 FUNC GLOBAL DEFAULT 15 __fstat64
33020: 0000000001274bd0 26 FUNC GLOBAL DEFAULT 15 __lstat64
# No output (the problem?):
$ strip jj && readelf -W --version-info --syms jj | grep stat64
# NOTE: clang 15 was released in Sep 2022, zig 0.10 was used to build
$ readelf -p .comment jj
String dump of section '.comment':
[ 1] clang version 15.0.7 (https://github.com/ziglang/zig-bootstrap a3a6e85f9ec95b1772f5ace363e46df2f336c6b8)
[ 6a] Linker: LLD 15.0.7
[ 7d] rustc version 1.78.0 (9b00956e5 2024-04-29) Local JJ build# zig 0.13.0 rust 1.81.0 openssl 3.2.2
$ dnf install -y patchelf git gcc rustup zig perl openssl-devel openssl-devel-engine
$ rustup-init -y --profile minimal && source "$HOME/.cargo/env"
$ cargo install cargo-zigbuild
$ git clone --depth 1 https://github.com/martinvonz/jj /tmp/jj && cd /tmp/jj
$ RUSTFLAGS="-L /usr/lib64 -C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.17
$ patchelf --print-needed target/x86_64-unknown-linux-gnu/release/jj
libssl.so.3
libcrypto.so.3
libm.so.6
libpthread.so.0
libc.so.6
libdl.so.2
ld-linux-x86-64.so.2
$ patchelf --print-interpreter target/x86_64-unknown-linux-gnu/release/jj
/lib64/ld-linux-x86-64.so.2
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/jj | grep 'Name: GLIBC' | sed -re 's/.*GLIBC_(.+) Flags.*/\1/g' | sort -t . -k1,1n -k2,2n | tail -n 1
2.17
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/jj | grep stat64
399: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __lxstat64@GLIBC_2.2.5 (2)
400: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@GLIBC_2.2.5 (2)
402: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@GLIBC_2.2.5 (2)
29175: 0000000000000000 0 FILE LOCAL DEFAULT ABS stat64-2.32.c
29176: 00000000010e0680 21 FUNC LOCAL HIDDEN 16 stat64
29177: 0000000000000000 0 FILE LOCAL DEFAULT ABS fstat64-2.32.c
29178: 00000000010e06a0 20 FUNC LOCAL HIDDEN 16 fstat64
29179: 0000000000000000 0 FILE LOCAL DEFAULT ABS lstat64-2.32.c
29180: 00000000010e06c0 21 FUNC LOCAL HIDDEN 16 lstat64
33897: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __lxstat64
33898: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64
33900: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64
# Stripped:
$ strip target/x86_64-unknown-linux-gnu/release/jj
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/jj | grep stat64
402: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __lxstat64@GLIBC_2.2.5 (2)
404: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@GLIBC_2.2.5 (2)
405: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@GLIBC_2.2.5 (2)
# With glibc 2.33 target:
$ RUSTFLAGS="-L /usr/lib64 -C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.33
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/jj | grep stat64
349: 0000000000000000 0 FUNC GLOBAL DEFAULT UND fstat64@GLIBC_2.33 (12)
363: 0000000000000000 0 FUNC GLOBAL DEFAULT UND stat64@GLIBC_2.33 (12)
364: 0000000000000000 0 FUNC GLOBAL DEFAULT UND lstat64@GLIBC_2.33 (12)
33699: 0000000000000000 0 FUNC GLOBAL DEFAULT UND fstat64
33748: 0000000000000000 0 FUNC GLOBAL DEFAULT UND stat64
33749: 0000000000000000 0 FUNC GLOBAL DEFAULT UND lstat64 Minimal reproduction attemptInitial environment setup
[package]
name = "example"
version = "0.1.0"
edition = "2021"
[dependencies]
libc = "0.2.153"
use std::fs::File;
use std::os::unix::io::AsRawFd;
fn main() {
let file = File::open("hello.txt").expect("`hello.txt` should exist");
// Placeholder value:
let mut uid = 42;
unsafe {
let mut stat: libc::stat64 = std::mem::zeroed();
// Returns c_int, 0 is success, -1 is failure:
if libc::fstat64(file.as_raw_fd(), &mut stat) == 0 {
uid = stat.st_uid;
}
}
println!("`hello.txt` is owned by UID: {uid}");
} # Older release before glibc 2.32:
$ docker run --rm -it fedora:31
# Glibc 2.30
$ ldd --version
ldd (GNU libc) 2.30
Copyright (C) 2019 Free Software Foundation, Inc.
# Install deps:
$ dnf install -y gcc pip nano
$ pip install ziglang
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
$ . "$HOME/.cargo/env"
$ cargo install cargo-zigbuild
# Create demo project:
$ mkdir -p /example/src
$ nano /example/Cargo.toml
$ nano /example/src/main.rs Build attempts with zig## glibc 2.17 target
# NOTE: fstat64 has 2.32 version even though build host is glibc 2.30? However, these are only from debug symbols
# UPDATE: This was identified as a zig patch/workaround for proper glibc <= 2.32 support
$ RUSTFLAGS="-C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.17
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
60: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@GLIBC_2.2.5 (2)
61: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@GLIBC_2.2.5 (2)
745: 0000000000000000 0 FILE LOCAL DEFAULT ABS stat64-2.32.c
746: 000000000005a4a0 21 FUNC LOCAL HIDDEN 15 stat64
747: 0000000000000000 0 FILE LOCAL DEFAULT ABS fstat64-2.32.c
748: 000000000005a4c0 20 FUNC LOCAL HIDDEN 15 fstat64
749: 0000000000000000 0 FILE LOCAL DEFAULT ABS lstat64-2.32.c
1045: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64
1046: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep 'Name: GLIBC' | sed -re 's/.*GLIBC_(.+) Flags.*/\1/g' | sort -t . -k1,1n -k2,2n | tail -n 1
2.16
## glibc 2.32 target
# Same as above:
$ RUSTFLAGS="-C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.32
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
60: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@GLIBC_2.2.5 (2)
61: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@GLIBC_2.2.5 (2)
745: 0000000000000000 0 FILE LOCAL DEFAULT ABS stat64-2.32.c
746: 000000000005a4d0 21 FUNC LOCAL HIDDEN 15 stat64
747: 0000000000000000 0 FILE LOCAL DEFAULT ABS fstat64-2.32.c
748: 000000000005a4f0 20 FUNC LOCAL HIDDEN 15 fstat64
749: 0000000000000000 0 FILE LOCAL DEFAULT ABS lstat64-2.32.c
1045: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64
1046: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64
# Min glibc version did go up:
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep 'Name: GLIBC' | sed -re 's/.*GLIBC_(.+) Flags.*/\1/g' | sort -t . -k1,1n -k2,2n | tail -n 1
2.28
# NOTE: Without version should be equivalent to host glibc, thus 2.30, but outputs is same as for 2.32?:
# UPDATE: The 2.32 annotation to debug symbols is zig specific, not related to actual glibc min version requirements
$ RUSTFLAGS="-C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu
## glibc 2.33 target
# Min glibc version 2.33 now required, zig will build correctly for targets of glibc that are newer than the host glibc:
$ RUSTFLAGS="-C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.33
readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
3: 0000000000000000 0 FUNC GLOBAL DEFAULT UND fstat64@GLIBC_2.33 (3)
39: 0000000000000000 0 FUNC GLOBAL DEFAULT UND stat64@GLIBC_2.33 (3)
769: 0000000000000000 0 FUNC GLOBAL DEFAULT UND fstat64
950: 0000000000000000 0 FUNC GLOBAL DEFAULT UND stat64
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep 'Name: GLIBC' | sed -re 's/.*GLIBC_(.+) Flags.*/\1/g' | sort -t . -k1,1n -k2,2n | tail -n 1
2.33
$ readelf -p .comment /example/target/x86_64-unknown-linux-gnu/release/example
String dump of section '.comment':
[ 0] Linker: LLD 18.1.6
[ 13] clang version 18.1.6 (https://github.com/ziglang/zig-bootstrap 98bc6bf4fc4009888d33941daf6b600d20a42a56)
[ 7d] rustc version 1.81.0 (eeb90cda1 2024-09-04) Build attempt without zig# `cargo build` (Rust 1.81, uses fedora 31 provided glibc 2.30):
# Symbols differ a bit vs zig:
$ RUSTFLAGS="-C strip=none" cargo build --release --target x86_64-unknown-linux-gnu
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
20: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@GLIBC_2.2.5 (2)
34: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@GLIBC_2.2.5 (2)
59: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __lxstat64@GLIBC_2.2.5 (2)
587: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __xstat64@@GLIBC_2.2.5
603: 0000000000046120 19 FUNC GLOBAL HIDDEN 13 fstat64
612: 0000000000046100 20 FUNC GLOBAL HIDDEN 13 stat64
658: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __fxstat64@@GLIBC_2.2.5
685: 0000000000046140 20 FUNC GLOBAL HIDDEN 13 lstat64
767: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __lxstat64@@GLIBC_2.2.5
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep 'Name: GLIBC' | sed -re 's/.*GLIBC_(.+) Flags.*/\1/g' | sort -t . -k1,1n -k2,2n | tail -n 1
2.28 With In an attempt to match the Build attempt with zig 0.10.0 + Rust 1.78.0# From the same build Fedora 31 environment above:
$ dnf install -y xz patchelf && mkdir /opt/zig
$ curl -fsSL https://ziglang.org/download/0.10.0/zig-linux-x86_64-0.10.0.tar.xz \
| tar -xJ -C /opt/zig --strip-components=1
$ export PATH="/opt/zig:${PATH}"
# zigbuild prefers python installs before checking for binary installs, uninstall the zig 0.13.0 python package:
$ pip uninstall ziglang
$ rustup toolchain install 1.78.0 && rustup default 1.78.0
$ RUSTFLAGS="-C strip=none" cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.17
# Close enough?:
$ readelf -p .comment target/x86_64-unknown-linux-gnu/release/example
String dump of section '.comment':
[ 1] clang version 15.0.3 ([email protected]:ziglang/zig-bootstrap.git 85033a9aa569b41658404d0e8a5ab887b81d537b)
[ 6a] Linker: LLD 15.0.3
[ 7d] rustc version 1.78.0 (9b00956e5 2024-04-29)
# Now we have output for symbols weakly linked, similar to the broken jj release build, huzzah!:
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
752: 0000000000000000 0 FILE LOCAL DEFAULT ABS stat64.c
753: 0000000000000000 0 FILE LOCAL DEFAULT ABS fstat64.c
756: 0000000000000000 0 FILE LOCAL DEFAULT ABS lstat64.c
781: 000000000006a070 49 FUNC WEAK DEFAULT 15 fstat64
967: 000000000006a050 23 FUNC WEAK DEFAULT 15 stat64
1149: 000000000006a050 23 FUNC GLOBAL DEFAULT 15 __stat64
1153: 000000000006a070 49 FUNC GLOBAL DEFAULT 15 __fstat64
# No symbols output (expected):
$ strip target/x86_64-unknown-linux-gnu/release/example
$ readelf -W --version-info --syms target/x86_64-unknown-linux-gnu/release/example | grep stat64
# However it still functions correctly (unexpected):
# UPDATE: This was my mistake, I was testing the fstat64 call, not the failure condition with incorrect errno returned as detailed later
$ touch hello.txt && chown 77:77 hello.txt
$ target/x86_64-unknown-linux-gnu/release/example
`hello.txt` is owned by UID: 77
$ patchelf --print-needed target/x86_64-unknown-linux-gnu/release/example
libpthread.so.0
libc.so.6
libdl.so.2
ld-linux-x86-64.so.2 So... something else is going on that I'm missing... as I can't reproduce whatever else was going on 🤷♂️ |
Actually... I just realized the bug report wasn't about this zig Proper minimal reproductionThe zig issues provide reproduction in other languages, once I learned it was about
// Reproduction requires building with zig < 0.12.0 and glibc target < 2.33
fn main() {
unsafe {
// Reproduction requires the checked file path to be invalid:
let filepath = std::ffi::CString::new("this_file_does_not_exist").unwrap();
let mut stat: libc::stat64 = std::mem::zeroed();
let ptr = filepath.as_ptr();
// Returns c_int, 0 is success, -1 is failure:
let ret = libc::stat64(ptr, &mut stat);
// This and other stat calls will fail in the same way:
//let ret = libc::fstatat64(libc::AT_FDCWD, ptr, &mut stat as *mut libc::stat64, 0);
let errno = std::io::Error::last_os_error();
// NOTE: `errno.raw_os_error().unwrap();` can provide just the error code itself
match ret {
// Example of working functionality when successful:
0 => {
let uid = stat.st_uid;
println!("The file is owned by UID: {uid}");
},
// The `errno` code will incorrectly be 0 (Success) despite actually failing (`ret == -1`):
_ => println!("Failure!\n| ret: {ret:?}\n| err: {errno:?}")
}
}
} Initial environment setup: $ docker run --rm -it fedora:41
# Add zig releases:
$ mkdir /opt/zig-11 /opt/zig-12
$ curl -fsSL https://ziglang.org/download/0.11.0/zig-linux-x86_64-0.11.0.tar.xz \
| tar -xJ -C /opt/zig-11 --strip-components=1
$ curl -fsSL https://ziglang.org/download/0.12.0/zig-linux-x86_64-0.12.0.tar.xz \
| tar -xJ -C /opt/zig-12 --strip-components=1
# Add rust:
$ dnf install -y gcc rustup nano
$ rustup-init -y --profile minimal && source "$HOME/.cargo/env"
$ cargo install cargo-zigbuild
# Prep project:
$ cargo init /tmp/example && cd /tmp/example
$ cargo add libc
# Replace main.rs with above rust snippet:
$ rm -f src/main.rs && nano src/main.rs Here are the results: # `ret -1` is expected, but the `err` code should be `2`, not `0` as it wasn't actually successful:
$ CARGO_ZIGBUILD_ZIG_PATH=/opt/zig-11/zig cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.32
$ target/x86_64-unknown-linux-gnu/release/example
Failure!
| ret: -1
| err: Os { code: 0, kind: Uncategorized, message: "Success" }
# Correct for glibc 2.33+ where the fstat symbol changed to real function calls:
$ rm -rf target
$ CARGO_ZIGBUILD_ZIG_PATH=/opt/zig-11/zig cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.33
Failure!
| ret: -1
| err: Os { code: 2, kind: NotFound, message: "No such file or directory" }
# glibc 2.32 is working correctly from zig 0.12.0+:
$ rm -rf target
$ CARGO_ZIGBUILD_ZIG_PATH=/opt/zig-12/zig cargo zigbuild --release --target x86_64-unknown-linux-gnu.2.32
Failure!
| ret: -1
| err: Os { code: 2, kind: NotFound, message: "No such file or directory" } ReferencesRelease dates for context to the October 2020 glibc change referenced below:
Quoted reference links for technical background (glibc 2.32 below issue with fstat and errno + zig patch)
A related issue also cites
Associated PR fix adds some docs:
Which also explains where the
And another reference I came across while looking into the glibc 2.33 change (before landing on that zig issue above):
|
Related: jj-vcs/jj#3844
According to this comment by @yuja
which might. be the problem.
The text was updated successfully, but these errors were encountered: