undefined reference to `__alloca'

[Earnestly]

Hi,

For reference my system has the following versions and environs:

* gcc 7.2.0
* glibc 2.26.9000 (commit 2fac6a6cd5)
* make 4.2.90 (commit baa57d2) [patched, see below]

* CPPFLAGS -D_FORTIFY_SOURCE=2
* CFLAGS -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt
* LDFLAGS -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now

For awhile now I've been having an issue when building make from master
(currently 0c5a9f9) which results in the following error:

...
gcc -isystem /usr/include/guile/2.2 -pthread -Wall -Wextra -Werror -Wwrite-strings -Wshadow -Wpointer-arith -Wdeclaration-after-statement -Wbad-function-cast -Wformat-security -Wtype-limits -Wunused-but-set-parameter -Wlogical-op -Wignored-qualifiers -Wformat-signedness -Wduplicated-cond -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -Wl,--export-dynamic -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o make ar.o arscan.o commands.o default.o dir.o expand.o file.o function.o getopt.o getopt1.o guile.o implicit.o job.o load.o loadapi.o main.o misc.o posixos.o output.o read.o remake.o rule.o signame.o strcache.o variable.o version.o vpath.o hash.o remote-stub.o glob/libglob.a -lguile-2.2 -lgc -ldl
glob/libglob.a(glob.o): In function `glob_in_dir':
glob.c:(.text+0x2d4): undefined reference to `__alloca'
glob.c:(.text+0x450): undefined reference to `__alloca'
glob.c:(.text+0x609): undefined reference to `__alloca'
glob.c:(.text+0x66a): undefined reference to `__alloca'
glob/libglob.a(glob.o): In function `glob':
glob.c:(.text+0x985): undefined reference to `__alloca'
glob/libglob.a(glob.o):glob.c:(.text+0x103c): more undefined references to `__alloca' follow
collect2: error: ld returned 1 exit status

I've worked around this with a quick diff but I'm unsure why this is
necessary or if it breaks any other platforms so I'm not creating it as
a patch:

diff --git a/glob/glob.c b/glob/glob.c
index f3911bc..e4d551e 100644
--- a/glob/glob.c
+++ b/glob/glob.c
@@ -208,8 +208,6 @@ my_realloc (p, n)
#endif /* __GNU_LIBRARY__ || __DJGPP__ */


-#if !defined __alloca && !defined __GNU_LIBRARY__
-
# ifdef __GNUC__
# undef alloca
# define alloca(n) __builtin_alloca (n)
@@ -229,8 +227,6 @@ extern char *alloca ();

# define __alloca alloca

-#endif
-
#ifndef __GNU_LIBRARY__
# define __stat stat
# ifdef STAT_MACROS_BROKEN

Thanks

[Earnestly]

Post by Earnestly
Hi,
* gcc 7.2.0
* glibc 2.26.9000 (commit 2fac6a6cd5)
* make 4.2.90 (commit baa57d2) [patched, see below]
* CPPFLAGS -D_FORTIFY_SOURCE=2
* CFLAGS -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt
* LDFLAGS -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now
For awhile now I've been having an issue when building make from master
...
gcc -isystem /usr/include/guile/2.2 -pthread -Wall -Wextra -Werror -Wwrite-strings -Wshadow -Wpointer-arith -Wdeclaration-after-statement -Wbad-function-cast -Wformat-security -Wtype-limits -Wunused-but-set-parameter -Wlogical-op -Wignored-qualifiers -Wformat-signedness -Wduplicated-cond -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -Wl,--export-dynamic -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o make ar.o arscan.o commands.o default.o dir.o expand.o file.o function.o getopt.o getopt1.o guile.o implicit.o job.o load.o loadapi.o main.o misc.o posixos.o output.o read.o remake.o rule.o signame.o strcache.o variable.o version.o vpath.o hash.o remote-stub.o glob/libglob.a -lguile-2.2 -lgc -ldl
glob.c:(.text+0x2d4): undefined reference to `__alloca'
glob.c:(.text+0x450): undefined reference to `__alloca'
glob.c:(.text+0x609): undefined reference to `__alloca'
glob.c:(.text+0x66a): undefined reference to `__alloca'
glob.c:(.text+0x985): undefined reference to `__alloca'
glob/libglob.a(glob.o):glob.c:(.text+0x103c): more undefined references to `__alloca' follow
collect2: error: ld returned 1 exit status
I've worked around this with a quick diff but I'm unsure why this is
necessary or if it breaks any other platforms so I'm not creating it as
diff --git a/glob/glob.c b/glob/glob.c
index f3911bc..e4d551e 100644
--- a/glob/glob.c
+++ b/glob/glob.c
@@ -208,8 +208,6 @@ my_realloc (p, n)
#endif /* __GNU_LIBRARY__ || __DJGPP__ */
-#if !defined __alloca && !defined __GNU_LIBRARY__
-
# ifdef __GNUC__
# undef alloca
# define alloca(n) __builtin_alloca (n)
@@ -229,8 +227,6 @@ extern char *alloca ();
# define __alloca alloca
-#endif
-
#ifndef __GNU_LIBRARY__
# define __stat stat
# ifdef STAT_MACROS_BROKEN
Thanks

Just to update on this, I have found this patch does work as suspected:

diff --git a/glob/glob.c b/glob/glob.c
index f3911bc..8adbde3 100644
--- a/glob/glob.c
+++ b/glob/glob.c
@@ -208,7 +208,7 @@ my_realloc (p, n)
#endif /* __GNU_LIBRARY__ || __DJGPP__ */


-#if !defined __alloca && !defined __GNU_LIBRARY__
+#if !defined __alloca && defined __GNU_LIBRARY__

# ifdef __GNUC__
# undef alloca

However I am now getting:

glob.c: In function ‘glob’:
glob.c:814:11: warning: implicit declaration of function ‘__stat’; did you mean ‘__xstat’? [-Wimplicit-function-declaration]
: __stat (dirname, &st)) == 0
^~~~~~
__xstat

:(

[Paul Smith]

Thanks for the good info on your version of make and GCC... but... can
you provide details on what operating system you're using? I can't
reproduce this on any of my systems so I'm just curious. Is it just
the newer glibc causing this?

Also curious: why does the build decide to compile/link the version of
glob that comes with make? If you're using glibc then it should use
the one that comes with glibc instead.


You could try grabbing the glob.[ch] / fnmatch.[ch] from the latest
gnulib; I'm not sure how easy it would be to just drop in. The goal of
the current set of changes is to switch over to using gnulib.

[Earnestly]

Post by Paul Smith
Thanks for the good info on your version of make and GCC... but... can
you provide details on what operating system you're using?

I'm currently using Arch Linux currently on kernel 4.13.11 with a lot of
packages built from latest development, such as glibc and make (to find
issues like these).

My environment includes the following:

CPPFLAGS=-D_FORTIFY_SOURCE=2
CFLAGS='-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -g -fvar-tracking-assignments'
LDFLAGS='-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'
MAKEFLAGS=-j4

I believe I've included the versions for everything but if you'd like
any more details, feel free to ask.

Post by Paul Smith
Also curious: why does the build decide to compile/link the version of
glob that comes with make? If you're using glibc then it should use
the one that comes with glibc instead.

Wasn't this specifically done to workaround make using glibc internals
from this patch?

[PATCH] Do not assume glibc glob internals
http://lists.gnu.org/archive/html/bug-make/2017-09/msg00014.html

Post by Paul Smith
You could try grabbing the glob.[ch] / fnmatch.[ch] from the latest
gnulib; I'm not sure how easy it would be to just drop in. The goal of
the current set of changes is to switch over to using gnulib.

I had a go at replacing glob.[ch] and fnmatch.[ch] from the latest
gnulib git://git.savannah.gnu.org/gnulib.git but ran into several
issues.

I ended up having to copy flexmember.h, glob_internal.h,
scratch_buffer.h and malloc/scratch_buffer.h with it all ending in a
many syntax errors:

[nb. Sorry for the long output which is likely just noise]

In file included from ./scratch_buffer.h:4:0,
from glob.c:77:
./malloc/scratch_buffer.h:95:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
libc_hidden_proto (__libc_scratch_buffer_grow)
^~~~~~~~~~~~~~~~~
./malloc/scratch_buffer.h: In function ‘libc_hidden_proto’:
./malloc/scratch_buffer.h:100:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
./malloc/scratch_buffer.h:107:1: error: expected declaration specifiers before ‘libc_hidden_proto’
libc_hidden_proto (__libc_scratch_buffer_grow_preserve)
^~~~~~~~~~~~~~~~~
./malloc/scratch_buffer.h:124:1: error: expected declaration specifiers before ‘libc_hidden_proto’
libc_hidden_proto (__libc_scratch_buffer_set_array_size)
^~~~~~~~~~~~~~~~~
glob.c:79:20: error: storage class specified for parameter ‘next_brace_sub’
static const char *next_brace_sub (const char *begin, int flags) __THROWNL;
^~~~~~~~~~~~~~
glob.c:79:1: warning: ‘__nothrow__’ attribute ignored [-Wattributes]
static const char *next_brace_sub (const char *begin, int flags) __THROWNL;
^~~~~~
glob.c:81:22: error: storage class specified for parameter ‘dirent_type’
typedef uint_fast8_t dirent_type;
^~~~~~~~~~~
glob.c:100:3: error: expected specifier-qualifier-list before ‘dirent_type’
dirent_type type;
^~~~~~~~~~~
glob.c:96:1: warning: empty declaration
struct readdir_result
^~~~~~
glob.c:106:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘readdir_result_type’
readdir_result_type (struct readdir_result d)
^~~~~~~~~~~~~~~~~~~
glob.c:136:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:188:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:199:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:205:12: error: storage class specified for parameter ‘glob_in_dir’
static int glob_in_dir (const char *pattern, const char *directory,
^~~~~~~~~~~
glob.c:208:12: error: storage class specified for parameter ‘prefix_array’
static int prefix_array (const char *prefix, char **array, size_t n) __THROWNL;
^~~~~~~~~~~~
glob.c:208:1: warning: ‘__nothrow__’ attribute ignored [-Wattributes]
static int prefix_array (const char *prefix, char **array, size_t n) __THROWNL;
^~~~~~
glob.c:209:12: error: storage class specified for parameter ‘collated_compare’
static int collated_compare (const void *, const void *) __THROWNL;
^~~~~~~~~~~~~~~~
glob.c:209:1: warning: ‘__nothrow__’ attribute ignored [-Wattributes]
static int collated_compare (const void *, const void *) __THROWNL;
^~~~~~
glob.c:216:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:227:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:263:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:1145:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:1165:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
glob.c:1221:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘{’ token
{
^
In file included from ./scratch_buffer.h:4:0,
from glob.c:77:
./malloc/scratch_buffer.h:95:1: warning: type of ‘gl_scratch_buffer_grow’ defaults to ‘int’ [-Wimplicit-int]
libc_hidden_proto (__libc_scratch_buffer_grow)
^~~~~~~~~~~~~~~~~
glob.c:209:12: error: declaration for parameter ‘collated_compare’ but no such parameter
static int collated_compare (const void *, const void *) __THROWNL;
^~~~~~~~~~~~~~~~
glob.c:208:12: error: declaration for parameter ‘prefix_array’ but no such parameter
static int prefix_array (const char *prefix, char **array, size_t n) __THROWNL;
^~~~~~~~~~~~
glob.c:205:12: error: declaration for parameter ‘glob_in_dir’ but no such parameter
static int glob_in_dir (const char *pattern, const char *directory,
^~~~~~~~~~~
glob.c:81:22: error: declaration for parameter ‘dirent_type’ but no such parameter
typedef uint_fast8_t dirent_type;
^~~~~~~~~~~
glob.c:79:20: error: declaration for parameter ‘next_brace_sub’ but no such parameter
static const char *next_brace_sub (const char *begin, int flags) __THROWNL;
^~~~~~~~~~~~~~
In file included from glob.c:77:0:
./scratch_buffer.h:3:46: error: declaration for parameter ‘gl_scratch_buffer_set_array_size’ but no such parameter
#define __libc_scratch_buffer_set_array_size gl_scratch_buffer_set_array_size
^
./malloc/scratch_buffer.h:122:6: note: in expansion of macro ‘__libc_scratch_buffer_set_array_size’
bool __libc_scratch_buffer_set_array_size (struct scratch_buffer *buffer,
^
./scratch_buffer.h:2:45: error: declaration for parameter ‘gl_scratch_buffer_grow_preserve’ but no such parameter
#define __libc_scratch_buffer_grow_preserve gl_scratch_buffer_grow_preserve
^
./malloc/scratch_buffer.h:106:6: note: in expansion of macro ‘__libc_scratch_buffer_grow_preserve’
bool __libc_scratch_buffer_grow_preserve (struct scratch_buffer *buffer);
^
glob.c:1464:1: error: expected ‘{’ at end of input
}
^

[Paul Smith]

Post by Earnestly

Post by Paul Smith
Also curious: why does the build decide to compile/link the version
of glob that comes with make? If you're using glibc then it should
use the one that comes with glibc instead.

Wasn't this specifically done to workaround make using glibc
internals from this patch?

I'm not sure I understand what you mean. I didn't change anything in
the configure to change the way make detects whether to use the system
glob (from glibc) or the local implementation (for not-glibc systems).

That change was made to make's internal usage of glob so that it works
with the latest glibc without errors: before that change if you tried
to use the system glibc glob and it had the newest glibc, make could
crash.

If you have make's HEAD code then you should be able to use even the
newest glibc glob implementation and it should work.

[Earnestly]

Post by Paul Smith
That change was made to make's internal usage of glob so that it works
with the latest glibc without errors: before that change if you tried
to use the system glibc glob and it had the newest glibc, make could
crash.

Ah yes, I remember the segfault issues with glob now. Sorry for the
misunderstanding.

I've gone ahead and made sure I was on the latest glibc 5f9d19490b24 and
make 1af314465e5.

[I've also found that `make update' is apparently subject to `make -j4'
race conditions resulting in errors like:

wget: unable to resolve host address ‘cvs.savannah.gnu.org’
make: *** [Makefile;1584: get-doc/fdl.texi] Error 4

I'm a little upset that the build system is calling out to remote
resources at all but it does seem to only be during preamble...]

Nevertheless, I'm still getting the same issues:

gcc -isystem /usr/include/guile/2.2 -pthread -Wall -Wextra -Werror -Wwrite-strings -Wshadow -Wpointer-arith -Wdeclaration-after-statement -Wbad-function-cast -Wformat-security -Wtype-limits -Wunused-but-set-parameter -Wlogical-op -Wignored-qualifiers -Wformat-signedness -Wduplicated-cond -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt -Wl,--export-dynamic -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now -o make ar.o arscan.o commands.o default.o dir.o expand.o file.o function.o getopt.o getopt1.o guile.o hash.o implicit.o job.o load.o loadapi.o main.o misc.o output.o read.o remake.o rule.o signame.o strcache.o variable.o version.o vpath.o posixos.o remote-stub.o -lguile-2.2 -lgc glob/libglob.a -ldl
glob/libglob.a(glob.o): In function `glob_in_dir':
glob.c:(.text+0x2d4): undefined reference to `__alloca'
glob.c:(.text+0x450): undefined reference to `__alloca'
glob.c:(.text+0x609): undefined reference to `__alloca'
glob.c:(.text+0x66a): undefined reference to `__alloca'
glob/libglob.a(glob.o): In function `glob':
glob.c:(.text+0x985): undefined reference to `__alloca'
glob/libglob.a(glob.o):glob.c:(.text+0x103c): more undefined references to `__alloca' follow
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile;704: make] Error 1
make[2]: Leaving directory '/home/earnest/build/pkgbuilds/make-git/src/make'
make[1]: *** [Makefile;864: all-recursive] Error 1
make[1]: Leaving directory '/home/earnest/build/pkgbuilds/make-git/src/make'
make: *** [Makefile;551: all] Error 2

I don't really understand why this is happening, how it all relates to
glibc's implementation and whether or not this codepath should even be
used. And if it is supposed to be used, why it's not able to penetrate
the #ifdef tree to the correct conclusion:

# undef alloca
# define alloca(n) __builtin_alloca (n)

Is there perhaps a missing -D flag to that gcc command?

Similarly why __stat isn't being defined as:

# define __stat(fname, buf) __xstat (_STAT_VER, fname, buf)

Does it mean anything if my _GNU_GLOB_INTERFACE_VERSION is '2'? I'm not
sure if it helps but here are the definitions from gnu-versions.h:

https://ptpb.pw/JdWy

similarly here are all the definitions from glob.h:

https://ptpb.pw/wyY7

If it helps at all, here is how my glibc is being built:

https://github.com/Earnestly/pkgbuilds/blob/master/glibc-git/PKGBUILD

Sorry if this isn't too helpful, I can workaround the issues for the
time being so it doesn't appear to be urgent.

[Paul Smith]

Post by Earnestly
[I've also found that `make update' is apparently subject to `make -j4'
wget: unable to resolve host address ‘cvs.savannah.gnu.org’
make: *** [Makefile;1584: get-doc/fdl.texi] Error 4

I don't see any possible way that using -j could cause such an
error...? Unless it's not valid to run wget multiple times in the same
directory (for different files) somehow. Very bizarre.

Post by Earnestly
I'm a little upset that the build system is calling out to remote
resources at all but it does seem to only be during preamble...]

"make update" is designed to be run by maintainers, who have a complete
system installation such as autoconf, automake, gettext, etc. As part
of that process it will download the latest translation files, GNU
standards documents, etc.

I don't believe in checking in either generated (configure, etc.) or
remotely maintained files (translations, etc.) to GNU make's source
control system. Other packages may have different attitudes on this,
but that's how I do it.

A distribution tarball, such as you'd download from the gnu.org sites,
will of course come with all the content you need and will not require
any remote resources; indeed all you need is a C compiler and a shell
(you don't even need a version of make, if you don't have one).
In this try, as well, I can see that you're using the version of glob
that comes with make (by linking glob/libglob.a). That's the part I'm
trying to understand. If your system libc is glibc, then the configure
script should detect this and the build should not compile the local
glob implementation at all.

So, the first question is why is configure not detecting that your
system library supports GNU libc glob?

Post by Earnestly
Does it mean anything if my _GNU_GLOB_INTERFACE_VERSION is '2'?

Ah! Yes, that's the problem. If you examine configure.ac you'll see
this test there:

#define GLOB_INTERFACE_VERSION 1
#if !defined _LIBC && defined __GNU_LIBRARY__ && __GNU_LIBRARY__ > 1
# include <gnu-versions.h>
# if _GNU_GLOB_INTERFACE_VERSION == GLOB_INTERFACE_VERSION

Obviously this will never be true.

One way to fix this would be to change the second #if line above to be:

# if _GNU_GLOB_INTERFACE_VERSION >= GLOB_INTERFACE_VERSION

and see if that works.

Is there anywhere documented what the difference is between version 1
and version 2? Is it just this symlink change? I'm not sure if the
above change is absolutely correct since it means we'll always accept
the latest libc glob interface, which seems to defeat the purpose of
having a version in the first place.

[Earnestly]

Post by Paul Smith
# if _GNU_GLOB_INTERFACE_VERSION >= GLOB_INTERFACE_VERSION
and see if that works.

Yes! This solves the issue and it also solves the __stat issues as
well.

Post by Paul Smith
Is there anywhere documented what the difference is between version 1
and version 2? Is it just this symlink change? I'm not sure if the
above change is absolutely correct since it means we'll always accept
the latest libc glob interface, which seems to defeat the purpose of
having a version in the first place.

It seems to be a relatively recent change according to `git blame'.

https://sourceware.org/git/?p=glibc.git;a=commit;h=ccf970c7a77e86f4f5ef8ecc5e6

Apparently this patch committed in September 2017 to solve this bug
report:

https://sourceware.org/bugzilla/show_bug.cgi?id=22183

Within it someone mentions this beautiful gem:

* Bump _GNU_GLOB_INTERFACE_VERSION to 2 and forcing new GNUmake build to use its internal glob implementation.

Well, that sure backfired quite nicely, haha

[Paul Smith]

Post by Earnestly

Post by Paul Smith
# if _GNU_GLOB_INTERFACE_VERSION >= GLOB_INTERFACE_VERSION
and see if that works.

Yes! This solves the issue and it also solves the __stat issues as
well.

I pushed a change that should allow this to work properly. Note it
came on top of my other directory restructuring change so you will need
to clean your Git workspace after you pull.

[Earnestly]

you will need to clean your Git workspace after you pull.

Thank you, the new commits seem to have resolved all the issues I was
having. And thank you for your patience while trying to diagnose this.

Also no worries about the git workspace; Each build is pulled from a
local bare mirror into local checkout to avoid any kind of contamination
during testing.