Bug#440408: [x86,x64] tests argument instead of return value, fails to compile xmlrpc-c with optimization

["Nemui Ailin" <ailin.nemui@gmail.com>]

Package: gcc-4.1
Version: 4.1.2-15
Severity: important

Compiling xmlrpc-c CVS from <URL:http://xmlrpc-c.sourceforge.net/>
results in a broken xmlrpc library that fails to allocate memory for the
structs.

We have this call to realloc in src/method.c

	static __inline__ void
	reallocProduct(void **      const blockP,
						unsigned int const factor1,
						unsigned int const factor2) {
		
		 if (UINT_MAX / factor2 < factor1)
			  *blockP = NULL;
		 else
			  *blockP = realloc(*blockP, factor1 * factor2);
	}

	#define REALLOCARRAY(arrayName, nElements) \
		 reallocProduct((void **)&arrayName, nElements, sizeof(arrayName[0]))

	REALLOCARRAY(signatureP->argList, minArgCount);
	if (signatureP->argList == NULL) {
	// ...

This results in:

				movl    16(%esi), %eax
				movl    %eax, (%esp)
				call    realloc@PLT
	 .LVL80:
	 .LBE129:
				.loc 1 72 0
				movl    16(%esi), %ecx
				testl   %ecx, %ecx
	 .LBB130:
	 .LBB131:
				.loc 3 57 0
				movl    %eax, 16(%esi)
	 .LBE131:
	 .LBE130:
				.loc 1 72 0
				jne     .L97


On 1 Sep 2007 02:41:57 +0000, Bryan Henderson <bryanh@giraffe-data.com> wrote:

> It calls realloc(), which returns its return value in register %eax.
> It tests 16(%esi) for NULL, and THEN copies %eax to 16(%esi).
>
> So where it thinks it's testing the return value, it's really testing
> the argument of realloc(), which is NULL here, and it thinks realloc()
> returned NULL.
>
>
> I saw a reordering of code similar to this in Gcc 2.96 many years ago.
> It was because Red Hat had decided to ship a major release with an
> untested prerelease version of Gcc.  It caused pain for a couple of
> years;