bug in memset()

[Nicholas Clark <nick@ccl4.org>]

I can't claim credit for finding this - Robin Watts tracked this one down.

       void *memset(void *s, int c, size_t n);

isn't masking the second parameter to a byte.

Test code:

#include <stdio.h>
#include <string.h>

int main () {
  char fred[16];
  int i = 0;
  memset(fred, 256, sizeof(fred));
  
  while (i < sizeof(fred)) {
    printf ("%d", fred[i]);
    if (++i < sizeof(fred))
      printf (", ");
  }
  puts("");
  return 0;
}

Returns:

debian-arm:  0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1
debian-x86:  0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
FreeBSD-x86: 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

[tested with glibc 2.2.4 on debian-arm]

Only source code I have to hand is 2.1.3, but assuming that
sysdeps/arm/memset.c is unchanged since then:

ENTRY(memset)
        mov     a4, a1
        cmp     a3, $8          @ at least 8 bytes to do?
        blt     2f
        orr     a2, a2, a2, lsl $8
        orr     a2, a2, a2, lsl $16
1:

should become

ENTRY(memset)
        mov     a4, a1
        cmp     a3, $8          @ at least 8 bytes to do?
        blt     2f
        and     a2, a2, #255
        orr     a2, a2, a2, lsl $8
        orr     a2, a2, a2, lsl $16
1:

(untested, suggested by Robin Watts, syntax errors my fault)

should glibc have a (platform independent) regression test to catch this?

Nicholas Clark