Re: NUMA setup on Tyan s2895
On Wed, Sep 28, 2005 at 10:51:57AM +0200, Lionel Elie Mamane wrote:
> Still with my Tyan s2895, I have two CPUs and 4GB, made of 8*512MB
> DIMM modules, of RAM. The motherboard's documentation clearly shows
> that four memory slots (two banks) are attached to each CPU. See the
> block diagram on page 2 of
> ftp://ftp.tyan.com/datasheets/d_s2895_100.pdf .
>
> What I find strange is the NUMA setup Linux reports:
>
> Scanning NUMA topology in Northbridge 24
> Number of nodes 2
> Node 0 MemBase 0000000000000000 Limit 000000007fffffff
> Node 1 MemBase 0000000080000000 Limit 000000015fffffff
So physically the memory appears to be mapped 2G at base 0 and 2G at
base 2G. Unless the bios maps memory out of the PCI space, some will be
missing.
> node 1 shift 24 addr ff000000 conflict 0
> Using node hash shift of 25
> Bootmem setup node 0 0000000000000000-000000007fffffff
> Bootmem setup node 1 0000000080000000-000000015fffffff
> On node 0 totalpages: 524287
> DMA zone: 4096 pages, LIFO batch:1
> Normal zone: 520191 pages, LIFO batch:31
> HighMem zone: 0 pages, LIFO batch:1
> On node 1 totalpages: 917503
> DMA zone: 0 pages, LIFO batch:1
> Normal zone: 917503 pages, LIFO batch:31
> HighMem zone: 0 pages, LIFO batch:1
>
> How come it is so different: 917503 (a tad less than 3584MB) pages
> versus 524287 (a tad less than 2GB)! I'd have expected an at least
> roughly half-half split? Err... As I realise things, the addition of
> the two is much more than my RAM. Would the PCI hole / mappings be
> included herein?
I have no idea what those page totals mean. Maybe it is keeping a
running total. And I think 3584 is likely the limit before the PCI
memory space, and any more probably goes after 4G
> More info on system:
>
> master@capsaicin:~$ cat /proc/mtrr
> reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1
> reg01: base=0x100000000 (4096MB), size=1024MB: write-back, count=1
> reg02: base=0x140000000 (5120MB), size= 512MB: write-back, count=1
> reg03: base=0x80000000 (2048MB), size= 512MB: write-back, count=1
So that looks same. 2G at base 0, another 512M at base 2G, 1G at base
4G and another 512M at base 5G. So 2.5G from 0 to 2.5G, a hole of 1.5G
then another 1.5G starting at 4G.
Maybe at this point the kernel has remapped memory to better places than
the bios had.
What does 'free' show you have for memory?
Len Sorensen
Reply to: