Re: NUMA setup on Tyan s2895
On Wed, Sep 28, 2005 at 10:28:17AM -0400, Lennart Sorensen wrote:
> On Wed, Sep 28, 2005 at 10:51:57AM +0200, Lionel Elie Mamane wrote:
>> Still with my Tyan s2895, I have two CPUs and 4GB, made of 8*512MB
>> DIMM modules, of RAM. The motherboard's documentation clearly shows
>> that four memory slots (two banks) are attached to each CPU. See the
>> block diagram on page 2 of
>> ftp://ftp.tyan.com/datasheets/d_s2895_100.pdf .
>> What I find strange is the NUMA setup Linux reports:
>> Scanning NUMA topology in Northbridge 24
>> Number of nodes 2
>> Node 0 MemBase 0000000000000000 Limit 000000007fffffff
>> Node 1 MemBase 0000000080000000 Limit 000000015fffffff
> So physically the memory appears to be mapped 2G at base 0 and 2G at
> base 2G.
Nah, more than that on node 1: from 2GiB to 5.5GiB. 15fffffff is
5.5Gi - 1.
>> node 1 shift 24 addr ff000000 conflict 0
>> Using node hash shift of 25
You know what these two lines mean?
>> On node 0 totalpages: 524287
>> DMA zone: 4096 pages, LIFO batch:1
>> Normal zone: 520191 pages, LIFO batch:31
>> HighMem zone: 0 pages, LIFO batch:1
>> On node 1 totalpages: 917503
>> DMA zone: 0 pages, LIFO batch:1
>> Normal zone: 917503 pages, LIFO batch:31
>> HighMem zone: 0 pages, LIFO batch:1
>> How come it is so different: 917503 (a tad less than 3584MB) pages
>> versus 524287 (a tad less than 2GB)! I'd have expected an at least
>> roughly half-half split? Err... As I realise things, the addition of
>> the two is much more than my RAM. Would the PCI hole / mappings be
>> included herein?
> I have no idea what those page totals mean. Maybe it is keeping a
> running total. And I think 3584 is likely the limit before the PCI
> memory space, and any more probably goes after 4G
No, without PCI memory remap, I have only about 2700MiB usable. I
think I've read that PCIe takes 700MiB of address space on its own or
something like that.
>> More info on system:
>> master@capsaicin:~$ cat /proc/mtrr
>> reg00: base=0x00000000 ( 0MB), size=2048MB: write-back, count=1
>> reg01: base=0x100000000 (4096MB), size=1024MB: write-back, count=1
>> reg02: base=0x140000000 (5120MB), size= 512MB: write-back, count=1
>> reg03: base=0x80000000 (2048MB), size= 512MB: write-back, count=1
> So that looks same. 2G at base 0, another 512M at base 2G, 1G at base
> 4G and another 512M at base 5G. So 2.5G from 0 to 2.5G, a hole of 1.5G
> then another 1.5G starting at 4G.
> Maybe at this point the kernel has remapped memory to better places
> than the bios had.
Can it do that? I thought only the bios could decide of these
remappings.
> What does 'free' show you have for memory?
4029228 kiB = 3934MiB
--
Lionel
Reply to: