Re: Re: f-cpu and Debian
hello everybody,
and forgive me if some of my writings are
a bit obscure (you know what sleep deprivation
can do on any brain, so imagine with mine)
>De: Russell Coker
>A: whygee
>
>On Wed, 9 Oct 2002 00:36, whygee wrote:
>> I have also seen that the Hurd kernel has been ported
>> on Debian (or the contrary, who knows) and a FreeBSD
>> support team started working on using this other kernel.
>
>Debian supports Hurd, and support for FreeBSD, NetBSD,
> and OpenBSD are all being worked on.
i don't know everything but it looks good anyway.
I am also in contact with the GIPI Linux project
and the french Linux and Hurd community (among other
things). But i'm happy to speak to other people as well :-)
>> - verify all the endianness problems in all packages
>> (F-CPU is little endian but can work with big endian
>> data : this might be a bit complex for network codes
>> that are used to single-endian CPUs)
>> - verify all the dependencies against architectural
>> specific code (inlines, PCI stuffs, hardcoded addresses...)
>
>Debian currently runs on i386, SPARC, Alpha,
> M68k, PPC, ARM, MIPS, HPPA, IA64, and S/390.
i already know about this (more or less).
my point is : the F-CPU ISA (Instruction Set Architecture)
can behave more or less like a MIPS or 680x0 (to some
extent) but, because it is then underused, it would
be an overkill. I know that one part of the issue is
with GCC (and it will probably be very long before
F-CPU is correctly supported) but the other part is
in the algorithms. The network example is a good one :
F-CPU can do dual-endian data reads and writes, and
it's probably going to confuse both the compiler
and the code because the machine endian can be changed
for each instruction. Ok, F-CPU is relatively fast
but it's a big waste anyway...
> I think that for the majority of software (and all the really
>important stuff) the endianness issues and architecture dependencies have
>been sorted out.
It will probably depend on GCC, too, and how it can understand
the ISA. Now the problem is that neither C or GCC are made
for handling dual-endian data. maybe a new data characteriser
must be created, such as in
unsigned long big_endian IP_address;
?
>> - verify that F-CPU idioms don't collide with existing ones
>> (for example, the "FCPU_*" name space should be unused
>> in all preprocessing directives and C/++ libraries)
>
>Why does it need this?
a namespace for f-cpu is useful for some libraries
that could be written specifically for this architecture.
But there can be a lot of upcoming reasons that can't
be planned, so i prefer to make sure we won't bumpt into
a problem at any time.
>Why not __fcpu__?
well, it is not determined for good, now, so
we can still discuss about this. If a proposition
is logical and realistic, it's a good candidate :-)
>> - examine all the bootstrap-related problems (F-CPU
>> has no BIOS, but rather something that looks like
>> the monitors of the SPARC or ALPHA computers)
>
>Debian runs on SPARC and Alpha, shouldn't be any Debian
> issues there. Of course someone has to write a boot loader...
i have already started investigating that issue.
as a lot of others did, but in a PC-centric way...
F-CPU is going to be used for parallel computing,
mainly because the first silicons (if ever) will be
too slow for what people expect (a first figure is
to reach 200MHz and a theoretic maximum of 400MHz
for FC0) when compared to current CPUs (xxx GHz).
A "BIOS" like this of a PC is definitely *NO* because
it drags a lot of compatibility and scalability issues.
So i started doing a romfs-like utility that can
help anybody boot the image one wants.
The FLASH contains the f-romfs functions,
a short bootstrap code that tests and setups
the SDRAM, and then uses f-romfs to load an
image in memory and execute it there. This image
can then load other "files" with the f-romfs functions.
>From there, i don't care much what kernel boots
(even though i have some personal ideas).
This is important if people want to "play" with
F-CPU, develop and debug small funny or embedded
sofware. A simple board with F-CPU, a FLASH and
SDRAM is enough (i'm currently investigating
the use of the JTAG interface, usually used for
PCB testing purpose, for communicating with a small
virtual console). This can even be completely
simulated, in C or VHDL (at the gate level) in
a single PC. Then new interfaces can be developped
(serial lines, keyboard/mouse/usb, network....)
but the JTAG "console" will still remain useful
when a driver hangs...
>Personally I'd rather see LinuxBIOS support and
> have the kernel and initrd in the flash memory.
yup, that's the logical choice. However, you can
still develop a small program (probably in asm
but that's not too complex) that can do "multi-boot"
if you want to select an image. Using fromfs, this
is possible.
>> - create and verify new data types that are adapted to the
>> F-CPU principles : UMAX, SMAX, FPTR ...
>> (A certain F-CPU implementation instanciates a small
>> part of the F-CPU specification, which states that
>> the registers have a "default" width of 64 bits at
>> program startup, but the hardware can be larger ...)
>> [ read the F-CPU manual for more explanations ]
>
>Initially we'll do without that support.
that's what i guess.
> Most programs don't need such things, there's a lot
> of work to get it basically booting, we can go back and
>add F-CPU specific things later.
yup. However, one of the challenges is that F-CPU must
be fast from the start. People would be disapointed if
it doesn't seem competitive enough. One way to be fast
is to superpipeline the core (and it's done). However,
if the SW doesn't exploit the features, it will be
rather disapointing, and this can lead some people to
think that F-CPU is not efficient...
>> [though pointers should not have a size, but be defined
>> just as : "up to as wide as a register"]
>
>This sounds difficult.
As long as the kernel and the compiler are correctly
setup and in sync, there should be no problem with
correclty written code (ie that does not expect pointers
to be 4-bytes long). The policy of clearing the unused MSB
in integer computations should make pointers safe across
many F-CPU implementations. However, pointer arithmetics
can become nasty as wider registers appear : unaligned
accesses are not allowed and if this is a problem for
porting code from 32 to 64-bit data, it can become a
nightmare when the architecture says that loaded
and stored words can be as wide as any power of two
(> 8 bits) but must be aligned. This can pop up in
mallocs, in stacks, as well as any other random location...
the price of performance...
>> A lot of these problems are linked to the compiler
>> and we have not ported GCC decently yet. The other parts
>> come from the way developers program .... And this is
>> where the Debian project is of critical usefulness.
>> Any company or structure could simply use GCC to recompile
>> the existing applications and port their SW base to F-CPU.
>> However F-CPU will be underused, particularly because most
>> computers now are 32-bit ...
>
>Actually most applications don't do anything that
> requires more than 32bit math.
32-bit math, yes. 64-bit ints are often discussed
but are slowly accepted as file sizes and memory
widths increase. However the whole point of having
even wider registers is not to compute 256-bit ints,
but more ints at a time. F-CPU evaluation by other
teams members showed that it is both more efficient
and easier to increase the register width than to
execute more instructions per cycle. And when it is
well programmed, it can even need no recompilation :-)
Currently, the first F-CPU core ("FC0") is very simple
and reaches its best performance in numerical code
(graphics, crypto, communications, DSP...). Performance
scales "almost" linearly with the register width
(depending on the compiler's ability to vector the
code and it's still subject to Amdahl's law, of course).
So one way to keep the impression of "speed" is to
avoid "bloatware" as much as possible... :-)
> None of the programs that I am currently working on do it.
>
>64bit and SIMD should be good for SSL, GPG, bzip2, RAID-5, ray tracing, and
>DVD/AVI playing.
i'm not sure for ray-tracing and bzip2 (there is no
scatter/gather instruction yet) but other applications
will LOVE it. just think about this as a "super-SSE2" :-)
we can add : playing chess or network routing.
A cool "router" could even be built with a network of
F-CPUs (around a crossbar) and with dedicated Ethernet
hardware on each chip. To add more ports and power,
add more F-CPUs. Same goes for other applications.
> While these are among the greatest CPU hogs they are not
>the most common tasks to use a computer for.
unless your computer is setup as a firewall and/or
multimedia streaming server :-) and if it is sitting
idle, it can achive nice SETI@home or RC5 scores.
>> F-CPU has more registers, much wider registers, more adressing
>> range, and SIMD capabilities since the beginning, compared to
>> a SPARC, MIPS-32 or x86 computer. Maybe most of the work can
>> reuse MIPS-64 or ALPHA-specific code. But this will not be
>> enough to exploit the SIMD operators and the potentially larger
>> registers (a simple line in a configuration file is enough
>> to make a 128-bit wide computer, or 256-bit, or any other power
>> of two...).
>
>What do you mean here by "a simple line in a configuration file"? Are you
>referring to configuration for a CPU emulator?
(ooops sorry)
i meant, in the VHDL source code.
in fact, this code is pre-processed by m4/make
and register widths are computed and adjusted in the
whole F-CPU source tree (comprising currently C, bash
and VHDL code). Change one file in the m4 definition file,
type "make" and it builds a whole new complete CPU
with as wide registers as you want.
> If so where can I find a copy
>of the FCPU emulator? Can I package it for Debian?
F-CPU "snapshots" are becoming rare these days
(i had to rest this summer after LSM@Bordeaux)
but they are stored there : http://f-cpu.seul.org/new/
(my latest one is http://f-cpu.seul.org/new/snapshot_yg_29_07_2002tbz
but i haven't integrated Jaap's latest simulator updates).
Making a Debian package is not a problem in the
principle. But in practice it is highly "beta",
it is 25 or 30% complete, few people are actively
working on this, and the C code is tightly bound
to the VHDL code. So either the Debian community
can find several "supermen", or you'll have to wait
a bit... (which is probably best).
>> Recompilation is not enough to speed things up,
>> some critical algorithms and coding practices must be reviewed.
>> For example, glibc, mpeg and jpeg libraries, X (though i'd
>> rather stick with DirectFB and XDirectFB...), network, etc.
>> The kernel side will be a whole other project on its own
>> and AFAIK it's not the main subject of concern for Debian.
>
>Debian developers include people who work on upstream code development for
>almost type of Linux software. While the Debian project is not focussed on
>upstream development, to fulfil our aim of developing the best possible
>packages for Debian we often need to write code for contribution upstream.
>
>> So if Debian can do something, the first is to inspect all
>> the existing packages for conflicts or problems, in order
>> to make a "F-CPU white list". To do this, there should be
>> a coding advice list, but the above gives you an good idea.
>> I am here if you need more help (though my paid job doesn't
>> leave much time).
>
>Apart from the unlikely issue of programs using the macro name FCPU_ name
>space (which is easy for an autobuilder to find and for us to fix once we get
>things going properly)
great !
> I don't think that there's anything that needs to be
>done. Anything that could be done by static analysis has already been done
>(and more) through the other ports.
i guess so. but sticking to "compatibility mode"
is a bit sad because it doesn't force people to think
more about the platform, the coding practices ....
think of it a while : x86 never succeeded implanting
MMX and most other extensions. One reason is that they
are hard to use and the newer extensions used different
opcodes. But in the end, C and ia32 remains. OTOH if F-CPU
tries to force people to rewrite all the code, it will
never succeed. It is friendly with most data types for
this reason (among others). But if people don't know that
there are orthogonal features that can help their code
behave better, they will not use them and some guys
will want to remove these features "because it's not used"...
And the performance will get back to what you'd expect from
a classical MIPS.
>> When the "white list" is ready, the remaining packages
>> can be examined and then modified more or less deeply,
>> on a case by case basis. But this work will require that
>> a working GCC port and a whole F-CPU simulator is ready
>> (not yet, it's still going to take a long while).
>
>Do you have anything that's ready to be packaged for Debian?
i wouldn't call our code "packageable". the structure might
change at any time (though much efforts are done to avoid this
but this is what makes this eventuality more likely to happen
anyway). Most tool are not complete. And only a few units
are more or less ready (and only "execution units", not
"control units"). However, much effort is done on testability,
scalability, easing customization, scripting for regression
tests, etc...
>It doesn't have to be fully operational,
it is not "operational" at all :-)
we're mostly trying to setup the ground level
with all the scripts and the coding practice
(it gets intricate because a lot of skills are
required).
> just to a stage where
> it can demonstrate the concept and
> encourage people to try it out.
Currently, Jaap Stolk has made a part of the
"execution pipeline" with a few units, register set
and some scheduling stuffs. However i won't encourage
people to "use" it because of several problems :
- it's not "scalable". registers are fixed-size 64-bits
because Jaap uses C's long long ints. With VHDL,
a bit vector of any width is possible (and used).
- each C unit must remain in sync with the corresponding
VHDL version. This must keep the whole thing from
breaking and forking. Each unit should (ideally)
be written in C AND VHDL and tested with the same
test patterns. However, some units are not yet
recoded in C, and some have been written by Jaap in C...
- and to make it clearer : the F-CPU sources depend a lot
on the availability of a VHDL compiler/simulator.
2 are currently working and available "for free" on the Net.
i have been provided with 2 other proprietary tools by
companies in order to ensure portability. If someone wants to
use the F-CPU source, he'll have to understand and install
one of these tools, and (believe me) it's not always
straight-forward. I have written an extensive HOWTO
on this matter, but you know that nobody reads the fucking
manual...
>If I could package a CPU simulator that can run "hello world" in the FCPU
>instruction set then it would excite many people, some of whom would be
>likely to have the skill and time to contribute code.
all the F-CPU "lurkers" are waiting for that but none
have done the necessary steps forward... and when they
speak, we drown under some trolls of some sort or another...
However, the few people who actively contribute (i'm fortunately
not alone) have solved some highly important problems
that go beyong writing "hi".
- what format to use
- what testing methodologies to use
- what tools to use
- how the damns stuff must be structured
but when all the "fun" of "discussing" these problems
had gone, nobody was there to implement what was
agreed/decided so painfully...
now the whole dirty work remains.
- finishing ALL the execution units
- defining the memory architecture
- defining the protection and VM supports
- implementing them
- testing them
you get the picture. However, when it will work,
it will be close to flawless.
>Getting some FCPU related programs in Debian will raise the visibility of your
>project a lot!
sure. But F-CPU is only at 30% after 4 years of
trol^H^H^H^Hdiscussions. The "core" will work roughly
when implemented at 60 or 70% for simple programs.
>> So the first thing to do is know which packages
>> do not need modifications :-) i presume that it is going
>> to take a while, asking all the package maintainers
>> which of their package comply to some rules. Maybe we must
>> write a survey and send it to all the maintainers ?
>
>I think that the results for all this can already be found in the build daemon
>results:
>http://buildd.debian.org/
>
>The stats page http://buildd.debian.org/stats/ shows the statistics of how
>many packages build on each platform (over 90% for every platform), and
>allows querying which packages don't build for each platform.
>
>So I think that almost 90% of the packages will have a chance of building for
>FCPU without any modifications being required.
\o/ yeah ! \o/
>> You are free to copy this message wherever needed.
>> You can also post technical questions to the
>> english f-cpu mailing list.
>
>Thanks, I have forwarded it to the debian-devel list (which is CC'd on this
>reply).
cool.
i have forwarded the last message to the F-CPU list
and this triggered a lot of (mostly off-topic) posts...
but you can follow them here :
http://archives.seul.org/f-cpu/f-cpu/Oct-2002/threads.html
:-)
Thanks for the informations,
YG
Reply to: