On 6/28/20 6:37 AM, echo test wrote:
Hello,
Thank you for all your answers and sorry to be late for answering.> I prefer ZFS but I find that lots of corps prefer mdadm. I really think that's simply > because ZFS came from Sun and they lack Solaris backgrounds. Now, in a low-> RAM environment with simpler disc needs, I would probably go with mdadm.> Anything else I would choose ZFS. It's ability to take care of itself is surprisingly > strong. Less work for me after the set up and installation.
ZFS beeing a filesystem and mdadm an utility software, I think I'll go for mdadm. I didn't know that Debian was supporting ZFS I always used Ext4.
It's a stack. You build up from disk, to block-level raid, to
volume manager, to file system, to access protocol.
ZFS includes multiple levels of the stack. And yes there are ZFS
implementations for Debian, along with a dozen or more other file
systems.
Sorry if I'm misunderstanding, are you saying that Debian cannot scale in a bigger enterprise ?Can you tell me what happened with hardware RAID solutions?
> small" could be anything from 10 to 1000 users. Mentioning some numbers> could get you more useful recommendations.
> In any case, some interesting hardware not mentioned so far (don't
> forget about the power consumption).
Small here is for me about 2000 users all are restaurants that save their selling history locally on their own server then 2 or 3 times in the morning they will rsync their postgres data on my data center.About the power consumption, any advice about some low power hardware are also welcome.
Something seems a little flakey about that architecture. Rsyncng
postgress data? There are lots of better ways to merge data into
a database. Particularly in these days of cheap, continuous,
broadband connectivity. And if you can't get business broadband
in all your locations, cellular modems are dirt cheap. (I speak
as someone who's designed more than a few mobile data collection
systems - everything from tactical military systems to transit
buses).
> Supermicro 1U servers - run two or more of them> and it's easy to turn them into a high-available cluster> ...> Note: I'm seriously considering migrating from Debian for our> next refresh - I really don't like systemd - might go all the way to BSD
>or an OpenSolaris distro.
Supermicro seems definitely to propose some great stuff I will take them in account. Why do you dislike systemd ? I heard many people saying the same thing and I don't really understand what are their motivation except initd is less invasive.
It's a spaghetti coded package of crap, that takes over your
system and does things its way. I prefer modularity, and control
over my systems.
I don't really know how to answer to your question but let's try. We are a startup and for the moment we have a production and a development, in fact the production is just like a test environment because we do continuous delivery, we push everyday in order to know more quickly when something has been broken and our semi-automated tests didn't detect it. Personally, I'm a self learner, and probably many guys of my team are too. So some advices here are also welcome.We want to be able to handle 2500+ rsync in the morning (probably distributing them in time in order to avoid a single big load acting as a ddos) and for each client of my clients (restaurants) a get and put profile request.Note: client's profile are shared across restaurants and clients can find/filter restaurants on the website which is not yet built but we are working on it.
Rsync is just wrong for that kind of application. What are you
syncing, anyway. 2500 postgress instances, some raw data files
for input to a single instance of postgress, something else? And
if the postgress instance is feeding live data to apps, you really
need to focus first on your high-availability strategy - single
points of failure will kill you.
Come to think of it, you're a poster child for doing everything in
the cloud. As a startup, you've got way too many other things to
worry about than home brewing an IT environment - focus on your
core product/service, whatever that is. (Now, if you're setting
up a service bureau, that's another story - in which case, hire
some folks who actually know how to do this stuff. Here, I'm
speaking as someone who HAS homebrewed a small service bureau,
with serious experience in computing & IT - back before any of
this stuff was available off the shelf. It's a royal PITA. These
days, I'm far more likely to set up a new domain, or app, on a
hosting service, than on our cluster - unless & until I know
that it needs to be around for a while. Life's too short.)
Miles Fidelman
-- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown