[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: sysadmin qualifications (Re: apt-get vs. aptitude)

On 10/15/2013 11:37 PM, Miles Fidelman wrote:

Jerry Stuckle wrote:

On 10/15/2013 6:50 PM, Miles Fidelman wrote:

Jerry Stuckle wrote:

On 10/15/2013 2:26 PM, Miles Fidelman wrote:






Geeze Jerry, you're just so wrong, on so many things.


What's a "coder"?  In over 40 years of programming, I've met many
programmers, but no "coders".  Some were better than others - but none
had "limited and low-level skill set".  Otherwise they wouldn't have
been employed as programmers.


If you've never heard the term, you sure have a narrow set of
experiences over those 40 years.  And I've seena LOT of people with very
rudimentary skills hired as programmers.  Not good ones, mind you. Never
quite sure what they've been hired to do (maybe .NET coding for business
applications?).  All the serious professionals I've come across have
titles like "software engineer" and "computer engineer."



I didn't say I haven't heard of the term.  I said I've never met any.
I have, however, seen a lot of half-assed programmers call others they
consider their inferiors "coders".

And most of the serious professionals I've come across have the title
"programmer".  In many U.S. states, you can't use the term "engineer"
legally unless you are registered as one with the state.

For someone to claim they are an "Engineer" without the appropriate
qualifications (i.e. 4-year degree and passing the required test(s) in
their jurisdiction is not only a lie, it is illegal in many places.


Ummm.... no.  In the US, it's illegal to call yourself a "professional
engineer" without a state license - required of civil engineers, some
mechanical engineers, some electrical engineers (typically in power
plant engineering), and other safety critical fields.  There have been
periodic attempts to require professional licensing of software
engineers - but those have routinely failed (arguably it's would be a
good idea for folks who write safety-critical software, like aircraft
control systems, SCADA software, and such).
Try again. States do not differentiate between civil engineers, mechanical engineers, etc. and other engineers. Use of the term "Engineer" is what is illegal. Check with your state licensing board. The three states I've checked (Maryland, Texas and North Carolina) are all this way.
And yes, there have been attempts to license programmers. But that is a separate issue. 


Generally, it's considered misrepresentation to call yourself an
engineer without at least a 4-year degree from a uniersity with an
accredited program.



And a state license, as indicated above.



And "Systems Programming" has never mean someone who writes operating
systems; I've known a lot of systems programmers, who's job was to
ensure the system ran.  In some cases it meant compiling the OS with
the require options; other times it meant configuring the OS to meet
their needs.  But they didn't write OS's.


It's ALWAYS meant that, back to the early days of the field.



That's very interesting.  Because when I was working for IBM (late
70's on) on mainframes, all of our customers had "Systems
Programmers".  But NONE of them wrote an OS - IBM did that.  The
systems programmers were, however, responsible for installation and
fine tuning of the software on the mainframes.

There are a lot of "Systems Programmers" out there doing exactly that
job.  There are very few in comparison who actually write operating
systems.

It seems your experience is somewhat limited to PC-based systems.

Hmm...., in rough cronological order:
- DG Nova
- IBM 360 (batch and TSO)
- Multics
- pretty much every major flavor of DEC System (PDP-1, PDP-10/20, PDP-8,
PDP-11, VAX, a few others)
-- including some pretty funky operating systems - ITS, Cronus, and
TENEX come to mind
- both varieties of LISP machine
- various embedded systems (wrote microcode for embedded avionic
machines at one point)
- various embeded micro-controllers (both basic TTL logic and z80-class)
- various Sun desktop and server class machines
- BBN Butterfly
- IBM RS/6000
- a lot of server-class machines (ran a hosting company for a while)
- yeah, and a lot of Macs, some PCs, a few Android devices, and a couple
of TRS-80s in there along the way
So young? I started on an IBM 1410, several years before the 360 was introduced. And I see you've played with a few minis. But you obviously have limited experience in large shops. 





Can't find any "official definition" - but the WikiPedia definition is
reasonably accurate: "*System programming* (or *systems programming*) is
the activity of computer programming
<http://en.wikipedia.org/wiki/Computer_programming> system software
<http://en.wikipedia.org/wiki/System_software>. The primary
distinguishing characteristic of systems programming when compared to
application programming
<http://en.wikipedia.org/wiki/Application_programming> is that
application <http://en.wikipedia.org/wiki/Application_software>
programming aims to produce software which provides services to the user
(e.g. word processor <http://en.wikipedia.org/wiki/Word_processor>),
whereas systems programming aims to produce software which provides
services to the computer hardware
<http://en.wikipedia.org/wiki/Computer_hardware> (e.g. disk defragmenter
<http://en.wikipedia.org/wiki/Defragmentation>). It requires a greater
degree of hardware awareness.



And Wikipedia is an authority?  Nope.  It's just one person's opinion.
Tomorrow someone else may update it with another opinion.

No.. I quoted it because it's accurate - and it's consistent with usage
since at least 1971, when I first encountered the term.
Not at all accurate in my experience - since 1967. And my experience included large shops and working with systems programmers at those shops. 


That's been the usage since the days I took courses in it at MIT (early
1970s), and how the term is used in all the textbooks by folks like
Jerry Saltzer, John Donovan, Corbato - names you should recognive if
you've been in the field for 40 years.



That's very interesting, because some of the Systems Programmers I
knew also graduated from MIT.  And they agreed they were systems
programmers.

And yes, I recognize those names.  But they never were that highly
regarded except in academia.


Hmmm... John Donovan was THE corporate guru for about 20 years - until
he had a personal scandle that blew up his career.  I'd consider Saltzer
and Corbato to have some pretty good creds beyond academia - places like
DARPA, DoD in general, all those large corporations (like IBM) who
funded a lot of their work.
You can consider them that way. I'm just talking about what my experience was - including when I was at IBM. And AFAIK, IBM never funded any of their work. Please show which projects of theirs were funded by IBM. Even one would be great.
And I never said they didn't work beyond academia (DARPA is basically an extension of academia, as are many DOD research jobs). I said they weren't highly regarded beyond academia - including John Donovan. And if he were so highly regarded, a personal scandal would not have ruined a professional reputation. 





The programmers where I'm currently working - application systems for
buses (vehicle location, engine monitoring and diagnostics,
scheduling,
passenger information) -- yeah, they have to worry about things
like how
often vehicles send updates over-the-air, the vageries of data
transmission over cell networks (what failure modes to account for),
etc., etc., etc.



That's not "real time".  "Real time" is when you get an input and you
have to make an immediate decision and output it.  Motor controllers
are one example; constantly adjusting motor speed to keep a conveyor
belt running at optimum capacity as the load changes.  Another is
steering a radiotelescope to aim at a specific point in the sky and
keep it there.  Oh, and once they radiotelescope is properly aimed,
process the information coming from it and 30-odd others spaced over a
couple of hundred square miles, accounting for the propagation delay
from each one, and combining the outputs into one digital signal which
can be further processed or stored.


There's a spectrum of real-time.  Everything from radar jamming
(picoseconds count), to things that happen on the order of seconds
(reporting and predicting vehicle locations).  Basically anything where
timing and i/o count.



Reporting and predicting vehicle locations is considered "real time"
by a small portion of the computing world, I will agree.



And worrying about the vagaries of data transmission over cellular
networks requires no network knowledge below the application level. In
fact, I doubt your programmers even know HOW the cellular network
operates at OSI layers 6 and below.


Ummm... yes.  Our guys write protocols for stuffing vehicle state data
into UDP packets, drivers for talking across funny busses (e.g. J1908
for talking to things like message signs, engine control units,
fareboxes).  Occaisionally we have to deal with controlling on-board
video systems and distributing video streams.  Busses also typically
have multiple systems that share a router that talks both cellular
(on-the-road) and WiFi (at the depot) - lots of resource management
going on.  And don't get me started on running data over trunked radio
networks designed for voice.



So?  How much do they (or YOU, for that matter) know about what goes
on in your transmissions?  Do you actually create the bits in the UDP
packets?  Do you actually modulate the packets for sending across the
airwaves, then demodulate them at the bus?  Do you ACK the message
reception, and resend if a packet (or ACK) doesn't make it?

Or do you call a set of API's with a destination and data and lest the
system take over?


For our OTAR protocol, we munge the bits.  Same again for J1908.
You actually changed the bits in the header/trailers? Or just the data? If the former, how did you get the network to handle them? Did you rewrite the network code all along the way, also? 






When I worked on military systems - trainers, weapons control,
command &
control, intelligence, ..... - you couldn't turn your head without
having to deal with "real world" issues - both of the hardware and
networks one was running on, and the external world you had to
interact
with.



I'm sorry your military systems were so unstable.  But programmers
don't worry about the hardware.


Hah.. Tell that to someone who's designing a radar jammer, or a fire
control system.  Now we're talking "hard real-time" - no timing jitter
allowed as control loops execute.  Yeah, now we're talking about custom
microcode to execute funny algorithms, and interrupt-driven programming.



I'm sure you've designed a lot of radar jammers.  And I'll bet you've
designed fire control systems, also.


Radar jammers yes.  Fire control no.


Whoops - you said microcode.  You just exposed yourself.  The only
people who write microcode are the designers of the CPU.  Once it's
out of the factory, people may write assembler (translated into
machine code).  But they don't write microcode.


Umm yup.  That was 1984, designing an embedded processor that went into
electronic warfare pods.  16-layer board, wall-to-wall ECL flatpacks,
mostly MSI.  We designed to a standard Air Force macro-instruction set;
and added about 6 very specialized additional instructions to twiddle
bits in very funny ways.  I wrote a bunch of the microcode, personally.



And real time systems do not use interrupts.  They cannot handle the
semi-random timing changes caused by interrupts.  If there are
interrupts which must be handled, they are handled by an entirely
different processor whose only job is to deal with interrupts.


Tell me how many real-time systems you've designed.  Most of the ones
I've dealt with have been very much event driven.
Only about a dozen. And while all of them were event driven, most (a couple weren't all that critical in the timing) handled interrupts via a separate processor from the main controller. 




For that matter, anybody who has to deal with displays - e.g. for
simulators or games - has to worry about hardware.



There are thousands of display cards out there for the PC alone. Each
one has a different interface.  Those who write real simulators do it
for a specific set of hardware.  Game developers can't (and don't)
worry about all the different hardware out there.


Then again, we never hired "programmers" - we hired software engineers,
who were expected to have some serious electrical engineering and
computer hardware background.



Are they licensed engineers?  See above.


They tend to have Masters and Doctoral degrees from electrical
engineering, computer engineering, and computer science programs.


That's not answering the question.






If you think anybody can code a halfway decent distributed
application,
without worrying about latency, transmission errors and recovery,
network topology, and other aspects of the underlying "stuff" - I'd
sure
like some of what you've been smoking.



Been doing it for 30+ years - starting when I was working for IBM back
in '82.


For example?  Distributed systems were still a black art in the early
1980s.  Those were the days of SNA, DECNET, and X.25 public networks -
most of what went across those was remote job entry and remote terminal
access.  Distributed applications were mostly research projects. Network
links were still 56kbps if you were lucky (and noisy as hell), and an
IBM 3081 ran at what, 40 MIPS or so.



Not at all.  There were a lot of distributed systems on mainframes
back then.  Most large companies had them; airlines were a great
example. But even large banks and insurance companies used distributed
systems.

IBM had perhaps the largest; hundreds of computers around the world
capable of talking to each other and getting data from each other. And
I could log into any of them (if I had authorization) from almost any
terminal in an IBM office, and even many customer sites.


As I recall, most of the business stuf was running on centralized
mainframes, with distributed terminals.  Though I guess SAGE dates back
to the 50s and that was certainly IBM, and both air traffic control and
things like the SABRE and APOLLO reservation systems emerged from that
technology base.
Then you don't know much about large companies. Smaller companies, yes. But not the big companies. Way too much for one mainframe to handle. And none of them had anything to do with SAGE, SABRE or any other reservation system. 




What could you have been working on in the 1980s that WASN'T incredibly
sensitive to memory use, disk usage, cpu use, and network bandwidth?



Large mainframes.  Distributed computing resolved much of the memory
and cpu usage; network bandwidth was a problem, but then programmers
knew how to minimize data transfer.  For instance, rather than
transfer large amounts of data across the network then filter on the
receiving end, programs on the hosting computer would do the filtering
and only send what was necessary.


In other words, programmers had to know enough about the network's
limitations to worry about minimizing data transfer (as opposed to
today's coders who seem to think that bandwidth is infinite, and then
get messed up when they start writing apps for cell phones, where people
get charged for bandwidth).  I expect latency and error recovery were
also considerations for those systems.  (I also seem to recall learning
an awful lot about HDLC framing back in the day).
Nope. They knew nothing about the network. All they knew about was how to code efficiently - including minimizing data transfer. They didn't know whether the data was going across a high speed SDLC connection using SNA or a 120 baud ASCII connection. And it wasn't important, because the connection could change at some point without rewriting any of their code. They were good programmers, because they wrote code which did not depend on underlying hardware or network protocols. 








Oh, and by the way, an awful lot of big data applications are run on
standard x86 hardware - in clusters and distributed across networks.
Things like network topology, file system organization (particularly
vis-a-vis how data is organized on disk) REALLY impacts performance.



That's what I mean about what people thing is "big data". Please show
me ANY X86 hardware which can process petabytes of data before the end
of the universe.  THAT is big data.

Too many people who have never seen anything outside of the PC world
think "big data" is a few gigabytes or even a terabyte of information.
In the big data world, people laugh at such small amounts of data.


There are an awful lot of Beowulf clusters out there.  Stick a few 100
(or a few thousand) multi-core processors in a rack and you've got a
supercomputer - and that's what most of the big data processing I know
of is being done on.  Heck, Google's entire infrastrucure is built out
of commodity x86 hardware.  (There's a reason we don't see a lot of
surviving supercomputer manufacturers.)



Google isn't a "supercomputer".  It is a bunch of processors handling
individual requests.  The real supercomputers do things like weather
forecasting, and nuclear explosion simulations, where billions of
individual pieces of data must be processed concurrently.


Google's indexing and search algorithms (and advertising analytics) are
very definitely "big data" applications.
Google is a data warehouse. While they deal with lots of data, it is not considered a "supercomputer" by anyone who knows supercomputers. 


As to "real supercomputers" - NCAR's Yellowstone processor is built out
of something like 10,000 Xeon 8-core processors.  It's the backplace
design and operating system environment that turn it into a
"supercomputer."
I didn't say they weren't commodity chips. But they aren't just a bunch of PC's in a huge network, like Google is. 


Yes, the early generations of supercomputers had small numbers of
processors that ran REALY fast (for their day), and did all kinds of
fancy pipelining.  Today, most "supercomputers" tend to exploit massive
parallelism, but when you dig under the covers, the building blocks are
commodity chips.  The days of really specialized hardware (like the
Connetion Machine, or BBN Butterfly) seem to be over (though there are
some interesting things being done both for graphics processing and very
low level telecom. functions).



And the reason we don't see a lot of surviving supercomputer
manufacturers is there never were very many of them in the first
place.  The cost of the supercomputers was so high only a very few
could afford them - generally governments or government contractors.


Disk i/o is the problem - again, a problem that requires find tuning
algorithms to file system design to how one stores and accesses data on
the physical drives.



Yes, that has always been the biggest problem on PC's.  Mainframes use
an entirely different means, and transfer data at rates PC's only
dream about.  Additionally, the transfer is done entirely in hardware,
requiring no CPU between the time the command list (and there could be
dozens or even hundreds of commands in the list) is turned over to the
I/O system and the last command is completed (or an I/O error occurs).


Big deal - fiber channel SANS, with Oracle running on top.  Works fine
for some things, not so well for large classes of operations. The
bottleneck remains the drives themselves (physical seek times for
heads).  That's why we've been seeing a new generation of file systems
tailored for massive search.
You're the one who brought it up, so it must be a big deal. And I'm not talking about just the path between the disk and the processor. For instance, disks for mainframes typically have faster seek times than PC disks. Also, mainframe disks spin faster and can read from multiple heads concurrently. The controllers also have larger buffers (multi-gb is not abnormal today). The result is a mainframe disk controller reads an entire cylinder in less time than it takes a PC disk to read one track.
Plus one disk controller can handle multiple disks concurrently and a mainframe can have multiple controllers concurrently. Spreading data across multiple disks and controllers further reduces access time. 






I might also mention all the folks who have been developing algorithms
that take advantage of the unique characteristics of graphics
processors
(or is algorithm design outside your definition of "programming" as
well?).



What about them?


Do you consider figuring out algorithms to be part of "programming?"  If
so, how do you exclude a knowledge of machine characteristics from the
practice of "programming?"



Yes, I consider figuring out algorithms to be a part of programming.
But unless you are programming a real-time system with critical timing
issues (see above),   Otherwise, a good programmer will write the
algorithm in a machine-independent manner for greater portability.


I'm trying to figure out what kinds of things you see "programmers"
working on that don't need serious knowledge of the underlying operating
system, computer hardware, and i/o environment.




How about any application program?  How much do you need to know about
the OS to write a word processor, for instance?  Or an IDE such as
Eclipse - written entire in Java, and runs on several different
platforms.  Nothing in the application knows anything about the
underlying OS, hardware or I/O.


Not a of people writing word processors these days, or IDEs for that
matter. (Actually, any halfway decent development environment needs to
support low-level debugging - process trace, timing analysis, resource
usage - all kinds of things that take you deep into the o/s and hardware).
You asked for examples. These are two of them. I can list them for days on end. Good programmers can and do) write code which is hardware and OS agnostic. 

Even most of Debian and its applications are hardware agnostic.





In fact, very few applications need to know details about the
underlying OS or hardware.  Any any application who programs
unnecessarily to the hardware is doing his employer a huge disservice.


Judging from what crosses job boards these days, MOST applications tend
to be resource intensive, and hence algorithm design tends to require
deep knowledge of o/s and hardware:
- military systems for sure
- avionics
- industrial control
- video (lots of demand for developers of mobile video applications -
need a lot of knowledge of both video hardware and compression schemes
and the hardware that supports them)
- gaming (need to know a lot about specialized graphics hardware)
- automotive computing (both user facing and on-board systems control)
Job boards are not a good indication of what is being used. It is only an indication of what people can't find. They have always been heavy on specialized experience - and not just in the computer industry.
And different job boards lean towards different specializations. Looking at one job board is far from any reasonable indication of the state of the industry and what is in demand. 


Just for the hell of it, I went to a job board (indeed.com) and searched
on "programmer" jobs in the Boston area, and there seems to be an awful
lot of call for people who can write data collection systems (i/o
intensive, and video systems).
How many COBOL programmers were requested? Never mind the fact it is still the most used language in the world. 


Anybody who thinks that being able to write code (be it Java, C, or .NET
crap), without knowing a lot about the environment their code is going
to run in, much less general analytic and design skills, is going to
have a very short-lived career.
Anyone who can't write good cross-platform code which doesn't depend on specific hardware and software has already limited his career. Anyone who can write good cross-platform code has a much greater career ahead of him. It is much harder than writing platform-specific code. 

Jerry
Reply to: