...to know him was to love him. And to love him was to know him. Those who know him LOVED him - while those who did not know him, loved him from afar... -- Data/Graves, "The Schizoid Man", stardate 42437.5

Know Your Unix System Administrator
---A Field Guide

There are four major species of Unix sysad:

  1. The TECHNICAL THUG. Usually a systems programmer who has been forced into system administration; writes scripts in a polyglot of the Bourne shell, sed, C, awk, perl, and APL.
  2. The ADMINISTRATIVE FASCIST. Usually a retentive drone (or rarely, a harridan ex-secretary) who has been forced into system administration.
  3. The MANIAC. Usually an aging cracker who discovered that neither the Mossad nor Cuba are willing to pay a living wage for computer espionage. Fell into system administration; occasionally approaches major competitors with indesp schemes.
  4. The IDIOT. Usually a cretin, morpohodite, or old COBOL programmer selected to be the system administrator by a committee of cretins, morphodites, and old COBOL programmers.

How To Identify Your System Administrator:

--SITUATION: Low disk space. --

TECHNICAL THUG: Writes a suite of scripts to monitor disk usage, maintain a database of historic disk usage, predict future disk usage via least squares regression analysis, identify users who are more than a standard deviation over the mean, and send mail to the offending parties. Places script in cron. Disk usage does not change, since disk-hogs, by nature, either ignore script-generated mail, or file it away in triplicate.

ADMINISTRATIVE FASCIST: Puts disk usage policy in motd. Uses disk quotas. Allows no exceptions, thus crippling development work. Locks accounts that go over quota.


# cd /home # rm -rf `du -s * | sort -rn | head -1 | awk '{print $2}'`; IDIOT: # cd /home # cat `du -s * | sort -rn | head -1 | awk '{ printf "%s/*\n", $2}'` | compress

--SITUATION: Excessive CPU usage. --

TECHNICAL THUG: Writes a suite of scripts to monitor processes, maintain a database of CPU usage, identify processes more than a standard deviation over the norm, and renice offending processes. Places script in cron. Ends up renicing the production database into oblivion, bringing operations to a grinding halt, much to the delight of the xtrek freaks.

ADMINISTRATIVE FASCIST: Puts CPU usage policy in motd. Uses CPU quotas. Locks accounts that go over quota. Allows no exceptions, thus crippling development work, much to the delight of the xtrek freaks.


# kill -9 `ps -augxww | sort -rn +8 -9 | head -1 | awk '{print $2}'` IDIOT: # compress -f `ps -augxww | sort -rn +8 -9 | head -1 | awk '{print $2}'`

--SITUATION: New account creation. --

TECHNICAL THUG: Writes perl script that creates home directory, copies in incomprehensible default environment, and places entries in /etc/passwd, /etc/shadow, and /etc/group. (By hand, NOT with passmgmt.) Slaps on setuid bit; tells a nearby secretary to handle new accounts. Usually, said secretary is still dithering over the difference between 'enter' and 'return'; and so, no new accounts are ever created.

ADMINISTRATIVE FASCIST: Puts new account policy in motd. Since people without accounts cannot read the motd, nobody ever fulfills the bureaucratic requirements; and so, no new accounts are ever created.

MANIAC: "If you're too stupid to break in and create your own account, I don't want you on the system. We've got too many goddamn sh*t-for-brains a**holes on this box anyway."


# cd /home; mkdir "Bob's home directory" # echo "Bob Simon:gandalf:0:0::/dev/tty:compress -f" > /etc/passwd

--SITUATION: Root disk fails. --

TECHNICAL THUG: Repairs drive. Usually is able to repair filesystem from boot monitor. Failing that, front-panel toggles microkernel in and starts script on neighboring machine to load binary boot code into broken machine, reformat and reinstall OS. Lets it run over the weekend while he goes mountain climbing.

ADMINISTRATIVE FASCIST: Begins investigation to determine who broke the drive. Refuses to fix system until culprit is identified and charged for the equipment.

MANIAC, LARGE SYSTEM: Rips drive from system, uses sledgehammer to smash same to flinders. Calls manufacturer, threatens pets. Abuses field engineer while they put in a new drive and reinstall the OS.

MANIAC, SMALL SYSTEM: Rips drive from system, uses ball-peen hammer to smash same to flinders. Calls Requisitions, threatens pets. Abuses bystanders while putting in new drive and reinstalling OS.

IDIOT: Doesn't notice anything wrong.

--SITUATION: Poor network response. --

TECHNICAL THUG: Writes scripts to monitor network, then rewires entire machine room, improving response time by 2%. Shrugs shoulders, says, "I've done all I can do," and goes mountain climbing.

ADMINISTRATIVE FASCIST: Puts network usage policy in motd. Calls up Berkeley and AT&T, badgers whoever answers for network quotas. Tries to get xtrek freaks fired.

MANIAC: Every two hours, pulls ethernet cable from wall and waits for connections to time out.


# compress -f /dev/en0

--SITUATION: User questions. --

TECHNICAL THUG: Hacks the code of emacs' doctor-mode to answer new users questions. Doesn't bother to tell people how to start the new "guru-mode", or for that matter, emacs.

ADMINISTRATIVE FASCIST: Puts user support policy in motd. Maintains queue of questions. Answers them when he gets a chance, often within two weeks of receipt of the proper form.

MANIAC: Screams at users until they go away. Sometimes barters knowledge for powerful drink and/or sycophantic adulation.

IDIOT: Answers all questions to best of his knowledge until the user realizes few UNIX systems support punched cards or JCL.

--SITUATION: *Stupid* user questions. --

TECHNICAL THUG: Answers question in hex, binary, postfix, and/or French until user gives up and goes away.

ADMINISTRATIVE FASCIST: Locks user's account until user can present documentation demonstrating their qualification to use the machine.


# cat >> ~luser/.cshrc alias vi 'rm \!*;unalias vi;grep -v BoZo ~/.cshrc > ~/.z; mv -f ~/.z ~/.cshrc' ^D IDIOT: Answers all questions to best of his knowledge. Recruits user to system administration team.

--SITUATION: Process accounting management. --

TECHNICAL THUG: Ignores packaged accounting software; trusts scripts to sniff out any problems & compute charges.

ADMINISTRATIVE FASCIST: Devotes 75% of disk space to accounting records owned by root and chmod'ed 000.

MANIAC: Laughs fool head off at very mention of accounting.


# lpr /etc/wtmp /usr/adm/paact

--SITUATION: Religious war, BSD vs. System V. --

TECHNICAL THUG: BSD. Crippled on System V boxes.

ADMINISTRATIVE FASCIST: System V. Horrified by the people who use BSD. Places frequent calls to DEA.

MANIAC: Prefers BSD, but doesn't care as long as HIS processes run quickly.


# cd c:

--SITUATION: Religious war, System V vs. AIX --


ADMINISTRATIVE FASCIST: AIX-- doesn't much care for the OS, but loves the jackboots.

MANIAC: System V, but keeps AIX skills up, knowing full well how much Big Financial Institutions love IBM...


--SITUATION: Balky printer daemons. --

TECHNICAL THUG: Rewrites lpd in FORTH.

ADMINISTRATIVE FASCIST: Puts printer use policy in motd. Calls customer support every time the printer freezes. Tries to get user who submitted the most recent job fired.

MANIAC: Writes script that kills all the daemons, clears all the print queues, and maybe restarts the daemons. Runs it once a hour from cron.


# kill -9 /dev/lp ; /dev/lp &

--SITUATION: OS upgrade. --

TECHNICAL THUG: Reads source code of new release, takes only what he likes.

ADMINISTRATIVE FASCIST: Instigates lawsuit against the vendor for having shipped a product with bugs in it in the first place.


# uptime 1:33pm up 19 days, 22:49, 167 users, load average: 6.49, 6.45, 6.31 # wall Well, it's upgrade time. Should take a few hours. And good luck on that 5:00 deadline, guys! We're all pulling for you! ^D IDIOT: # dd if=/dev/rmt8 of=/vmunix

--SITUATION: Balky mail. --

TECHNICAL THUG: Rewrites sendmail.cf from scratch. Rewrites sendmail in SNOBOL. Hacks kernel to implement file locking. Hacks kernel to implement "better" semaphores. Rewrites sendmail in assembly. Hacks kernel to . . .

ADMINISTRATIVE FASCIST: Puts mail use policy in motd. Locks accounts that go over mail use quota. Keeps quota low enough that people go back to interoffice mail, thus solving problem.


# kill -9 `ps -augxww | grep sendmail | awk '{print $2}'` # rm -f /usr/spool/mail/* # wall Mail is down. Please use interoffice mail until we have it back up. ^D # write max I've got my boots and backpack. Ready to leave for Mount Tam? ^D IDIOT: # echo "HELP!" | mail tech_support.AT.vendor.com%kremvax%bitnet!BIFF!!!

--SITUATION: Users want phone list application. --

TECHNICAL THUG: Writes RDBMS in perl and Smalltalk. Users give up and go back to post-it notes.

ADMINISTRATIVE FASCIST: Oracle. Users give up and go back to post-it notes.

MANIAC: Tells the users to use flat files and grep, the way God meant man to keep track of phone numbers. Users give up and go back to post-it notes.


% dd ibs=80 if=/dev/rdisk001s7 | grep "Fred"


-- TYPICAL ROOT .cshrc FILE: --

TECHNICAL THUG: Longer than eight kilobytes. Sources the output of a perl script, rewrites itself.

ADMINISTRATIVE FASCIST: Typical lines include:

umask 777 alias cd 'cd \!*; rm -rf ching *hack mille omega rogue xtrek >& /dev/null &' MANIAC: Typical lines include: alias rm 'rm -rf \!*' alias hose kill -9 '`ps -augxww | grep \!* | awk \'{print $2}\'`' alias kill 'kill -9 \!* ; kill -9 \!* ; kill -9 \!*' alias renice 'echo Renice\? You must mean kill -9.; kill -9 \!*' IDIOT: Typical lines include: alias dir ls alias era rm alias kitty cat alias process_table ps setenv DISPLAY vt100


TECHNICAL THUG: Writes entries for Obsfuscated C contest. Optimizes INTERCAL scripts. Maintains ENIAC emulator. Virtual reality.

ADMINISTRATIVE FASCIST: Bugs office. Audits card-key logs. Modifies old TVs to listen in on cellular phone conversations. Listens to police band.

MANIAC: Volunteers at Survival Research Labs. Bugs office. Edits card-key logs. Modifies old TVs to listen in on cellular phone conversations. Jams police band.

IDIOT: Ties shoes. Maintains COBOL decimal to roman numeral converter. Rereads flowcharts from his salad days at Rand.


TECHNICAL THUG: Drinks "Smart Drinks." Attends raves. Hangs out at poetry readings and Whole Earth Review events and tries to pick up Birkenstock MOTAS.

ADMINISTRATIVE FASCIST: Reads _Readers Digest_ and _Mein Kampf_. Sometimes turns up car radio and sings along to John Denver. Golfs. Drinks gin martinis. Hangs out in yuppie bars and tries to pick up dominatrixes.

MANIAC: Reads _Utne Reader_ and _Mein Kampf_. Faithfully attends Dickies and Ramones concerts. Punches out people who say "virtual reality." Drinks damn near anything, but favors Wild Turkey, Black Bush, and grain alcohol. Hangs out in neighborhood bars and tries to pick up MOTAS by drinking longshoremen under the table .

IDIOT: Reads _Time_ and _Newsweek_-- and *believes* them. Drinks Jagermeister. Tries to pick up close blood relations-- often succeeds, producting next generation of IDIOTs.


TECHNICAL THUG: Clinton, but only because he liked Gore's book.

ADMINISTRATIVE FASCIST: Bush. Possibly Clinton, but only because he liked Tipper.

MANIAC: Frank Zappa.

IDIOT: Perot.


TECHNICAL THUG: Richard Stallman - Larry Wall.


MANIAC: Frank Zappa.

IDIOT: Quayle.

Compound System Administrators:

TECHNICAL FASCIST: Hacks kernel & writes a horde of scripts to prevent folk from ever using more than their fair share of system resources. Resulting overhead and load brings system to its knees.

TECHNICAL MANIAC: Writes scripts that SEEM to be monitoring the system, but are actually encrypting large lists of passwords. Uses nearby nodes as beta test sites for worms.

TECHNICAL IDIOT: Writes superuser-run scripts that sooner or later do an "rm -rf /".

FASCISTIC MANIAC: At first hint of cracker incursions, whether real or imagined, shuts down system by triggering water-on-the-brain detectors and Halon system.

FASCISTIC IDIOT: # cp /dev/null /etc/passwd


-Stephan Zielinski

Last Modified: January 19, 1995
And on the 7th day, he said: there will be kernel And it shall multitask And it shall support virtual memory Then the lamb broke the fourth seal, and there was a great coredump.... And the Lamb was without gdb And no core could be traced. And all was darkness For it was sun ; and therefore crippled Though i walk through the valley of the shadow of passwd, i fear no access In the beginning God created the heaven and the earth. And the earth was without form, and void, and darkness was upon the face of the earth. For Bill Gates hath rode high. The earth was however, with silicon Out of the silicon and chaos rose the great CPU And it was SPARC And SPARC was good. And on the sacred scroll, there was GPL. And he broke the ninety-five seal on the package. Bill Gates and his army of darkness rode forth And they claimed all the CPU time in the name of the WYSIWIG, a horrible creature. And BSD spawn forth daemons. And the daemons proceeded to make the best of the remaining cpu time... For they were efficient. And Jesus spoke, "Brothers, fork not what you will not exec." "Always free your mallocs." And never time() when one can gettimeofday() And the commandments said: Thou shalt not worship DOS ; for it is lame Love thy kernel like thyself Thou shalt not covet thy neighbors P99-29873529857 and his X fonts And thou shalt not rm -rf /* as root, even to install upgrades. And thou shalt not kill -9 -1 as root, whatever thy reason Cast thy proceses upon the /proc filesystem ; for you shall still find them running after many days And when Jesus turned Intels into Alphas, there was a great rejoicing. "Truly it is a miracle," exclaimed Peter... And they ran OSF/1...and it was slow, and buggy And the admins sagged in despair, for it was like SYSV SYSV was the beast they had run off gthe corners of the earth before And they feared the dark star And the dark star descended across the earth. and all domains were registered with att.com And the lusers lost faith in the admins And were banished to the wastelands of Dynix and XENIX And mesg no longer saved them, for the dark one had confiscated all setuid bits And Pilate asked the crowd, "Do you want BSD or OS/2?" And the crowd shouted, "WE WANT BSD! WE WANT BSD!" So Pilate freed BSD and sent OS/2 to be crucified... The kernel was evil ; for it paniced It looped endlessly, and suffocated the daemons And crond could not save them For it was wandering for 40 days and 40 nights across the swap And it was lagged ; for it was A/UX, and required a load average of 1 just to run And the ircd held on to the bitter end. But there was a RunBug (tm) And it was evil. And difficult to find. For the evil sorceror crypt() had rendered all comments invisible And the dark forces of AT&T hath extended the keylength 7 times seventy Thinking Machines worked for 40 days and 40 nights And they cored often On the 40th day, one cried out: Broadcast message from root on console: We hath found the key And it was multicast And all the kernels in all the world waited patiently for the UDP packet to arrive so they could crush the dark forces and return to glory But AT&T hath discovered the plan And they unleashed evil ICMP And many kernels were ultrix, and therefore naked And they believed the lie.