From stevegt@roton.terraluna.org Sun Dec 2 11:08:38 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id LAA16657 for infrastructures@scramjet.TerraLuna.Org; Sun, 2 Dec 2001 11:08:38 -0800 Date: Sun, 2 Dec 2001 11:08:37 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011202110837.A16449@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Test Message Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: ...testing empty list and archives after initial Mailman setup. -- . . ` * Steve Traugott ` . * + Infrastructure Architect + ` stevegt@TerraLuna.Org ' * . ' +` * http://www.stevegt.com/ From stevegt@roton.terraluna.org Sun Dec 2 11:19:32 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id LAA22284 for infrastructures@scramjet.TerraLuna.Org; Sun, 2 Dec 2001 11:19:32 -0800 Date: Sun, 2 Dec 2001 11:19:31 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011202111931.A17109@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by roton.terraluna.org id LAA22284 Subject: [Infrastructures] Test Message 2 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: ...testing list with one subscriber after Mailman setup. -- . . ` * Steve Traugott ` . * + Infrastructure Architect + ` stevegt@TerraLuna.Org ' * . ' +` * http://www.stevegt.com/ From stevegt@roton.terraluna.org Sun Dec 2 23:28:38 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id XAA07668 for infrastructures@scramjet.TerraLuna.Org; Sun, 2 Dec 2001 23:28:38 -0800 Date: Sun, 2 Dec 2001 23:28:37 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011202232837.A2554@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="tThc/1wpZn/ma/RB" Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Archives are up! Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi All! Infrastructures list migration to mailman complete. So far it looks like the patient survived. Feel free to let me know if you have any problems. I've also migrated the majordomo archives (which were never available via the web) to mailman -- so now every stinkin' message anyone has ever posted to the infrastructures list is indexed and formatted and available at http://mailman.terraluna.org/pipermail/infrastructures/. This URL is also on the 'community' page at infrastructures.org, replacing the dead archive URL which was @skylab. Who's at LISA? Joyce and I just got here tonight... =20 Steve --=20 . . ` * =20 Steve Traugott ` . * + =20 Infrastructure Architect + ` =20 stevegt@TerraLuna.Org ' * . ' +` * =20 http://www.stevegt.com/ --tThc/1wpZn/ma/RB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE8Cymk8rKIxO1Fc9MRAqhPAJ9+9TSkeXQf73g+n0/HT4N3sTpD4wCglywp rchpDu1KHuEqz2Gr0RgvzjU= =siRf -----END PGP SIGNATURE----- --tThc/1wpZn/ma/RB-- From mbarr@mbarr.net Thu Dec 13 20:44:42 2001 Received: from neuromancer.mbarr.net (66-108-143-133.nyc.rr.com [66.108.143.133]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id UAA22761 for ; Thu, 13 Dec 2001 20:44:34 -0800 Received: from [192.168.0.10] (unknown [192.168.0.10]) by neuromancer.mbarr.net (Postfix) with ESMTP id 77E42211D3D for ; Thu, 13 Dec 2001 23:44:31 -0500 (EST) Mime-Version: 1.0 X-Sender: mbarr@192.168.0.10 (Unverified) Message-Id: Date: Thu, 13 Dec 2001 23:44:28 -0500 To: infrastructures@roton.terraluna.org From: Matthew Barr Content-Type: text/plain; charset="us-ascii" ; format="flowed" Subject: [Infrastructures] Introduction... Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I met some of you at LISA, but I figured I should at least say hello :-) I'm actually a Unix admin/etc with an interest in the larger picture, but less interest in actually coding my own solutions from scratch. Steven suggested that this might be novel set of preferences for this list, so here I am.. Basically, I'm guessing that I'm a typical user of these design and architecture and large scale configuration tools, but not a scratch designer. I've worked at Lehman Brothers in the past, and am now somewhat independent. (translation= still looking for full time work ;-) Hope everyone is having a happy holiday season.. (Happy Chanukah for now!) Matthew -- _______________________________________________________________________ Matthew Barr mailto:mbarr@mbarr.net AIM: MBarr1244 ICQ: 22130424 From ken@scriba.org Tue Dec 11 06:12:14 2001 Received: from scriba.org (ken@scriba.org [208.178.122.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id GAA28773 for ; Tue, 11 Dec 2001 06:12:13 -0800 Received: from localhost (ken@localhost) by scriba.org (8.8.5/8.7.3) with ESMTP id JAA21580 for ; Tue, 11 Dec 2001 09:12:01 -0500 Date: Tue, 11 Dec 2001 09:12:01 -0500 (EST) From: Kenneth McCracken X-Sender: ken@mansfield To: infrastructures@TerraLuna.Org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] non-profit ISP setup Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hello Steve, I am managing Toronto Freenet(now Toronto's oldest dialup, ISP since 1994 supplying barrier-free access to Toronto citizens) these days. We are attempting to offer a reliable service, but it has been difficult and a lot of the problems have been around (volunteer) sysadmin issues and hardware failures. I was wondering about a pathway to a more solid footing. We are migrating from antiquated sun equipment to AMD equipment with dual IDE drives configured for software RAID1 running freebsd and a single cisco as5200 router. We need to grow the ISP operation in an orderly way to provide a (income) foundation for other more forward looking projects. I've mentioned the Gold server concept/infrastructures.org site to the technical people involved, and I pretty much get silence. We also will soon be bringing to various low income people a few sites(hopefully remotely administered) where they access the internet via a high speed connection through lans of various configurtions: 1. 10-20 low end pentiums and and a lan server netbooting to recover client systems if files are messed with. 2. Similar lans with fewer mid-level pentiums, with fileserver to preserve working hooome directories 3. Higher end pentiums as internet kiosks in tight quarters The president of our sponsoring (colocating and bandwidth wholesaling) host is willing to take an evening to do some of the hands on configuation stuff, but I am not sure he will take the time to set it up so that it is easily replicatible. Our budget is quite limited, so I was wondering if you might already have some out of the box solutions or prescriptions for these particular situations. Ken McCracken Executive Director Toronto Free-Net 416-828-9323 From stevegt@roton.terraluna.org Fri Dec 14 00:04:40 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id AAA28957 for infrastructures@scramjet.TerraLuna.Org; Fri, 14 Dec 2001 00:04:40 -0800 Date: Fri, 14 Dec 2001 00:04:39 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011214000439.A5671@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="mP3DRpeJDSE+ciuQ" Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Wiki stuff Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: --mP3DRpeJDSE+ciuQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Paul Holbrook's got me looking at UseModWiki... And I know Bud liked his SDF TWiki. Anyone else have any other recommendations, pro or con? One thing I'd like to support, for instance, is file uploads -- which opens a hairball of security issues; would be nice to integrate the Wiki username space with Mailman in some way, and only allow uploads from list members using their Mailman password. I've just discovered Bud's "Annotated Infrastructure Links" page, which looks like a copy of the html from Bud's old TWiki. I've added a link to it from the Infrastructures.Org 'lists_and_community' page. It's at http://www.sistema.it/infrastructures/infrastructures.html. Several people at last week's LISA mentioned Bud's TWiki, so I'm happy we can make these links available again. (Bud also sent me the raw data a while back, but I've never had the chance to put it up on a web or Wiki anywhere. It's likely Bud also told me about the above URL around the same time... If so, I blame senility on my part.) Steve --=20 . . ` * =20 Steve Traugott ` . * + =20 Infrastructure Architect + ` =20 stevegt@TerraLuna.Org ' * . ' +` * =20 http://www.stevegt.com/ --mP3DRpeJDSE+ciuQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE8GbKW8rKIxO1Fc9MRAgkgAKCb+/11SKFvbevJyLlbiDLOnkF0kACglbQR ojSEaq7fOa4MZ7QxYcg2VRs= =Piqf -----END PGP SIGNATURE----- --mP3DRpeJDSE+ciuQ-- From partain@dcs.gla.ac.uk Fri Jan 4 07:29:36 2002 Received: from iona.dcs.gla.ac.uk (iona.dcs.gla.ac.uk [130.209.240.35]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA16678 for ; Fri, 4 Jan 2002 07:29:34 -0800 Received: from slicker.dcs.gla.ac.uk ([130.209.242.51] helo=dcs.gla.ac.uk) by iona.dcs.gla.ac.uk with esmtp (Exim 3.13 #1) id 16MW8Q-0001Tx-00 for infrastructures@terraluna.org; Fri, 04 Jan 2002 15:19:55 +0000 To: infrastructures@terraluna.org Date: Fri, 04 Jan 2002 15:19:53 +0000 From: Will Partain Message-Id: Subject: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Greetings, infrastructural folks! Great to meet some of you at LISA. I have been musing slightly about doing backups without tapes, and pass along my current notes (below) for your comment and criticism. (Once we get beyond the high-level issues that kinda make sense for the infrastructures list, we should probably chat elsewhere -- I'd suggest ark-dev [http://lists.sf.net/mailman/listinfo/ark-dev], as that's where I'll probably do any detailed implementation chit-chat (if it happens).) Will ========================= A tapeless backups system for a workgroup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ At a LISA 2001 session on "recovery-oriented computing" [1], Dave Patterson (Berkeley) commented that "tape is dead". Reason: if tape doesn't cost more per byte than disk yet, it soon will. Quick check (2002.01): 120GB EIDE disk for $208 (Pricewatch) = $1.73/GB, 40GB DLT IV tape for $53 (Pricescan) = $1.32/GB. Close enough for me! And, if you haven't noticed lately, tape is a *pain*. So, what can we do backup-wise [2] (NB: not `archiving'... [3]) using only cheap disks? Assumptions: * I have a place "far away" (a fire won't get there...) on my intranet where I can park a machine with disks hanging off of it. * My "workgroup" world isn't *that* big... Nor is it a nothing-must-ever-be-down-ever-no-matter-what environment. (Even in such an environment, where you will probably RAIDify every disk-bit, you still need backups...) Terminology: `disk chunk' [4] - a logically-coherent chunk of stuff on disk "in one piece" (i.e. under a single directory). Examples: "Fred's home directory" (/home/fred); "where we keep our source tarballs" (/system/open-src); ... [If you have a record of all of your 'disk chunks', you can generate your automount maps (for example).] `replica' - a copy of a disk chunk. The `primary' replica is the one you really use (perhaps read-write); all other (`secondary') replicas of a disk chunk are read-only (if made available at all). So, the idea is very simple: rsync [5] copies of your disk chunks to a remote diskful machine in an organized way. Those remote chunks (replicas) are then available themselves (read-only). Details: * A five-nights-a-week thing rsyncs all live 'disk chunks' to a remote diskful box. This set of (secondary) replicas are available (read-only) through a /yesterday automount map, using the following mapping: primary replica called... secondary called... /home/partain /yesterday/home--partain /system/open-src /yesterday/system--open-src (and so on) * A weekly (every Sunday?) thing rsyncs all live disk chunks to a remote diskful box. These sets of (secondary) replicas are available (read-only) through automount maps /weekend-1, /weekend-2, /weekend-3, and /weekend-4. (If there's a fifth Sunday, I think I just won't bother.) Thus, my home directory at the first weekend of the month: /weekend-1/home--partain * System-y bits (/usr, /, /opt) -- stuff that came straight from the vendor -- are not backed up. (You could do differently, but we're inclined to just rebuild the machine.) That's, er, um, *it*! Use cases: * User loses a file? They can go find it themselves, in /yesterday and/or /weekend-{1,2,3,4}. * Data disk dies? Either: (a) Copy the stuff from /yesterday to some other disk, creating a new primary replica; adjust automount maps accordingly; or (b) Switch affected people's machines' automount maps to point to the /yesterday replica (now read-write :-). Replace disk at leisure and shuffle things about. * System disk dies? (machine now dead) (a) [unlikely] Physical move of the data disk to another machine; adjust, season, enjoy; or (b) For all the primary disk-chunk replicas that were on this machine, as for the data-disk case. Fix machine at leisure. Note: for critical machines, it may be worth having a spare system disk ready to go. * Building burns down? You've got all the bits far away. Finer details: * rsync over ssh: yes * It's better that the "diskful boxes" *pull* the rsync copies, rather than having every client push to them. * Yes, we'll use the rsync 'excludes' mechanism to avoid backing up large recreatable files. * Database-y disk chunks (e.g. Oracle databases, ClearCase VOBs, ...): have enough smarts to wrap the rsyncs in a `lock'/`unlock' pair. * Can 'cut over' from a traditional tape-based backup scheme gradually (a disk or machine at a time...). * Need some kind of tidy-up mechanism: e.g. if a /weekend-1 replica didn't get made correctly on the Sunday, perhaps should re-try on the Monday? (Or just live with a not-quite-right /weekend-1 replica?) Variants: * Have the nightly rsync go to a *local* diskful box; it is then really quite painless to switch a user/system to that copy (no WAN delays). The remote one could then be /day-before-yesterday, taken from /yesterday before it gets refreshed... (Synchronization issues there...) * If you just can't get over that tape feeling, attach a tape drive to the remote diskful boxes, and take/keep copies of the /weekend-1 replicas. Comments?? What have I forgotten/overlooked? Notes: [1] http://roc.cs.berkeley.edu/ [2] `Backing up' solves problems ranging from `I deleted my cookie recipe' to `The building burned down'. [3] `Archiving' takes copies of logical entities (`the project that just finished', `the month-end accounts') to preserve "forever", perhaps for legal reasons. [4] `Disk chunk' (or 'dchunk' [pronounced "duh-*chunk*"]) is an Arusha Project (Sidai team) term. [5] http://rsync.samba.org/ == end From oetiker@ee.ethz.ch Fri Jan 4 07:59:57 2002 Received: from tardis.ee.ethz.ch (tardis-delek-fast.ee.ethz.ch [129.132.2.199]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA30579 for ; Fri, 4 Jan 2002 07:59:56 -0800 Received: from nova.ee.ethz.ch (nova.ee.ethz.ch [129.132.7.86]) by tardis.ee.ethz.ch (Postfix) with ESMTP id 5EAC33C336; Fri, 4 Jan 2002 16:59:54 +0100 (MET) Date: Fri, 4 Jan 2002 16:59:54 +0100 (MET) From: Tobias Oetiker To: Will Partain Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Today Will Partain wrote: > Greetings, infrastructural folks! Great to meet some of > you at LISA. > > I have been musing slightly about doing backups without > tapes, and pass along my current notes (below) for your > comment and criticism. > > And, if you haven't noticed lately, tape is a *pain*. Hi Will, One property I like about tapes very much is that they do not usually die as a whole but rather slowly degrade gracefully ... there is no controler in the tape ... Our take at the problem is, that we provide a HSM system for the whole department (30 TB) based on a Solaris Box with an IBM Robot and Magstar tapedrives. On the solaris box we run LSCIs SAMFS www.lsci.com (recently acquired by sun) this software makes the space on the tapes appear as one big filesystem to the users ... All labs run their backups to this filesystem either via ufsdump or some other means (ntbackup via samba works as well) ... We are very succesful with the service check http://jabba.ethz.ch for stats ... cheers tobi > So, what can we do backup-wise [2] (NB: not > `archiving'... [3]) using only cheap disks? Assumptions: > > * I have a place "far away" (a fire won't get there...) on > my intranet where I can park a machine with disks hanging > off of it. > > * My "workgroup" world isn't *that* big... Nor is it a > nothing-must-ever-be-down-ever-no-matter-what environment. > (Even in such an environment, where you will probably > RAIDify every disk-bit, you still need backups...) > > Terminology: > > `disk chunk' [4] - a logically-coherent chunk > of stuff on disk "in one piece" (i.e. under a single > directory). Examples: "Fred's home directory" (/home/fred); > "where we keep our source tarballs" (/system/open-src); ... > > [If you have a record of all of your 'disk chunks', you > can generate your automount maps (for example).] > > `replica' - a copy of a disk chunk. The `primary' replica > is the one you really use (perhaps read-write); all other > (`secondary') replicas of a disk chunk are read-only (if > made available at all). > > So, the idea is very simple: rsync [5] copies of your disk > chunks to a remote diskful machine in an organized way. > Those remote chunks (replicas) are then available themselves > (read-only). Details: > > * A five-nights-a-week thing rsyncs all live 'disk chunks' > to a remote diskful box. This set of (secondary) replicas > are available (read-only) through a /yesterday automount > map, using the following mapping: > > primary replica called... secondary called... > > /home/partain /yesterday/home--partain > /system/open-src /yesterday/system--open-src > (and so on) > > * A weekly (every Sunday?) thing rsyncs all live disk chunks > to a remote diskful box. These sets of (secondary) > replicas are available (read-only) through automount maps > /weekend-1, /weekend-2, /weekend-3, and /weekend-4. (If > there's a fifth Sunday, I think I just won't bother.) > Thus, my home directory at the first weekend of the > month: /weekend-1/home--partain > > * System-y bits (/usr, /, /opt) -- stuff that came straight > from the vendor -- are not backed up. (You could do > differently, but we're inclined to just rebuild the > machine.) > > That's, er, um, *it*! Use cases: > > * User loses a file? They can go find it themselves, in > /yesterday and/or /weekend-{1,2,3,4}. > > * Data disk dies? Either: (a) Copy the stuff from > /yesterday to some other disk, creating a new primary > replica; adjust automount maps accordingly; or (b) Switch > affected people's machines' automount maps to point to the > /yesterday replica (now read-write :-). Replace disk at > leisure and shuffle things about. > > * System disk dies? (machine now dead) (a) [unlikely] > Physical move of the data disk to another machine; adjust, > season, enjoy; or (b) For all the primary disk-chunk > replicas that were on this machine, as for the data-disk > case. Fix machine at leisure. Note: for critical > machines, it may be worth having a spare system disk ready > to go. > > * Building burns down? You've got all the bits far away. > > Finer details: > > * rsync over ssh: yes > > * It's better that the "diskful boxes" *pull* the rsync > copies, rather than having every client push to them. > > * Yes, we'll use the rsync 'excludes' mechanism to avoid > backing up large recreatable files. > > * Database-y disk chunks (e.g. Oracle databases, ClearCase > VOBs, ...): have enough smarts to wrap the rsyncs in > a `lock'/`unlock' pair. > > * Can 'cut over' from a traditional tape-based backup scheme > gradually (a disk or machine at a time...). > > * Need some kind of tidy-up mechanism: e.g. if a /weekend-1 > replica didn't get made correctly on the Sunday, perhaps > should re-try on the Monday? (Or just live with a > not-quite-right /weekend-1 replica?) > > Variants: > > * Have the nightly rsync go to a *local* diskful box; it is > then really quite painless to switch a user/system to that copy > (no WAN delays). > > The remote one could then be /day-before-yesterday, taken > from /yesterday before it gets > refreshed... (Synchronization issues there...) > > * If you just can't get over that tape feeling, attach a > tape drive to the remote diskful boxes, and take/keep > copies of the /weekend-1 replicas. > > Comments?? What have I forgotten/overlooked? > > Notes: > > [1] http://roc.cs.berkeley.edu/ > > [2] `Backing up' solves problems ranging from `I deleted my > cookie recipe' to `The building burned down'. > > [3] `Archiving' takes copies of logical entities (`the > project that just finished', `the month-end accounts') > to preserve "forever", perhaps for legal reasons. > > [4] `Disk chunk' (or 'dchunk' [pronounced "duh-*chunk*"]) is > an Arusha Project (Sidai team) term. > > [5] http://rsync.samba.org/ > > == end > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures > -- ______ __ _ /_ __/_ / / (_) Oetiker, ETZ J97, ETH, 8092 Zurich, Switzerland / // _ \/ _ \/ / phoneto:+41(0)1-632-5286 faxto:+41(0)1-632-1517 /_/ \.__/_.__/_/ mailto:oetiker@ee.ethz.ch http://people.ee.ethz.ch/~oetiker From don.faulkner@infosec.spectria.com Fri Jan 4 17:29:16 2002 Received: from strider (cdm-37-194.sprd.tcac.net [208.180.37.194]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA12555 for ; Fri, 4 Jan 2002 17:29:08 -0800 Received: from def by strider with local (Exim 3.33 #1 (Debian)) id 16Mfdy-0006Z4-00 for ; Fri, 04 Jan 2002 19:29:06 -0600 Date: Fri, 4 Jan 2002 19:29:06 -0600 From: Don Faulkner To: infrastructures@terraluna.org Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Message-ID: <20020105012906.GB24873@infosec.spectria.com> Mail-Followup-To: infrastructures@terraluna.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.25i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, Jan 04, 2002 at 03:19:53PM +0000, Will Partain wrote: > * A five-nights-a-week thing rsyncs all live 'disk chunks' > to a remote diskful box. This set of (secondary) replicas > are available (read-only) through a /yesterday automount > map, using the following mapping: > > primary replica called... secondary called... > > /home/partain /yesterday/home--partain > /system/open-src /yesterday/system--open-src > (and so on) > The one thing (probably the only thing!) I liked about tapes was the idea of differential and incremental backups. Now, I'm one of these frugal types, in that I prefer not to back up the same bits twice. I prefer diffs, deltas, and other similar mechanisims, especially when there's a convienent (and "intuitive") mechanisim to use them. The trouble with diffs, of course, is that you can't directly access a diff'd file. I seem to remember an experimental file system or something from a few years ago that would do this transparently, so that you could access any revision of a file. I guess now I have a reason to go find that again. -- Don Faulkner, KB5WPM | | "All that is gold does not glitter." ( This space unintentionally | "not all those who wander are lost." left blank ) | --J.R.R. Tolkien From bryang@soccer.fc.hp.com Sat Jan 5 07:32:00 2002 Received: from atlrel7.hp.com (atlrel7.hp.com [156.153.255.213]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA25927 for ; Sat, 5 Jan 2002 07:31:46 -0800 Received: from soccer.fc.hp.com (soccer.fc.hp.com [15.1.42.73]) by atlrel7.hp.com (Postfix) with ESMTP id 4917CE007AA for ; Sat, 5 Jan 2002 10:31:07 -0500 (EST) Received: (from bryang@localhost) by soccer.fc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id IAA20156 for infrastructures@terraluna.org; Sat, 5 Jan 2002 08:30:52 -0700 (MST) From: Bryan Gartner Message-Id: <200201051530.IAA20156@soccer.fc.hp.com> Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) To: infrastructures@terraluna.org Date: Sat, 05 Jan 2002 8:30:51 MST In-Reply-To: ; from "Will Partain" at Jan 4, 102 12:10 (noon) Organization: Hewlett-Packard Company Entity: Computing Systems IT, Fort Collins Phone: (970 or telnet) 898-2470 X-Url: http://ncs-web.fc.hp.com/~bryang/ X-Mailer: Elm [revision: 212.2] Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Will, > I have been musing slightly about doing backups without > tapes, and pass along my current notes (below) for your > comment and criticism. I also have been toying with this notion for awhile now. An extension to what you provided is to have the remote disk farm using a journal-based file system that can also perform snapshots. In this way, the remote system has views of what changed during each snapshot period, and given enough disk space can be retained according to your desires/budget. Plus, with the right rsyncd.conf (ie. managed files) on both ends, the user could simply "restore" the data from any period themselves. All of this is really intended for an environment like you cited with a good OS/Data distinction, and the intent here was the OS would be rebuilt quicker than could be restored. The only exception would possibly be for files yet to be managed which would be linked from the OS partition into the Data partition. > * Building burns down? You've got all the bits far away. Of course if the network latency becomes a huge barrier and you can afford the luxury of yet another replica, layer either AFS or Coda onto the remote rsync data farm and you can replicate very nicely across the WAN. bryang From dert@eecs.tufts.edu Sat Jan 5 08:12:55 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA14422 for ; Sat, 5 Jan 2002 08:12:55 -0800 Received: from piano.eecs.tufts.edu (Piano.EECS.Tufts.EDU [130.64.23.40]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 2BD671A6FC for ; Sat, 5 Jan 2002 11:12:54 -0500 (EST) Received: by piano.eecs.tufts.edu (Postfix, from userid 13132) id C55A0AF9E; Sat, 5 Jan 2002 11:12:53 -0500 (EST) Date: Sat, 5 Jan 2002 11:12:53 -0500 From: Andy Davidoff To: infrastructures@terraluna.org Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Message-ID: <20020105111253.A11239@eecs.tufts.edu> Reply-To: dert@pobox.com References: <200201051530.IAA20156@soccer.fc.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.2i In-Reply-To: <200201051530.IAA20156@soccer.fc.hp.com>; from bryang@soccer.fc.hp.com on Sat, Jan 05, 2002 at 08:30:51AM -0700 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I just met with some salesweasels from NetApp a couple weeks ago. The have a new near-line storage product built from N 160gig EIDE disks. The performance is secondary to capacity in this offering, but it's a commercial implementation of what many of us have been playing with in the back rooms. I think the densities start at around 20tb. Personally, I like the gee-whiz factor of mixed-media filesystems, but they aren't as robust as a filer and don't offer anywhere near the performance and features on the back-end. When I can't justify the NetApp, I'll go with a fast RAID for on-line cache and a DVD changer for near-line. Mix 'em up with the aforementioned LSCI SAM-FS or OTG's competitive product for an archiving filesystem. It all depends on your network; do you need speed and snapshots or do you need raw capacity above all else. #if Bryan Gartner /* Jan 05, 08:30 */ > An extension to what you provided is to have the remote disk farm > using a journal-based file system that can also perform snapshots. > In this way, the remote system has views of what changed during > each snapshot period, and given enough disk space can be retained > according to your desires/budget. Plus, with the right rsyncd.conf > (ie. managed files) on both ends, the user could simply "restore" > the data from any period themselves. #endif /* bryang@soccer.fc.hp.com */ -- Andy Davidoff Sen. Unix SysAdmin Tufts University From joelh@cyberzod.com Mon Jan 7 10:36:22 2002 Received: from cyberzod.com (12-253-25-167.client.attbi.com [12.253.25.167]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA28116 for ; Mon, 7 Jan 2002 10:36:20 -0800 Received: from saber (saber.nabs.com [192.168.128.7]) by cyberzod.com (8.9.3/8.9.3) with SMTP id LAA09928 for ; Mon, 7 Jan 2002 11:36:16 -0700 From: "Joel Huddleston" To: Subject: RE: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Date: Mon, 7 Jan 2002 11:33:09 -0700 Message-ID: <003801c197a9$c1b953a0$0780a8c0@nabs.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 In-Reply-To: Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Since this specter has been raised, I'd like to vent briefly (if that is within my abilities) on a property of every backup system I have reviewed to date that makes them unacceptable. The crux of my complaint is, backup is useless; restore is invaluable. Yet, every vendor spends 99% effort on backup and 1% (if that) on restore. (OK, in some cases 40% on backup, 65% on marketing and 5% on restore.) My ideal data integrity system is one in which, when I have to recover from disaster, I am not dragging out manuals and screwing around with some arcane restore system. I am not reinstalling the OS and then reinstalling the backup software including some kind of host key and THEN recovering over both of them. A data integrity system should provide: 1) Near zero effort, high speed, full restores without reinstall of ANY software 2) End-user interface for backup now and restore now 3) Low system impact backup 4) Differential, Incremental and Full backups 4a) Incremental backups that understand that: restore v1, v1.1, v1.2 is not the same as restore v1.2 (files deleted between incrementals are universally restored and kept, occasionally filling otherwise low-utilization file systems, grr.) 5) A host level comprehension of backup policies, status and configuration metadata that is available when the host itself is gone (e.g. what disks existed, what were their SCSI/Fiber addresses, how were they partitioned, were they VMed and so on) 6) Encryption on the wire 7) Available encryption on the media using public/private keys (these should be end-user maintainable, selectable by owner, group and/or directory and "secure" from system personnel) In order to satisfy condition #1, the backup vendor must provide bootable media that can perform a full restore to any file system they support for backup. That means FAT, FAT32, NTFS, ufs, 64bit ufs, vfs, jfs and so on. Is anyone aware of such a system? Of course not, it does not exist. I am not even aware of such a system that is available for a single platform from a single vendor, never mind multi-vendor heterogeneity. I have come to the point in my venting where I believe that if I continue, it is no longer brief. I'd really appreciate any comments or corrections the group can provide to my comments above. If anyone can suggest a framework that (even partially) satisfies condition #1 above, please tell us what it is. I'll happily proselytize to the net any solution that comes close. -- Joel Huddleston eka joelh@cyberzod.com > -----Original Message----- > From: infrastructures-admin@terraluna.org > [mailto:infrastructures-admin@terraluna.org]On Behalf Of Will Partain > Sent: Friday, January 04, 2002 8:20 AM > To: infrastructures@terraluna.org > Subject: [Infrastructures] a tapeless backups system for a workgroup > (drafty proposal v1) > > > Greetings, infrastructural folks! Great to meet some of > you at LISA. > > I have been musing slightly about doing backups without > tapes, and pass along my current notes (below) for your > comment and criticism. > > (Once we get beyond the high-level issues that kinda make > sense for the infrastructures list, we should probably chat > elsewhere -- I'd suggest ark-dev > [http://lists.sf.net/mailman/listinfo/ark-dev], as that's > where I'll probably do any detailed implementation chit-chat > (if it happens).) > > Will > > ========================= > > A tapeless backups system for a workgroup > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > At a LISA 2001 session on "recovery-oriented computing" [1], > Dave Patterson (Berkeley) commented that "tape is dead". > Reason: if tape doesn't cost more per byte than disk yet, it > soon will. Quick check (2002.01): 120GB EIDE disk for $208 > (Pricewatch) = $1.73/GB, 40GB DLT IV tape for $53 > (Pricescan) = $1.32/GB. Close enough for me! > > And, if you haven't noticed lately, tape is a *pain*. > > So, what can we do backup-wise [2] (NB: not > `archiving'... [3]) using only cheap disks? Assumptions: > > * I have a place "far away" (a fire won't get there...) on > my intranet where I can park a machine with disks hanging > off of it. > > * My "workgroup" world isn't *that* big... Nor is it a > nothing-must-ever-be-down-ever-no-matter-what environment. > (Even in such an environment, where you will probably > RAIDify every disk-bit, you still need backups...) > > Terminology: > > `disk chunk' [4] - a logically-coherent chunk > of stuff on disk "in one piece" (i.e. under a single > directory). Examples: "Fred's home directory" (/home/fred); > "where we keep our source tarballs" (/system/open-src); ... > > [If you have a record of all of your 'disk chunks', you > can generate your automount maps (for example).] > > `replica' - a copy of a disk chunk. The `primary' replica > is the one you really use (perhaps read-write); all other > (`secondary') replicas of a disk chunk are read-only (if > made available at all). > > So, the idea is very simple: rsync [5] copies of your disk > chunks to a remote diskful machine in an organized way. > Those remote chunks (replicas) are then available themselves > (read-only). Details: > > * A five-nights-a-week thing rsyncs all live 'disk chunks' > to a remote diskful box. This set of (secondary) replicas > are available (read-only) through a /yesterday automount > map, using the following mapping: > > primary replica called... secondary called... > > /home/partain /yesterday/home--partain > /system/open-src /yesterday/system--open-src > (and so on) > > * A weekly (every Sunday?) thing rsyncs all live disk chunks > to a remote diskful box. These sets of (secondary) > replicas are available (read-only) through automount maps > /weekend-1, /weekend-2, /weekend-3, and /weekend-4. (If > there's a fifth Sunday, I think I just won't bother.) > Thus, my home directory at the first weekend of the > month: /weekend-1/home--partain > > * System-y bits (/usr, /, /opt) -- stuff that came straight > from the vendor -- are not backed up. (You could do > differently, but we're inclined to just rebuild the > machine.) > > That's, er, um, *it*! Use cases: > > * User loses a file? They can go find it themselves, in > /yesterday and/or /weekend-{1,2,3,4}. > > * Data disk dies? Either: (a) Copy the stuff from > /yesterday to some other disk, creating a new primary > replica; adjust automount maps accordingly; or (b) Switch > affected people's machines' automount maps to point to the > /yesterday replica (now read-write :-). Replace disk at > leisure and shuffle things about. > > * System disk dies? (machine now dead) (a) [unlikely] > Physical move of the data disk to another machine; adjust, > season, enjoy; or (b) For all the primary disk-chunk > replicas that were on this machine, as for the data-disk > case. Fix machine at leisure. Note: for critical > machines, it may be worth having a spare system disk ready > to go. > > * Building burns down? You've got all the bits far away. > > Finer details: > > * rsync over ssh: yes > > * It's better that the "diskful boxes" *pull* the rsync > copies, rather than having every client push to them. > > * Yes, we'll use the rsync 'excludes' mechanism to avoid > backing up large recreatable files. > > * Database-y disk chunks (e.g. Oracle databases, ClearCase > VOBs, ...): have enough smarts to wrap the rsyncs in > a `lock'/`unlock' pair. > > * Can 'cut over' from a traditional tape-based backup scheme > gradually (a disk or machine at a time...). > > * Need some kind of tidy-up mechanism: e.g. if a /weekend-1 > replica didn't get made correctly on the Sunday, perhaps > should re-try on the Monday? (Or just live with a > not-quite-right /weekend-1 replica?) > > Variants: > > * Have the nightly rsync go to a *local* diskful box; it is > then really quite painless to switch a user/system to that copy > (no WAN delays). > > The remote one could then be /day-before-yesterday, taken > from /yesterday before it gets > refreshed... (Synchronization issues there...) > > * If you just can't get over that tape feeling, attach a > tape drive to the remote diskful boxes, and take/keep > copies of the /weekend-1 replicas. > > Comments?? What have I forgotten/overlooked? > > Notes: > > [1] http://roc.cs.berkeley.edu/ > > [2] `Backing up' solves problems ranging from `I deleted my > cookie recipe' to `The building burned down'. > > [3] `Archiving' takes copies of logical entities (`the > project that just finished', `the month-end accounts') > to preserve "forever", perhaps for legal reasons. > > [4] `Disk chunk' (or 'dchunk' [pronounced "duh-*chunk*"]) is > an Arusha Project (Sidai team) term. > > [5] http://rsync.samba.org/ > > == end > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures > From cwcjr@bestweb.net Tue Jan 8 12:16:23 2002 Received: from pimout3-int.prodigy.net (pimout3-ext.prodigy.net [207.115.63.102]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA12128 for ; Tue, 8 Jan 2002 12:16:22 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout3-int.prodigy.net (8.11.0/8.11.0) with SMTP id g08KGKH248282; Tue, 8 Jan 2002 15:16:20 -0500 Message-ID: <000701c19881$56908700$c5c0a8c0@prodigy.com> From: "Will Cox" To: "Joel Huddleston" , References: <003801c197a9$c1b953a0$0780a8c0@nabs.com> Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Date: Tue, 8 Jan 2002 15:16:20 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Just ran across this item addressing bare metal restore problems: http://www.tkg.com/products/baremetal/baremetal.php /cwc From: "Joel Huddleston" > The crux of my complaint is, backup is > useless; restore is invaluable. From duprec@scorec.rpi.edu Tue Jan 8 12:55:08 2002 Received: from mail.rpi.edu (root@mail.rpi.edu [128.113.22.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA32423 for ; Tue, 8 Jan 2002 12:55:07 -0800 Received: from smtp.scorec.rpi.edu (smtp.scorec.rpi.edu [128.113.123.156]) by mail.rpi.edu (8.11.3/8.11.3) with ESMTP id g08Kt4543554 for ; Tue, 8 Jan 2002 15:55:05 -0500 Received: from mastermind.scorec.rpi.edu (mastermind.scorec.rpi.edu [10.0.5.17]) by smtp.scorec.rpi.edu (8.10.2/8.10.2) with ESMTP id g08KssK13641 for ; Tue, 8 Jan 2002 15:54:54 -0500 (EST) Date: Tue, 8 Jan 2002 15:54:54 -0500 (EST) From: Christophe Dupre To: In-Reply-To: <200201082001.MAA04494@roton.terraluna.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.1 Subject: [Infrastructures] Re: Infrastructures digest, Vol 1 #6 - 2 msgs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: While having a backup system that does everything you list in your mail below would be nice, I don't know how much of a real gain you'd get. > From: "Joel Huddleston" > To: > Subject: RE: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) > Date: Mon, 7 Jan 2002 11:33:09 -0700 > > Since this specter has been raised, I'd like to vent briefly (if that is > within my abilities) on a property of every backup system I have reviewed to > date that makes them unacceptable. The crux of my complaint is, backup is > useless; restore is invaluable. Yet, every vendor spends 99% effort on > backup and 1% (if that) on restore. (OK, in some cases 40% on backup, 65% on > marketing and 5% on restore.) > > My ideal data integrity system is one in which, when I have to recover from > disaster, I am not dragging out manuals and screwing around with some arcane > restore system. I am not reinstalling the OS and then reinstalling the > backup software including some kind of host key and THEN recovering over > both of them. A data integrity system should provide: > > 1) Near zero effort, high speed, full restores without reinstall of ANY > software Use ufsdump. Boot from the network or cdrom, and ufsrestore. It may not be pretty, or user-friendly, but it will get the job done. > 2) End-user interface for backup now and restore now Don't most backup software do that already ? > 3) Low system impact backup What do you mean by low-impact ? > 4) Differential, Incremental and Full backups That's also very common. > 4a) Incremental backups that understand that: restore v1, v1.1, v1.2 is not > the same as restore v1.2 (files deleted between incrementals are universally > restored and kept, occasionally filling otherwise low-utilization file > systems, grr.) Netbackup's True Image Recovery feature does that, at the expense of disk space for a database. > 5) A host level comprehension of backup policies, status and configuration > metadata that is available when the host itself is gone (e.g. what disks > existed, what were their SCSI/Fiber addresses, how were they partitioned, > were they VMed and so on) That's not for the backup system, that's for your rebuilding infrastructure. The backup should be media-independant, as you want to be able to restore to a different machine the data was backed up from. That's the major gripe I have with NAS and the NDP protocol - not device-independant. In a controlled environments, it's quicker to just rebuild a box than to try to recover the OS from tape. In our case here, that rebuilding includes the backup client. Once the box is rebuilt, all the data can be recovered from tape from the backup server. If it's the backup server that died, well we can take the few more hours required to recover the backup system from backup with a clean copy, as that machine doesn't do anything else. As for the efforts from vendor - we spend most of our time doing backup, and comparatively little time doing restores, so it follows that that's where they spend their time. For the encryption, I agree with you - I haven't found a system that does decent encryption with any reasonable amount of ease of administration. Work is definitely required in this area. -- Christophe Dupre System Administrator, Scientific Computation Research Center Rensselaer Polytechnic Institute Troy, NY USA Phone: (518) 276-2578 - Fax: (518) 276-4886 From jmr@computing.com Tue Jan 8 13:13:55 2002 Received: from zuum.computing.com (cs2869-253.austin.rr.com [24.28.69.253]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA09982 for ; Tue, 8 Jan 2002 13:13:54 -0800 From: jmr@computing.com Received: from zuum.computing.com.computing.com (localhost [127.0.0.1]) by zuum.computing.com (8.11.6/8.11.6) with ESMTP id g08LDgO26303; Tue, 8 Jan 2002 15:13:43 -0600 (CST) (envelope-from jmr@computing.com) Date: Tue, 08 Jan 2002 15:13:42 -0600 Message-ID: <87hepwk0l5.wl@zuum.computing.com> To: "Joel Huddleston" Cc: Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) In-Reply-To: <003801c197a9$c1b953a0$0780a8c0@nabs.com> References: <003801c197a9$c1b953a0$0780a8c0@nabs.com> User-Agent: Wanderlust/2.4.1 (Stand By Me) SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Joel: I think your rant is right on target! I don't personally know anything about this one... but: In today's paper there was an article about TKG (The Kernel Group) having been acquired by someone. TKG has apparently developed something they call "Bare Metal Restore". Sounds like they're aiming for the same target... You might check it out. TKG is in Austin. -- Jim Rowan jmr@computing.com DCSI (512) 374-1143 From partain@dcs.gla.ac.uk Wed Jan 9 09:00:50 2002 Received: from iona.dcs.gla.ac.uk (iona.dcs.gla.ac.uk [130.209.240.35]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id JAA08474 for ; Wed, 9 Jan 2002 09:00:49 -0800 Received: from slicker.dcs.gla.ac.uk ([130.209.242.51] helo=dcs.gla.ac.uk) by iona.dcs.gla.ac.uk with esmtp (Exim 3.13 #1) id 16OM5o-0003LU-00 for infrastructures@terraluna.org; Wed, 09 Jan 2002 17:00:48 +0000 To: infrastructures@terraluna.org X-Mailer: mh-e 6.0; nmh 1.0.4; Emacs 21.4 Date: Wed, 09 Jan 2002 17:00:47 +0000 From: Will Partain Message-Id: Subject: [Infrastructures] Re: a tapeless backups system ... Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Folks, thanks for the many useful comments that have come in re my "tapeless backups proposal" (prev msg). It is my faint hope to consolidate the info and have it appear on a Web page (probably in Arusha land); I will contact authors before stealing their words :-) But to back up a bit (as it were)... At least one shy reader took me to task for my numbers, asserting that $/GB for tape is still ~3x cheaper than disk. (Yes, I admit it, I don't know what I'm talking about :-) Rather than argue about that (though good references welcome!), perhaps it would be useful to collect a few back-of-envelope *sketches* of what people would do for backups, assuming they were starting fresh today... Something like... So, for the scenario I've been thinking about: 300GB of non-system data to be backed up; Shopping list: Price Items $3500 15 x 120GB cheap disks = 300GB (approx) x 5 backup copies $2000 a few PCs to hang them on Park the PCs+disks at my pal's place on the intranet and "backup" using rsync+ssh. The nightly rsync available as /yesterday, and four weekend ones are available as /weekend-[1234]. If some of these are forthcoming, I will add them to the putative document. Thanks as ever, Will From dert@eecs.tufts.edu Wed Jan 9 11:25:03 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id LAA18643 for ; Wed, 9 Jan 2002 11:25:03 -0800 Received: from sensei.EECS.Tufts.EDU (Sensei.EECS.Tufts.EDU [130.64.23.173]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 658191A50B; Wed, 9 Jan 2002 14:25:02 -0500 (EST) Received: by sensei.EECS.Tufts.EDU (Postfix, from userid 13132) id 07028F6123; Wed, 9 Jan 2002 14:25:01 -0500 (EST) Date: Wed, 9 Jan 2002 14:25:01 -0500 From: Andy Davidoff To: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: a tapeless backups system ... Message-ID: <20020109142501.A26020@eecs.tufts.edu> Reply-To: andy+infrastructures@cs.tufts.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from partain@dcs.gla.ac.uk on Wed, Jan 09, 2002 at 05:00:47PM +0000 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I'm still seeing some rough edges in this system... With tape, I can align full-backups on tape boundaries so archiving a full-backup for archival storage is easy; just swap n tapes from the library. With a disk-based solution, I'd need to build a separate removable array specifically for each autonomous replica. If you use a cheap disk array for backups, be prepared to lose a disk in the array. You should also be prepared for someone to ask why they are spending the extra bucks to put random-access media in storage. If you never archive backups, this isn't an issue. If you do, I can't see why you wouldn't want to stick with tape. At that point, the disk array becomes most relevant as a snapshot server. Ideally, /my/ only backups are the periodic tape archives I create for off-site storage. The rest of my backup needs are handled by snapshots. If you turn this proposal into the engineering of a "free" snapshot server, I think you'll find much more support and a LISA paper or two to boot. #if Will Partain /* Jan 09, 17:00 */ > So, for the scenario I've been thinking about: > > 300GB of non-system data to be backed up; Shopping list: > Price Items > $3500 15 x 120GB cheap disks = 300GB (approx) x 5 backup copies > $2000 a few PCs to hang them on > > Park the PCs+disks at my pal's place on the intranet and "backup" > using rsync+ssh. The nightly rsync available as /yesterday, > and four weekend ones are available as /weekend-[1234]. #endif /* partain@dcs.gla.ac.uk */ -- Andy Davidoff Sen. Unix SysAdmin Tufts University From luke@madstop.com Wed Jan 30 21:00:41 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA05365 for ; Wed, 30 Jan 2002 21:00:40 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g0V4uEH8025456; Wed, 30 Jan 2002 22:56:14 -0600 (CST) Date: Wed, 30 Jan 2002 22:56:14 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Joel Huddleston cc: infrastructures@terraluna.org Subject: RE: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) In-Reply-To: <003801c197a9$c1b953a0$0780a8c0@nabs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 7 Jan 2002, Joel Huddleston wrote: > 1) Near zero effort, high speed, full restores without reinstall of ANY > software [SNIP] > In order to satisfy condition #1, the backup vendor must provide bootable > media that can perform a full restore to any file system they support for > backup. That means FAT, FAT32, NTFS, ufs, 64bit ufs, vfs, jfs and so on. > > Is anyone aware of such a system? Of course not, it does not exist. I am > not even aware of such a system that is available for a single platform from > a single vendor, never mind multi-vendor heterogeneity. I know I'm jumping onto the end of a long-gone train here, but I just noticed this thread... I don't have much comment about most of your rant, largely because I agree and have been mostly able to avoid the backup monkey, but HP-UX actually can make a bootable tape ofa system. I can't personally vouch for its efficacy, because I haven't used it, but considering how pervasive it appears to be within the HP-UX "community" (I'm not convinced there actually is one), it's apparently pretty decent. You do what they call a "make_recovery", which makes a bootable tape which will restore "the entire system". I'm assuming that there are some limits to this definition, because most tapes won't store what we would consider the entire system, but I don't know how hit draws the lines. I'm assuming you can decide that at creation time. I know my shop is in the habit of doing this on maintenance weekends, and everytime I mention Ignite-UX (HP's install tool, kind of a bloated, insecure, GUI version of jumpstart with a bunch of stuff that makes it hard to use), people think I am only talking about make_recovery. In fact, I recently had some trouble convincing some relatively high-level HP techs that Ignite-UX could even do bare-metal HP-UX installs. The point is, though, HP-UX has such a tool, or something similar, and many people who use HP-UX appear to use the tool. Obviously, though, its cross-platform ability is probably nil. And, of course, one of the biggest failings of the product is that it depends on being able to dump to a single tape. It would be best if make_recovery worked from within a backup program to create a bootable tape of a system _from_backups_, so that you didn't have to make a recovery tape unless you actually needed to recover. -- Tobacco stocks have taken a big tumble," says Jay Leno. "Phillip Morris fell 6 points. They lost so much money they may have to lay off two senators." From luke@madstop.com Wed Jan 30 21:42:53 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA27976 for ; Wed, 30 Jan 2002 21:42:52 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g0V5cXYm025553 for ; Wed, 30 Jan 2002 23:38:33 -0600 (CST) Date: Wed, 30 Jan 2002 23:38:33 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] Sysadmin collaboration tool Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi all, Since I've broken the ice with my previous comment on tape recovery, I figured I'd go ahead and toss this out. I spoke with some of you about this at LISA, but for the rest of you... Steve and I ran a BoF together at LISA, discussing two basic themes. The reason we ran it together was because he and I realized that we had scheduled two different BoFs for the same time, we each wanted to go to the other's, and there seemed to be a significant amount of cross-over. One of the topics, unsurprisingly, was directly related to this list--the next version of isconf, and basic infrastructure concepts. The other topic, my topic, was building an online sysadmin collaboration tool. I use the word tool here because I don't really have a better word. To make a long story short, here is a conceptual summary of what I am looking for: ---------------------- We propose to build an online tool with the specific purpose of providing a means for sysadmins to collaborate on all aspects of system administration. This document is not inclusive and will be updated as the tool is developed. The assumptions which provide the premise for this online tool are the following: -Most problems we as sysadmins encounter are problems that most sysadmins encounter -The current state of system administration is limited by the lack of pervasive collaboration between sysadmins -The current lack of collaboration largely means that collaboration is done only by those with significant technial ability, the time and ability to document, and confidence enough to publish -The work of a sysadmin can be sufficiently separated from the IP of the company s/he works for that it is worth publishing -The derivative content of system administration is comprises multiple disparate types, such as code, best practices, documentation, and theory; a tool which only addresses a subset of this non-comprehensive list is inadequate. Because specific tools or code we use are only the final results of all of the planning and methodologies which make system administration difficult, this collaboration must be specifically developed for general information, methodologies, best practices, and implementation specifics, in addition to the actual code. In addition to providing a forum for the sharing of system administration related tools and information, this tool must also provide a method for the best, as determined by an as-yet undecided metric, content and authors of content to rise to the top. In short, this tool should provide a place for sysadmins to call their home, totally indispensable to their jobs but also engendering a greater sense of community among sysadmins. ------------------------- One of the main topics of discussion at our BoF was whether it made sense to try to overlay or underlay this tool into or onto the infrastructures web site and/or mailing list. The only conclusions we could come to were that most people on this list would at least be interested, and thus it should be brought up here, and also that it is the list who should answer that question, not Steve or I. My personal perspective is that the goal of this tool (and if anyone can come up with a better word or phrase for it, that would be great) is orthogonal to the infrastructure process as outlined in the bootstrapping paper. That is, the goal of this tool is to engender collaboration among sysadmins in all ways and make it easier to become a good sysadmin, whereas the goal of the infrastructure process is to take advantage of the best sysadmin tools and practices out there to actually implement a specific type of system. They definitely would create a nice feedback loop, and certainly there would be many people who were heavily involved in both, but I'm not really convinced there is a sufficient overlap. I have been contemplating such a tool for about two years now, and have not begun it for a number of reasons, largely related to laziness and not quite enough hubris to get in that far over my head. I've reached the point now, though, where I'm ready to commit to getting this out there, because I want it, I know a bunch of other people who would be interested in it, and I basically smack myself for every day it doesn't exist. Content I know that the above summary lacked any real details, and that was purposeful. I don't have a clear idea of what I want in terms of implementation, I just know a number of the high-level features that I want. One of the main differences I want between this and most of the other sysadmin sites I see out there is that I want this to be a place for sysadmins to stage their original content. I haven't been following advogato.com much, but from what I can tell, anyone can post an article there. I want something like that--any sysadmin can wander up and say "Hey, I've got some great docs for this site", post them, and let the chips fall. What I don't want is just a link fest with maybe some comments. Links are easy, but if all you do is link, then it's the sysadmin's responsibility to provide the page in the first place, and that's too high of a barrier of entry, too high of a maintenance cost, and collaboration is far too difficult. Modification Another thing that I definitely want is to provide other sysadmins the ability to basically patch someone's content. If someone writes something and you have something simple to add, either send your comments in as a patch, or have a content fork and maintain your own content if the original author won't accept your patch. By content here I mean code, documentation, a white paper on best practices, basically anything that provides a window into your sysadmin head. Scoring As mentioned in the summary, another thing I think will be extremely important, once this site has gained momentum, is the ability to score both authors and content. I believe this could easily be the hardest part of the project. Tentative plans Because I am as lazy as they come, and I recently noticed that SlashCode now has a Wiki plugin, I am planning on using it as the initial basis of this tool. I'm in the process of getting it all installed and working now, so I haven't actually looked at the slashcode yet, and thus can't really comment on its suitability. Anyone's comments would be appreciated. So, my questions to this list are: 1) Should I take this discussion somewhere else? 2) If not, should we discuss this long enough to get something more concrete and then take it somewhere else? 3) Does anyone have a better word than "tool"? 4) Is the lack of a good domain a restriction here? I own madstop.com and am planning on using that until something better comes up, and I have a couple other ideas, but none of them really shout out at me. I basically believe that the entry point is important, and so I'd like something that everyone agreed was decent. 5) Does anyone have any great ideas as to how to implement this, either conceptually, technically, or somewhere in between? 6) Finally, does anyone want to work with me on this? I've got a box on the end of a largely empty T1, ready to go. It's not a great machine, and if this tool suddenly makes the T1 very non-empty, then I might be in trouble, but for now I'm not worried. If people think it would be a good idea, I could write up a longer, more formal mission statement/white paper on what I am trying to do and why and all that jazz. At the very least, I would like any discussion process to come up with a semi-formal statement of goals, because unlike a plain chunk of software, this isn't just a question of straight feature sets and user interfaces. It's all that and a good bit more. And frankly, any piece of software should also have such a formal statement of goals... If you've read this far, you're probably interested, so I'd love to read your feedback. Luke Kanies -- "A computer lets you make more mistakes faster than any invention in human history--with the possible exceptions of handguns and tequila." -- Mitch Ratcliffe From mbarr@mbarr.net Wed Jan 30 23:32:22 2002 Received: from neuromancer.mbarr.net (postfix@66-108-143-133.nyc.rr.com [66.108.143.133]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id XAA19303 for ; Wed, 30 Jan 2002 23:32:21 -0800 Received: from [192.168.0.10] (unknown [192.168.0.10]) by neuromancer.mbarr.net (Postfix) with ESMTP id 3B7142AF428 for ; Thu, 31 Jan 2002 02:32:13 -0500 (EST) User-Agent: Microsoft-Entourage/10.0.0.1331 Date: Thu, 31 Jan 2002 02:32:10 -0500 From: Matthew Barr To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Infrastructures] Webmin... Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Short note: ran across an interesting tool that seems to work on half a zillion OS's, with some degree of accuracy, called webmin. It's an interesting piece of work... www.webmin.com Scaling would be hard, but a lot of the code could be very useful for controlling systems elsewhere. Matthew _______________________________________________________________________ Matthew Barr mailto:mhb8@cornell.edu AIM: MBarr1244 ICQ: 22130424 PGP Key Fingerprint = 35DC DC87 4F38 2E80 F327 2B50 FD82 A2CB CB80 80F3 From oddbjorn@tricknology.org Thu Jan 31 04:36:25 2002 Received: from home.tricknology.org (213-187-162-20.dd.nextgentel.com [213.187.162.20]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id EAA12012 for ; Thu, 31 Jan 2002 04:36:23 -0800 Received: by home.tricknology.org (Postfix, from userid 1000) id 1DFAF3EAC; Thu, 31 Jan 2002 13:35:36 +0100 (CET) Date: Thu, 31 Jan 2002 13:35:35 +0100 From: Oddbjorn Steffensen To: "Luke A. Kanies" Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020131123535.GA12474@tricknology.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.24i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Wed, Jan 30, 2002 at 11:38:33PM -0600, Luke A. Kanies wrote: > We propose to build an online tool with the specific purpose of > providing a means for sysadmins to collaborate on all aspects of system > administration. This document is not inclusive and will be updated > as the tool is developed. Have you looked at the following URLs: - Sysadmin Book of Knowledge: http://ace.delos.com/taxongate - Geoff Halprins SA Body of Knowledge: http://www.sysadmin.com.au/sa-bok/ While neither is nowhere near complete, they offer interesting ways to categorize the information, and could be used as a framework for a colloborative tool. -oddbjorn From dert@eecs.tufts.edu Thu Jan 31 12:32:55 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA00452 for ; Thu, 31 Jan 2002 12:32:54 -0800 Received: from sensei.EECS.Tufts.EDU (Sensei.EECS.Tufts.EDU [130.64.23.173]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 106CE1A504; Thu, 31 Jan 2002 15:32:48 -0500 (EST) Received: by sensei.EECS.Tufts.EDU (Postfix, from userid 13132) id 560FDF611B; Thu, 31 Jan 2002 15:32:47 -0500 (EST) Date: Thu, 31 Jan 2002 15:32:47 -0500 From: Andy Davidoff To: "Luke A. Kanies" Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020131153246.H19406@eecs.tufts.edu> Reply-To: andy+infrastructures@cs.tufts.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from luke@madstop.com on Wed, Jan 30, 2002 at 11:38:33PM -0600 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: #if Luke A. Kanies /* Jan 30, 23:38 */ > As mentioned in the summary, another thing I think will be extremely > important, once this site has gained momentum, is the ability to > score both authors and content. After the BoF, I tried to outline the issues and options that immediately came to my mind. This WIP would benefit from list-member input: http://www.eecs.tufts.edu/~dert/infrastructures/metrics.html I have only listed three scoring designs (Slashdot, Clever/Google, and Firefly/Joker) and only in the broadest sense. I think we should be able to come up with a design that has most of the benefits and few of the weaknesses of these others. This document is nearly two months old, now and I know it won't receive any more attention from me without attention from the community. I'm missing details and perspective; I'm hoping to receive those from the list. > I'm in the process of getting it all installed and working now, so I > haven't actually looked at the slashcode yet, and thus can't really > comment on its suitability. I often find that momentum is more important than quality when first starting a large project. So, good work. Let's see where SlashCode takes us. If nothing else, it will help us identify what we need and what we're missing. > 1) Should I take this discussion somewhere else? > 2) If not, should we discuss this long enough to get something more > concrete and then take it somewhere else? I'm here post-LISA, so I can't tell you how off-topic you are. :-) > 3) Does anyone have a better word than "tool"? Resource, for starters. > 4) Is the lack of a good domain a restriction here? Not to members of OUR community. > 5) Does anyone have any great ideas as to how to implement this, > either conceptually, technically, or somewhere in between? A passionate yet formal charter could draw more developers. I'd say this is the most important next-step. > 6) Finally, does anyone want to work with me on this? #endif /* luke@madstop.com */ The scoring system is most interesting to me because I've never seen it done /really/ well, though I believe the technology is there and the problem isn't particularly difficult, though it is not purely technical. -- Andy Davidoff Sen. Unix SysAdmin Tufts University From cwcjr@bestweb.net Thu Jan 31 13:39:33 2002 Received: from pimout4-int.prodigy.net (pimout4-ext.prodigy.net [207.115.63.103]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA01736 for ; Thu, 31 Jan 2002 13:39:32 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout4-int.prodigy.net (8.11.0/8.11.0) with SMTP id g0VLdON89858; Thu, 31 Jan 2002 16:39:25 -0500 Message-ID: <007c01c1aa9f$c12da2c0$c5c0a8c0@prodigy.com> From: "Will Cox" To: "Luke A. Kanies" , References: Subject: Re: [Infrastructures] Sysadmin collaboration tool Date: Thu, 31 Jan 2002 16:39:24 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-Mimeole: Produced By Microsoft MimeOLE V5.50.4910.0300 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > One of the topics, unsurprisingly, was directly related to this list--the > next version of isconf, and basic infrastructure concepts. The other > topic, my topic, was building an online sysadmin collaboration tool. > You've made some excellent points. And they tie in to a lot of what I've been thinking about since I had to manage a Notes environment. System Administration is just one particular field that needs better collaborative tools, which is what gives things like Notes, Groove, and Wiki something to do. The buzzword-compliant fields that deal with this are groupware, document management, content management, and knowlege management. Each tries to corral the problem of too much information -- while at the same time contributing to the problem by attempting to make it easy to exchange information. Jon Udell http://udell.roninhouse.com/ wrote a good paper on the subject during the course of the Software Carpentry contest at CodeSourcery. At my last position, we had lots of ways to share information: 1) the FAQ-O-Matic, 2) Wiki, 3) internal document repository built on Oracle with a web front-end, 4) Bugzilla, 5) flat html, 6) mailing lists. Nobody really put information into the documentation systems unless they were "prompted" to do so. It seemed mailing lists and Wiki yielded the best results, as the barriers to entry were the lowest. At my position now, information is hoarded, and has difficulty crossing functional boundaries. I'm finding it very easy to capture data from RSS newsfeeds and then post commentary with Radio Userland http://radio.userland.com/ . One of the UserLand staff has started a discussion on knowledge management using weblogs. I'm not certain that this _manages_ knowledge yet, but it does seem to stimulate the easy exchange of pertinent information. /cwc http://users.bestweb.net/~cwcjr/ From cwcjr@bestweb.net Thu Jan 31 13:49:10 2002 Received: from pimout4-int.prodigy.net (pimout4-ext.prodigy.net [207.115.63.103]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA07083 for ; Thu, 31 Jan 2002 13:49:09 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout4-int.prodigy.net (8.11.0/8.11.0) with SMTP id g0VLn1N50924; Thu, 31 Jan 2002 16:49:01 -0500 Message-ID: <008701c1aaa1$18e34a50$c5c0a8c0@prodigy.com> From: "Will Cox" To: , "Luke A. Kanies" Cc: References: <20020131153246.H19406@eecs.tufts.edu> Subject: Re: [Infrastructures] Sysadmin collaboration tool Date: Thu, 31 Jan 2002 16:49:02 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-Mimeole: Produced By Microsoft MimeOLE V5.50.4910.0300 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > http://www.eecs.tufts.edu/~dert/infrastructures/metrics.html > > I have only listed three scoring designs (Slashdot, Clever/Google, and > Firefly/Joker) and only in the broadest sense. I think we should be > able to come up with a design that has most of the benefits and few of > the weaknesses of these others. There's also Advogato, which uses group trust metrics for peer certification http://www.advogato.org/mission.html , thus avoiding poisoning problems that affect /. and others. I personally find it very difficult to pay much attention to a full /. thread: too much noise. /cwc From tim@techfocus.net Thu Jan 31 13:56:59 2002 Received: from gonzo.techfocus.net (dsl092-066-027.bos1.dsl.speakeasy.net [66.92.66.27]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA11784 for ; Thu, 31 Jan 2002 13:56:58 -0800 Received: from localhost (tim@localhost) by gonzo.techfocus.net (8.9.3+Sun/8.9.3) with ESMTP id QAA22257 for ; Thu, 31 Jan 2002 16:56:43 -0500 (EST) Date: Thu, 31 Jan 2002 16:56:43 -0500 (EST) From: Tim Quinlan To: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: <008701c1aaa1$18e34a50$c5c0a8c0@prodigy.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I saw an intersting site called alterslash.org It's an unofficial slashdot digest. They only show a few highly moderated comments for each article, plus the have a signal-to-noise meter for the comments. On Thu, 31 Jan 2002, Will Cox wrote: > I personally find it very difficult to pay much attention to a full /. > thread: too much noise. From luke@madstop.com Fri Feb 1 08:23:49 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA30089 for ; Fri, 1 Feb 2002 08:23:48 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g11GIbec000276; Fri, 1 Feb 2002 10:18:38 -0600 (CST) Date: Fri, 1 Feb 2002 10:18:37 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Andy Davidoff cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: <20020131153246.H19406@eecs.tufts.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Thu, 31 Jan 2002, Andy Davidoff wrote: > After the BoF, I tried to outline the issues and options that immediately > came to my mind. This WIP would benefit from list-member input: > > http://www.eecs.tufts.edu/~dert/infrastructures/metrics.html Great, I'll try to send my patch to it soon. > I often find that momentum is more important than quality when first > starting a large project. So, good work. Let's see where SlashCode > takes us. If nothing else, it will help us identify what we need and > what we're missing. I agree, which why I'll point out the following: I kind of picked a bad week to start this topic, since Steve T. is actually here in Nashville doing some consulting for my company, which means that I'm working an extra-long week. Even worse, I'm leaving town this weekend in hopes of finding a car in Atlanta (because there apparently aren't any here). So, I went and started this discussion but won't really have opportunity to make much meaningful input for a while. I'll definitely be collating over the weekend, though, and I plan to continue my contributions early next week. > > 4) Is the lack of a good domain a restriction here? > > Not to members of OUR community. Okay, then for fear of overloading the term infrastructure, I am going to use madstop.com as the root of this, unless someone else has a good reason not to or an even better URL. I've been meaning to spend a bit doing a brainstorming session, trying to come up with something good, but haven't sat down to do it yet. I keep thinking (sys)admin + meta + collection + some other stuff. > > 5) Does anyone have any great ideas as to how to implement this, > > either conceptually, technically, or somewhere in between? > > A passionate yet formal charter could draw more developers. I'd > say this is the most important next-step. I agree with this, and it will be my main goal next week. I fully expect it to be patched many times, but my hope is to get something out and published next week. I'll start with the blurb from my first email, and see if I can flesh it out a bit, including some of the information from your post. > The scoring system is most interesting to me because I've never seen > it done /really/ well, though I believe the technology is there and > the problem isn't particularly difficult, though it is not purely > technical. I definitely agree with this, especially since we are talking about grading so many different types of information. I mean, who's to say that just because you trust someone's opinion on code you should also trust their opinion on general methodologies? Ugh. -- "Did you know that black paint is an excellent stain remover?" - Dogbert From luke@madstop.com Fri Feb 1 08:39:39 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA05719 for ; Fri, 1 Feb 2002 08:39:38 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g11GYUgP000336; Fri, 1 Feb 2002 10:34:30 -0600 (CST) Date: Fri, 1 Feb 2002 10:34:29 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Will Cox cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: <007c01c1aa9f$c12da2c0$c5c0a8c0@prodigy.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Thu, 31 Jan 2002, Will Cox wrote: > You've made some excellent points. And they tie in to a lot of what I've > been thinking about since I had to manage a Notes environment. System > Administration is just one particular field that needs better collaborative > tools, which is what gives things like Notes, Groove, and Wiki something to > do. I certainly agree that we are not the only field lacking in good online collaboration. I guess what galls me the most is that it seems like we should be leading the conceptual charge, because we are exactly the people capable of doing so. > At my last position, we had lots of ways to share information: 1) the > FAQ-O-Matic, 2) Wiki, 3) internal document repository built on Oracle with a > web front-end, 4) Bugzilla, 5) flat html, 6) mailing lists. Nobody really > put information into the documentation systems unless they were "prompted" > to do so. It seemed mailing lists and Wiki yielded the best results, as the > barriers to entry were the lowest. At my position now, information is > hoarded, and has difficulty crossing functional boundaries. Interesting. That's one of the things that I've been thinking a lot about, how to keep a low barrier of entry. I also want to do what I can to maintain the concept of a conversation; one of the things that really frustrates me about /. is that if I respond to someone, then I have to basically refresh the page all the time to figure out if someone has responded to me. > I'm finding it very easy to capture data from RSS newsfeeds and then post > commentary with Radio Userland http://radio.userland.com/ . One of the > UserLand staff has started a discussion on knowledge management using > weblogs. I'm not certain that this _manages_ knowledge yet, but it does seem > to stimulate the easy exchange of pertinent information. I'll have to look into this, as I haven't heard of it before. As I mentioned in my previous email, I'll be picking this up next week when I get back in town. -- To define recursion, we must first define recursion. From rjw@alembic.com Fri Feb 1 20:26:45 2002 Received: from mail.alembic.net (root@mail.alembic.net [192.82.19.252]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id UAA12708 for ; Fri, 1 Feb 2002 20:26:44 -0800 Received: from tycho.alembic.com (tycho.alembic.com [192.82.19.225]) by mail.alembic.net (8.11.3/8.11.3) with SMTP id g124RtS18083; Fri, 1 Feb 2002 20:27:55 -0800 (PST) Received: from localhost by tycho.alembic.com (4.1/SMI-4.1) id AA28156; Fri, 1 Feb 02 20:30:14 PST Date: Fri, 1 Feb 2002 20:30:13 -0800 (PST) From: Ron Wickersham X-Sender: rjw@tycho To: "Luke A. Kanies" Cc: Will Cox , infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, 1 Feb 2002, Luke A. Kanies wrote: ---snip--- > > At my last position, we had lots of ways to share information: 1) the > > FAQ-O-Matic, 2) Wiki, 3) internal document repository built on Oracle with a > > web front-end, 4) Bugzilla, 5) flat html, 6) mailing lists. Nobody really > > put information into the documentation systems unless they were "prompted" > > to do so. It seemed mailing lists and Wiki yielded the best results, as the > > barriers to entry were the lowest. At my position now, information is > > hoarded, and has difficulty crossing functional boundaries. > > Interesting. That's one of the things that I've been thinking a lot > about, how to keep a low barrier of entry. I also want to do what I can > to maintain the concept of a conversation; one of the things that really > frustrates me about /. is that if I respond to someone, then I have to > basically refresh the page all the time to figure out if someone has > responded to me. ---snip--- i've used Discus as a web-based "conversation" builder. one nice advantage is that users can receive updates by e-mail, and on a user profile can selectively monitor topics of interest. while i haven't used it yet, the latest version also allows e-mail posting to topics...thus users can choose web-based or e-mail interaction. http://www.discusware.com -ron From tubaman@mailandnews.com Sat Feb 2 00:56:26 2002 Received: from MailAndNews.com ([199.29.68.111]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id AAA21548 for ; Sat, 2 Feb 2002 00:56:24 -0800 Received: from [66.68.36.205] (tubaman@mailandnews.com) by MailAndNews.com; Sat, 2 Feb 2002 03:56:14 -0500 X-WM-Posted-At: MailAndNews.com; Sat, 2 Feb 02 03:56:14 -0500 Received: from polishwonder (localhost [127.0.0.1]) by polishwonder.dyn.dhs.org (8.11.6/8.11.6) with ESMTP id g0VHOvu09155 for ; Thu, 31 Jan 2002 23:24:58 +0600 Date: Thu, 31 Jan 2002 23:24:57 +0600 From: Ryan Nowakowski To: infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020131232457.A9020@polishwonder.dyn.dhs.org> Reply-To: ryan_nowakowski@alumni.utexas.net References: Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit In-Reply-To: ; from luke@madstop.com on Thu, Jan 31, 2002 at 11:38:33 +0600 X-Mailer: Balsa 1.2.3 Lines: 41 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On 2002.01.31 11:38 Luke A. Kanies wrote: > > > 1) Should I take this discussion somewhere else? No, since the next revision of the infrastructures.org methods will probably come from this tool, we should have input into it's design. I think that once the tool is out of the design phase and into implementation, all the gory details probably shouldn't be posted here. > > 2) If not, should we discuss this long enough to get something more > concrete and then take it somewhere else? > > 3) Does anyone have a better word than "tool"? application? webapp? > > 4) Is the lack of a good domain a restriction here? I own madstop.com > and am planning on using that until something better comes up, and I have > a couple other ideas, but none of them really shout out at me. I > basically believe that the entry point is important, and so I'd like > something that everyone agreed was decent. I think the domain is not as important. If you build it, they will come (ex: freshmeat.net) > 5) Does anyone have any great ideas as to how to implement this, either > conceptually, technically, or somewhere in between? Technically, I believe Zope gives you everything you might need, including a Wiki and content management. I've created a few apps with Zope and have found it to be very flexible. > 6) Finally, does anyone want to work with me on this? Yes, sounds cool. - Ryan From tubaman@mailandnews.com Sat Feb 2 00:56:36 2002 Received: from MailAndNews.com ([199.29.68.111]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id AAA21551 for ; Sat, 2 Feb 2002 00:56:27 -0800 Received: from polishwonder.dyn.dhs.org [66.68.36.205] (tubaman@mailandnews.com) by MailAndNews.com; Sat, 2 Feb 2002 03:56:10 -0500 X-WM-Posted-At: MailAndNews.com; Sat, 2 Feb 02 03:56:10 -0500 Received: (from tubaman@localhost) by polishwonder.dyn.dhs.org (8.11.6/8.11.6) id g1256oL02672 for infrastructures@mailman.terraluna.org; Sat, 2 Feb 2002 11:06:50 +0600 Date: Sat, 2 Feb 2002 11:06:50 +0600 From: Ryan Nowakowski To: infrastructures@roton.terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020202110650.A2669@polishwonder.dyn.dhs.org> Reply-To: ryan_nowakowski@alumni.utexas.net References: Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from luke@madstop.com on Thu, Jan 31, 2002 at 11:38:33 +0600 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On 2002.01.31 11:38 Luke A. Kanies wrote: > > > 1) Should I take this discussion somewhere else? No, since the next revision of the infrastructures.org methods will probably come from this tool, we should have input into it's design. I think that once the tool is out of the design phase and into implementation, all the gory details probably shouldn't be posted here. > > 2) If not, should we discuss this long enough to get something more > concrete and then take it somewhere else? > > 3) Does anyone have a better word than "tool"? application? webapp? > > 4) Is the lack of a good domain a restriction here? I own madstop.com > and am planning on using that until something better comes up, and I have > a couple other ideas, but none of them really shout out at me. I > basically believe that the entry point is important, and so I'd like > something that everyone agreed was decent. I think the domain is not as important. If you build it, they will come (ex: freshmeat.net) > 5) Does anyone have any great ideas as to how to implement this, either > conceptually, technically, or somewhere in between? Technically, I believe Zope gives you everything you might need, including a Wiki and content management. I've created a few apps with Zope and have found it to be very flexible. > 6) Finally, does anyone want to work with me on this? Yes, sounds cool. - Ryan From luke@madstop.com Mon Feb 4 12:58:40 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA02266 for ; Mon, 4 Feb 2002 12:58:39 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14Kpx3D012188 for ; Mon, 4 Feb 2002 14:52:00 -0600 (CST) Date: Mon, 4 Feb 2002 14:51:59 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] isconf integration with existing install tools Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi all, I'm currently looking at deploying some version of Isconf in my enterprise. Steve was out here last week getting me up to speed and helping to convince my management that I'm doing the right thing, so I'm pretty familiar with the product. My situation is, I have a pretty significant investment in Jumpstart right now, specifically related to Sun's JASS toolkit. Should I attempt to develop in such a way that I can maintain backwards compatibility with JASS, thus helping to spread the love, or should I basically move away from it and bring my toys with me? I'm not very happy with the overall design of JASS, but that's the part that I would basically have to replace anyway. If I replaced all or part of the portions which actually do the work, I would think it shouldn't be too difficult to use my installed base of code while allowing both Isconf and JASS users to take advantage of additional code. I know that I'll have to jump through some hoops, but I'm just a bit hesitent to toss JASS and all of it's software out the window. For those who haven't used it, JASS provides basically two things for Jumpstart: First, a basic framework for all Jumpstart actions, such as running scripts and copying files, and second, a very strong foundation of security scripts, to do things like tighten file permissions, increase the randomness of the TCP/IP stack, and all kinds of other stuff. It basically provides a framework for Jumpstart, but one which also works in a normally booted environment, and also a lot of security scripts. As I mentioned, I don't much like the framework itself, but I like a lot of the scripts and I don't much look forward to trying to replace them (it's open source because it's all in sh, but not modifying or redistributable). For perspective, I currently have 93 different scripts set up in JASS, along with all of my file copies and my package installs, so this isn't two days worth of work or anything, and most of those ship with JASS. One of the things that I think is pretty important to the future of this level of automated systems administration is having chunkable code that we can share. Isconf provides a framework for that code, although that framework itself is in the process of being rewritten, and now we need to begin developing and distributing the little chunks of code that do all the interesting stuff. Of course, the next step is that we need to have a place to do this distribution, but that's maybe another email. It might even be one I sent last week. :) -- Nature and nature's laws lay hid in night, God said, "Let Newton be," and all was light. It did not last; the devil howling "Ho! Let Einstein be!" restored the status quo. From luke@madstop.com Mon Feb 4 13:08:46 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA07524; Mon, 4 Feb 2002 13:08:45 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14L2fc2012217; Mon, 4 Feb 2002 15:02:41 -0600 (CST) Date: Mon, 4 Feb 2002 15:02:41 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: stevegt@terraluna.org cc: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] state engine Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi Steve (I've CC'ed the infrastructures list because this is also pertinent to them), I'm still trying to get my brain around how make is being used as a state engine in isconf. As far as I can tell, all make does is verify that all scripts are run in the correct order and the correct number of times, that is, if a script is supposed to be run once then make guarantees that, but make is also used to make a script run all the time. So my question is, if we ignore the topic of config files for just a minute, does make currently provide functionality that couldn't be replaced with a script which essentially checked the stamps directory, and only ran a script if (a) the stamp didn't exist, or (b) the script was supposed to be run multiple times? Make is normally used to run a stanza multiple times, but in this case we generally don't actually want to run a stanza more than once. Or at least, we only want to run new members of the stanza. It's pretty different from code, where you want to recompile all of the basic chunks and then combine those basic chunks into large chunks. In this case, we are only worried about running the correct scripts in the correct order. Or am I missing something? That seems to be the case, and that difference between how Isconf uses make and how I am used to using make seems to be one of the hardest things to wrap my brain around. I guess the main reason I ask is because if this difference is as clear as I think it is, Isconf's requirements for a state engine are much less expansive than make's feature set. At this point, I'm not planning on doing anything about it, but I would definitely like to clarify this before I get much deeper. Obviously, the state engine is one of the most important aspects of this whole process. Thanks, Luke -- Some people are afraid of heights. I'm afraid of widths. -- Stephen Wright From hag@linnaean.org Mon Feb 4 13:30:24 2002 Received: from perdition.linnaean.org (h00d0b71b83ad.ne.mediaone.net [65.96.132.240]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA17862; Mon, 4 Feb 2002 13:30:24 -0800 Received: by perdition.linnaean.org (Postfix, from userid 31013) id BB2E31BA0B; Mon, 4 Feb 2002 16:30:02 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Luke A. Kanies" Cc: stevegt@terraluna.org, infrastructures@terraluna.org Subject: [Infrastructures] state engine In-Reply-To: References: X-Mailer: VM 6.34 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty Date: Mon, 4 Feb 2002 16:30:02 -0500 Message-Id: <20020204213002.BB2E31BA0B@perdition.linnaean.org> Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > From: "Luke A. Kanies" > Date: Mon, 4 Feb 2002 15:02:41 -0600 (CST) > > Hi Steve (I've CC'ed the infrastructures list because this is also > pertinent to them), > > I'm still trying to get my brain around how make is being used as a state > engine in isconf. As far as I can tell, all make does is verify that all Does this mean code for isconf is available? (maybe I was supposed to ask this years ago...) I've seen it talked about, but never seen code. From luke@madstop.com Mon Feb 4 15:44:02 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id PAA22985; Mon, 4 Feb 2002 15:44:01 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14Nbp86012595; Mon, 4 Feb 2002 17:37:51 -0600 (CST) Date: Mon, 4 Feb 2002 17:37:51 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Daniel Hagerty cc: stevegt@terraluna.org, infrastructures@terraluna.org Subject: Re: [Infrastructures] state engine In-Reply-To: <20020204213002.BB2E31BA0B@perdition.linnaean.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 4 Feb 2002, Daniel Hagerty wrote: > Does this mean code for isconf is available? (maybe I was supposed > to ask this years ago...) I've seen it talked about, but never seen > code. It's GPL'ed. Steve gave me a sanitized version recently, and I'm sure he'd give you one, or let me send you one. It's still very much a hand-roll process in terms of setup, but yeah, the code is available. -- I don't want the world, I just want your half. From luke@madstop.com Mon Feb 4 15:52:21 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id PAA25979 for ; Mon, 4 Feb 2002 15:52:20 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14Nki0l012619; Mon, 4 Feb 2002 17:46:44 -0600 (CST) Date: Mon, 4 Feb 2002 17:46:44 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Mitch Wyle cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] isconf integration with existing install tools In-Reply-To: <20020204142001.A8160@wyle.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 4 Feb 2002, Mitch Wyle wrote: > 2002-02-04 > > Dear Luke, > > I suggest you abstract the policies and requirements for your automation > need away from all frameworks and then map the needs to JASS. Then > (possibly) you can re-map your work to the Isconf-next-generation[tm] > framework when it stabilizes. My suggestion is that you spend your time > writing prose and pseudo-code that describes what you want to happen in > your environment without thinking about the capabilities of one > framework or another. Then start to map your needs to the JASS > framework first. I'm already past that point; I have a significant amount of code which does stuff which I would like to apply to any machines I build within my infrastructure. I don't need to do any planning or mapping, because the code is already there and I know I want to use it. I guess I am more looking for an overall direction of the software we as a group are planning on developing. On the one hand, we want to be able to develop sophisticated code chunks which can depend on a base setup, but on the other hand, I would really like my code to be usable by people who aren't willing or able to commit to full automation isconf-style. Currently JASS is a decent compromise between the two, because it provides a decent framework for shell scripting, but the code is already chunked into separate scripts. Well, I hope to starting plopping code into place this week, and I'm as lazy as they come so I'll be reusing as much of the code as I can. I'll probably provide a basic wrapper initially, but we'll see. I've already threatened the Sun guys with rewriting JASS, because its structure is so poor, and it may come to that. -- "The Microsoft Exchange Information Store service depends on the Microsoft Exchange Directory service which failed to start because of the following error: The operation completed successfully." From hag@linnaean.org Mon Feb 4 16:32:20 2002 Received: from perdition.linnaean.org (h00d0b71b83ad.ne.mediaone.net [65.96.132.240]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id QAA14156; Mon, 4 Feb 2002 16:32:19 -0800 Received: by perdition.linnaean.org (Postfix, from userid 31013) id E7D7B1BA14; Mon, 4 Feb 2002 19:31:57 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Luke A. Kanies" Cc: stevegt@terraluna.org, infrastructures@terraluna.org Subject: Re: [Infrastructures] state engine In-Reply-To: References: <20020204213002.BB2E31BA0B@perdition.linnaean.org> X-Mailer: VM 6.34 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty Date: Mon, 4 Feb 2002 19:31:57 -0500 Message-Id: <20020205003157.E7D7B1BA14@perdition.linnaean.org> Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > It's GPL'ed. Steve gave me a sanitized version recently, and I'm sure > he'd give you one, or let me send you one. It's still very much a Well, it's GPL'd, so technically steve doesn't enter into equation if you have a copy. :) I would love to have a copy, if someone would be so kind. From luke@madstop.com Mon Feb 4 17:00:39 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA29494; Mon, 4 Feb 2002 17:00:38 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g150suB6012779; Mon, 4 Feb 2002 18:54:56 -0600 (CST) Date: Mon, 4 Feb 2002 18:54:56 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Daniel Hagerty cc: stevegt@terraluna.org, infrastructures@terraluna.org Subject: Re: [Infrastructures] state engine In-Reply-To: <20020205003157.E7D7B1BA14@perdition.linnaean.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 4 Feb 2002, Daniel Hagerty wrote: > > It's GPL'ed. Steve gave me a sanitized version recently, and I'm sure > > he'd give you one, or let me send you one. It's still very much a > > Well, it's GPL'd, so technically steve doesn't enter into equation > if you have a copy. :) I agree, but I just wanted to make sure that he was satisfied with how sanitized this code is. He kind of rushed it out to me because he was on his way out here, and I figured I'd give him a chance to once-over it before I posted a link. -- "The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly goes wrong goes wrong it usually turns out to be impossible to get at or repair." -- Douglas Adams, Mostly Harmless From dert@eecs.tufts.edu Mon Feb 4 21:52:44 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA16870; Mon, 4 Feb 2002 21:52:43 -0800 Received: from piano.eecs.tufts.edu (Piano.EECS.Tufts.EDU [130.64.23.40]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 2038A1A791; Tue, 5 Feb 2002 00:52:36 -0500 (EST) Received: by piano.eecs.tufts.edu (Postfix, from userid 13132) id B9311AF94; Tue, 5 Feb 2002 00:52:35 -0500 (EST) Date: Tue, 5 Feb 2002 00:52:35 -0500 From: Andy Davidoff To: infrastructures@terraluna.org Cc: stevegt@terraluna.org Subject: Re: [Infrastructures] state engine Message-ID: <20020205005235.A18815@eecs.tufts.edu> Reply-To: andy+infrastructures@cs.tufts.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.2i In-Reply-To: ; from luke@madstop.com on Mon, Feb 04, 2002 at 03:02:41PM -0600 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I have problems with the use of make, and I'm biased towards Tufts' tools, so I won't enumerate my issues unless they are /your/ issues. Since I also have not seen any isconf[0] code, I can't comment on it. Maelstrom[1] is designed to discover a correct order in which one may run any number of chunks. It does this by attempting to run a chunk, which must succeed or fail in a convergent fashion. Via a simple algorithm, Maelstrom can determine the proper order of chunks (and in the process execute them in same) in /at most/ O(n^2). An admin may suggest dependencies to Maelstrom which it will try to use to further reduce O(n^2) towards O(n). These suggestions do not bind Maelstrom into a particular execution path; they're strictly advisory. The chunks may be hierarchial or flat, but they must be aware of their side-effects and must not impose upon other chunks. This may sound like a high price to pay for a meager reward, but I think properly asserting the necessary order of X operations on Y hosts is a difficult enough task that I'd like something else to do it for me. ;-) Note that adding (or altering!) an operation is O(1) work for you and (at most) O(YX^2) for Maelstrom. I think these are the kinds of scaling efficiencies we're all looking for. Of course, I'm still in the dark as far as a formal isconf definition goes, so maybe someone who knows could post a comprehensive abstract? I know the words "comprehensive abstract" sound like an oxymoron, but I'm really just looking for a formal definition of the problem domain and how isconf goes about filling it with a solution. [0] My presumptions on the intent of isconf are based purely upon LISA. [1] Alva Couch, LISA '01, http://www.eecs.tufts.edu/~couch/maelstrom #if Luke A. Kanies /* Feb 04, 15:02 */ > In this case, we are only worried about running the correct scripts > in the correct order. #endif /* luke@madstop.com */ -- Andy Davidoff Sen. Unix SysAdmin Tufts University From luke@madstop.com Tue Feb 5 08:45:18 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA02243; Tue, 5 Feb 2002 08:45:16 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g15GeXub015062; Tue, 5 Feb 2002 10:40:34 -0600 (CST) Date: Tue, 5 Feb 2002 10:40:33 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Andy Davidoff cc: infrastructures@terraluna.org, stevegt@terraluna.org Subject: Re: [Infrastructures] state engine In-Reply-To: <20020205005235.A18815@eecs.tufts.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Tue, 5 Feb 2002, Andy Davidoff wrote: > I have problems with the use of make, and I'm biased towards Tufts' > tools, so I won't enumerate my issues unless they are /your/ issues. > Since I also have not seen any isconf[0] code, I can't comment on it. Well, I plan on pinning Steve down today on whether I should go ahead and post a link for isconf, but in the mean time, I'll give a quick run-down of how isconf works. I expect Steve or someone else will walk behind me and correct my grave misconceptions, but... Basically, isconf starts with an rc script, which first verifies (using rsync) that the running copy is the most recent version of the script, pulls all of the latest isconf config info (into /var/isconf, usually), and then passes its arguments on to the actual isconf script. This secondary script parses a config file, currently called hosts.conf, which has a makefile-like syntax, and then uses the output of the parsing to run make. The hosts.conf file has a default entry for all hosts, and then has host or class entries, which essentially just set variables. For install, all hosts set a BOOT variable, which determines what make targets get run at boot time, and they'll also set any number of other variables, depending on what the host actually does (e.g., they might list the disk sizes if the host has an LVM which allows dynamic resizing, or they might list the NIS domain). So, when make is run, it has a list of targets, as compiled from hosts and classes in hosts.conf, and it basically just runs through that list of targets in the order specified, making sure that each target has been run. This is where the use of make deviates from the norm: Each target can be set up in one of two ways; it either leaves a timestamp, thus guaranteeing that the target never runs again, or it does not leave a timestamp, thus guaranteeing that it runs every time. As I mentioned in my previous email, it took me a while to pick up on this. It's important, because you never actually go back and change things; if a host has a list of make targets on it, and you want to modify what one of those targets does, then your only choice is to create another target which does the modification. This is where isconf seems to diverge from Maelstrom (the tool you mentioned, not the video game). Maelstrom seems to be more worried about taking a given list and making sure it will all run, whereas isconf assumes that the any item in the list will run essentially independently and is only concerned with making sure that each item is run in the correct order and the currect number of times. I can certainly see the reason for something like Maelstrom. The way isconf thinks about the world, though, is that it essentially just wants to keep track of every command ever issued on a box, and in what order those commands are issued. The importance of the order is not based on whether the code chunks complete or not, it's only based on ending up with the same box after everything. And I think this is the main point: From Alva's page, one of the three prerequisites of Maelstrom is that each code chunk be non-destructive WRT the other code chunks. Isconf, on the other hand, assumes that code chunks will be mutually destructive, and that's why it's so important to maintain the correct order. Otherwise, there's no way to completely reconstruct an exact copy of a system at a given date. For instance, if you initially tune a system by modifying the system file (using a make target within isconf), but later find that you want to change those tuning parameters, you can't go back and modify the previous tuning; you have to create a new make target which sets the system file again. If these scripts are later run in the opposite order, which Maelstrom might do because each of them would succeed in their efforts, then you will not get what you want. I'll have to think about this a bit more, because it does seem like Maelstrom (damn that word is hard to type) could be useful, but I'm still trying to get my brain around this whole process, so I can't really see its place yet. Isconf's goals do seem to be more modest in some ways, and yet they also seem to support greater long-term consistency. > Of course, I'm still in the dark as far as a formal isconf definition > goes, so maybe someone who knows could post a comprehensive abstract? > I know the words "comprehensive abstract" sound like an oxymoron, but > I'm really just looking for a formal definition of the problem domain > and how isconf goes about filling it with a solution. The above was pretty informal, but hopefully it was sufficient. Luke -- "DOS is the _only_ operating system -- and I'm using that term loosely -- which [exhibits this behavior]." -- Aeleen Frisch, "Essential System Administration" From luke@madstop.com Tue Feb 5 17:22:48 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA09051; Tue, 5 Feb 2002 17:22:47 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g161I6Jc016403; Tue, 5 Feb 2002 19:18:07 -0600 (CST) Date: Tue, 5 Feb 2002 19:18:06 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: mitch@wyle.org cc: "'Andy Davidoff'" , infrastructures@terraluna.org, stevegt@terraluna.org Subject: RE: [Infrastructures] state engine In-Reply-To: <002301c1aea6$cae9ab00$de01010a@wyle.org.wyle.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Tue, 5 Feb 2002, Mitch Wyle wrote: > Thank you very much for this isconf capsule summary for us newbies on > the list! Hmm, I kind of expected that everyone here would have a basic familiarity with the code that we're all theoretically talking about, but apparently not... I'm not blaming anyone, of course, just noting my expectation. > Luke> verifies (using rsync) that the running copy is the most recent > > Hmmm, then isconf assumes you already have rsync available and running > correctly on the client, that the rsync ports are open on the net, or > that rsync can otherwise make its connection and do work. For an "out > of box" virgin client we need a bootp or tftp or dhcp low-level > bootstrap process that pulls the rc baseline pieces needed to start > isconf. Here are what I see as the basic dependencies: -rsync -networking -DNS -perl -/bin/sh -make and some sundry standard shell tools. I think that's it. The reality is, there's no real way you can start with a totally virgin client; you have to put something on it. Assuming you put even one thing on it, you might as well put a few. In this case, to keep the code small the initial isconf script requires more than it might. If it didn't assume those things were there, it would have to include code to pull them down. I think the above requirements are reasonable, but I'll talk about that again in a minute. > With our dependency on rsync, the net ports, these config files, > /bin/sh, and $PATH/make we should be able to make a checklist of all > origins of possible failure. Recovery from "tightened down blocked net > ports by the brazen young cisco net admin," a corrupted config file, or > root user error does not appear simple (to me yet). But I am very new. I basically agree with this, and that's one of the issues I have yet to mentally resolve. I'm considering setting up rsync+ssh keys, so that the rsync is at least partially authenticated and we don't add another port to care about. This will probably be especially important as the number of networks this is used with increases. [SNIPPED example conditional] > The point is, you can set conditions in your action to decide if you > need to run a command or if it has already been run. You don't > necessarily need the timestamp trick. You can, but the nice thing about the current method is that you are not constantly confronted with the method itself, which you would be if you were constantly doing conditionals. This neatly pulls all of the timestamp code out so you don't have to actually see it--except, of course, the setting of the stamp itself. I'd personally like to wrap things so that my wrapper checked and placed the stamp, so I didn't have to see any of it. I'm still working on that, though, and will let the list know when I think I have something. > > find that you want to change those tuning parameters, you > > can't go back and modify the previous tuning; you have to > > create a new make target which sets the system file again. > > This is a very good feature unless you go through a decade (or so) of > frequent changes where parsing and re-creating a "fresh" environment > involves going through very many previous states. There should be some > simple way to branch and start afresh. There basically is, but I don't have any experience with it yet. If you are using images, rather than normal install processes, then you basically just create a new image from what you are considering your base install, and then mark that as the base for everyone. If you aren't using images, then I think it would actually be a bit tough. Like I said, I haven't run into this yet, because I'm still in the egg phase of this project. I'm calling these updated images 'rollups' in my head, but Steve might have a different name for them. -- "If I want your opinion, I'll read your entrails." --Doug Shewfelt From luke@madstop.com Wed Feb 6 07:25:09 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA26282 for ; Wed, 6 Feb 2002 07:25:07 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g16FKA2f018744 for ; Wed, 6 Feb 2002 09:20:11 -0600 (CST) Date: Wed, 6 Feb 2002 09:20:10 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] isconf code Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Well, I told you I'd pin him down, and I did. Steve's still laid up sick, but he told me to go ahead and post a link to his code, so here it is: http://www.infrastructures.org/tarballs/isconf-2i.tar.gz It's pretty aix-specific right now, but I have fledgling hp-ux and solaris examples if anyone wants to see them. They're very fledgling, though. Of course, because he wasn't specifically planning on releasing this code, YMMV. -- Is life worth living? That is a question for an embryo, not a man. --Samuel Butler From cwcjr@bestweb.net Wed Feb 6 11:08:55 2002 Received: from pimout3-int.prodigy.net (pimout3-ext.prodigy.net [207.115.63.102]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id LAA12653 for ; Wed, 6 Feb 2002 11:08:52 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout3-int.prodigy.net (8.11.0/8.11.0) with SMTP id g16J8g917850 for ; Wed, 6 Feb 2002 14:08:44 -0500 Message-ID: <017701c1af41$b38a39c0$c5c0a8c0@prodigy.com> From: "Will Cox" To: Date: Wed, 6 Feb 2002 14:08:44 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 Subject: [Infrastructures] re: sysadmin collab tool discussion, Fw: [K-Logs] The I in K-Logs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: The klogs discussion has hit a number of items mentioned in infrastructures e-mails previously, including participation and adoption rates. I think John Robb is on to something here, in how the blogging community is building the infrastructure to enable a effective data exchange and collaboration. /cwc ----- Original Message ----- From: "John Robb" To: Sent: Tuesday, February 05, 2002 2:58 PM Subject: [K-Logs] The I in K-Logs > Dear K-Loggers, > > There have been lots of efforts to turn knowledge management into merely > discussions or conversations. This is a wrong-headed approach. While > discussions (mailing lists, e-mail conversations, and chat) have merit as a > means of collaboration, they aren't necessarily a good way to produce > archival knowledge. Why? Given the tools used, most conversations are > either hidden (in e-mail inboxes), disappear (as in chat sessions), or > problematic (discussions are often dominated by a few individuals, they can > succumb to spam, and often offer poor user interaction due to performance > problems and threading issues). > K-Logs solve many of these problems. As David and Jim rightly point out, a > K-Log is partly a selfish activity. It promotes a personal brand (as a > domain expert in an organization or a smart person with a valuable POV). It > solves personal organization problems. It makes it possible to be heard in > the online environment despite noise generated by more vocal contributors. > It documents what you do in a way that is visible to a wider corporate > audience. As a result of the above, it provides people with motive to > contribute (a feature lacking in the solutions provided by Lotus and > others). Jon Udell gets this: > > (see: http://radio.weblogs.com/0100887/2002/01/31.html#a44 ) > > Hey, I like to collaborate using e-mail, chat and occasionally use > discussion groups. However, if I am going to put some real effort into > translating my thinking into bits, I want to make sure that effort is > archived and used by as many people as possible. I also want it to be part > of my body of work (which according to David and myself, is a representation > of me online to everyone that hasn't met me face to face). K-Logs make this > possible within a corporate context. > > A final thought. Collaboration tools should provide content (raw fodder) > that I can use in my K-Log. Right now, its possible to forward e-mails to > my desktop K-Log tool. A developer is working right now on sucking in IM > conversations. There has even been some talk about building a desktop > discussion group system that runs as a tool in Radio and uses Web Services > to send messages back and forth via a central server. This would not only > make all discussion items postable to my K-Log, it would also solve two > other problems with discussion groups: performance and off-line use (in > this tool, it would be possible for me to have a complete archive of a > discussion group on my desktop). > > Sincerely, > > John Robb > > > > > To unsubscribe from this group, send an email to: > klogs-unsubscribe@yahoogroups.com > > > > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ > > From stevegt@roton.terraluna.org Wed Feb 6 14:34:05 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id OAA21796; Wed, 6 Feb 2002 14:34:05 -0800 Date: Wed, 6 Feb 2002 14:34:05 -0800 From: Steve Traugott To: mitch@wyle.org Cc: "'Luke A. Kanies'" , "'Andy Davidoff'" , infrastructures@terraluna.org, stevegt@terraluna.org Subject: Re: Roll-ups, branches, blocks, images (was: [Infrastructures] state engine) Message-ID: <20020206143405.A3865@scramjet.TerraLuna.Org> References: <003601c1aeb2$060224a0$de01010a@wyle.org.wyle.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <003601c1aeb2$060224a0$de01010a@wyle.org.wyle.org>; from mfw@wyle.org on Tue, Feb 05, 2002 at 06:00:16PM -0800 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: ...starting to recover from a stopped-up head, been letting my mail go until I could think straight... On Tue, Feb 05, 2002 at 06:00:16PM -0800, Mitch Wyle wrote: > > > I'm calling these updated > > images 'rollups' in my head, but Steve might have a different > > name for them. > > I attended Steve's talk at SVLUG last month (and hope to see you all > tommorow again) There are large "chunks" in the naming of makefile rules > that Steve called "Blocks" from the Aerospace nomenclature naming > convention. He might call these roll-up images blocks. You're both right. To catch up everyone else -- "Block" means "a site-specific version of this operating system, including any installed packages and changes". A Block20 machine is a Block10 machine with stuff added, etc. Block00 generally starts as a vanilla vendor install. You start adding things to the "Block00" stanza of the makefile. Everything you add makes it take longer to upgrade a machine during that first reboot after initial network install. After a while you'll reach your limit of tolerance for how long that first reboot takes -- it's rebooting 8 times, rebuilding the kernel twice, and taking 30 minutes to layer other stuff in, before you ever get a login prompt. So you install one more Block00 machine, change its hostname and IP to something innocent (I use "baseline" and a reserved address), and cut a network install image of it. This is now your "Block10 image". Then you create a Block10 makefile stanza, like so: Block10: Block00 ... For all future installs you use the Block10 network install image as your baseline, and add any new patches to the Block10 stanza, thereby saving 30 minutes of churn but still getting results that are bit-for-bit the same as if you had installed Block00 and upgraded. Blocks don't have to be named "Block". You might call it Solaris00, SunOS_00, HPUX00, etc. During discussions they still end up being called "Block" though. Steve -- . . ` * Steve Traugott ` . * + Speaker Coordinator Infrastructure Architect + ` Silicon Valley stevegt@TerraLuna.Org ' * . ' +` * Linux Users Group http://www.stevegt.com/ http://www.svlug.org From stevegt@roton.terraluna.org Wed Feb 6 17:14:17 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id RAA07010 for infrastructures@scramjet.TerraLuna.Org; Wed, 6 Feb 2002 17:14:17 -0800 Date: Wed, 6 Feb 2002 17:14:17 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20020206171417.C3472@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Announcement: isconf2i Released Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All! While the conversation's already heated up about this, and Luke already posted a link, I'd like to "officially" direct you to the latest production version of isconf -- version 2i is a sanitized version of production code, optimized for AIX but portable to other platforms. Included are the configuration and makefiles used on a recent project to build AIX HACMP clusters in a couple of hours. Luke and I were able to port this code to Solaris and have a machine installing basic packages within several hours of unpacking the tarball, including CVS checkins and administrivia. http://www.terraluna.com/cgi-bin/cvsweb.cgi/isconf2i/README?rev=1.1 http://www.terraluna.com/cgi-bin/cvsweb.cgi/isconf2i/ http://www.infrastructures.org/tarballs/isconf-2i.tar.gz I'd suggest starting at that README. Steve -- . . ` * Steve Traugott ` . * + Speaker Coordinator Infrastructure Architect + ` Silicon Valley stevegt@TerraLuna.Org ' * . ' +` * Linux Users Group http://www.stevegt.com/ http://www.svlug.org From stevegt@roton.terraluna.org Mon Feb 11 18:52:54 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id SAA25992 for infrastructures@TerraLuna.Org; Mon, 11 Feb 2002 18:52:54 -0800 Date: Mon, 11 Feb 2002 18:52:54 -0800 From: Steve Traugott To: infrastructures@TerraLuna.Org Message-ID: <20020211185254.A5287@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] LISA 2002 Call for Papers Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All! The LISA 2002 Call for Papers is out; deadline for extended abstracts is early this year -- April 29th. http://www.usenix.org/events/lisa02/cfp/ In case you haven't heard yet, I'm now on the Program Committee, so I'm available to assist, advise, etc. if you have any paper ideas that you are considering. As we saw last year, the number of quality submissions has increased, making it harder to get accepted -- I'd recommend that you circulate draft extended abstracts for early feedback, on the Infrastructures list and elsewhere, in order to have a greater chance of putting together a competitive abstract ("release early, release often"). Collaborating in teams of two or three is also a good way to raise the chance of putting together a quality paper; I'll throw my name in the pot if anyone on this list is looking for collaborators. Who else would be up for working with potential authors? If you've never published anything yet and aren't sure of the actual process or time commitment, quite a few people on this list have been there before. Feel free to ask. Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From stevegt@roton.terraluna.org Mon Feb 11 22:38:28 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id WAA11939 for infrastructures@TerraLuna.Org; Mon, 11 Feb 2002 22:38:28 -0800 Date: Mon, 11 Feb 2002 22:38:28 -0800 From: Steve Traugott To: infrastructures@TerraLuna.Org Message-ID: <20020211223828.A26037@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Thoughts on Generally Accepted (Best?) Practices, Standards Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All, If there were a set of community-maintained standards or generally accepted practices for the systems administration and infrastructure architecture fields, what physical form do you think they should take if rendered on paper? I can think of a few: - RFC-style, serially numbered documents which cannot be changed; can only be superseded by later documents in the series. - Looseleaf binder(s) topically organized more or less like a conventional book, but with individual pages replaced as they evolve; each page has its own version number. If you are a pilot or have worked with military-style technical manuals you've probably seen this sort of thing. - A hybrid -- RFC-like serially numbered documents which are replaced rather than superseded; each document as a whole has a version letter or number. Topical organization might be improved by use of a decimal numbering scheme rather than strict serial numbers. This is probably closer to the IEEE standards document model. Can anyone think of any others? I'm interested in your thoughts about the readability and usability of the documents themselves, in both printed and electronic form. Each of these models have their advantages and drawbacks. Each would be handled slightly differently electronically, and would use different peer review, versioning, and change control models. There's an important reason why I'm asking this; I'll cover it in my next message. Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From stevegt@roton.terraluna.org Tue Feb 12 01:50:30 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id BAA13553 for infrastructures@TerraLuna.Org; Tue, 12 Feb 2002 01:50:30 -0800 Date: Tue, 12 Feb 2002 01:50:27 -0800 From: Steve Traugott To: infrastructures@TerraLuna.Org Message-ID: <20020212015027.B26037@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Industry Perspectives, and a Comparison Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All, Joyce and I have been doing a good deal of work to determine what a larger, more comprehensive Infrastructure Architecture consulting firm should look like. We've pinged many of you folks, as well as other technical people such as application developers. More recently we're finding ourselves spending more time with Silicon Valley area startup teams, venture capitalists, project managers, and management consultants, to get their perspectives on how to bring such a beast to market. This, added to my own experience as a V.P. at Chase, has been educational, to say the least -- and I think we had it easy at Chase. Most of the non-sysadmin folks we've talked with tend to carry some (often conflicting) combination of these misconceptions: - systems administration is a highly evolved and stable field - sysadmin staffs always keep UNIX systems under centralized management (confusing "root ownership" with management) - managing a group of small UNIX machines is the same amount of work as managing one large mainframe - more expensive, high-end systems include their own management software - an operating system is a static thing that rarely changes - sysadmin duties consist mainly of upgrading applications and doing backups - an application developer can administer their own development system, no problem - an operating system is a very straightforward piece of technology which comes pre-installed on the hardware, is therefore like firmware, and doesn't need much maintenance - install the hardware and you're ready for apps - reliability and failover are very difficult to do and are therefore the application's responsibility - systems administration is rarely a critical path item during application rollouts or upgrades - centralized administration software is extremely expensive and complex; only large shops can afford it; development of it requires lots of venture capital, followed by patents to protect the investment (usually includes references to Tivoli TME, CA UniCenter, HP OpenView, and/or BMC Patrol) While these misconceptions should be no surprise to anyone on this list, it's still a little disconcerting to face them head-on as part of an intentional research project. We've hit these day after day, while talking with otherwise smart people from whom we're seeking business advice. It's a delicate task, convincing advisers that they really know nothing about how systems administration is done in most shops. We as an industry have done a really lousy job of raising awareness of what we do. Our managers, customers, and clients only see the most superficial surface evidence. In some ways, we are *too* good at making what we do look easy. We're rugged individuals, mostly, and as a group tend to seek refuge in this weird passive-aggressive, BOFH culture that only we understand. And we're damn proud of it... We lack most of the normal attributes of a complex technological field -- standards, certifications, licenses, lobbying groups, regulatory agencies. We have no standards body at all equivalent to IETF. While I'm as much of a devout individualist as anyone, Joyce and I together have finally come to a tentative conclusion that, in order for this industry to mature, there needs to be a more visible body of common knowledge supporting it. Without at least a common set of generally accepted practices, we've got very little to show external parties. * * * I've a lot more to add in terms of what we're planning, but I want to leave you with one thought... In the early 1900's, flying was an individual endeavor. Instruction was by word of mouth, navigation ad-hoc. In 1920 Elrey Jeppesen learned how to fly, trained by Orville Wright. Ten years later, tired of following railroad tracks, tired of weather delays, and tired of losing good friends to bad visibility, Jepp began making detailed notes of airmail routes that worked for him. He started making copies for his friends. Then he began collecting route information from other pilots, printing it all on sets of charts, and selling them by mail. You can find Jeppesen's company today at http://www.jeppesen.com/. Today, every airline pilot in the world uses Jeppesen's charts. The "routes that worked for him" became the foundation of the FAA's US airway system. Anyone interested in drawing some maps? Steve (Have a Happy Chinese New Year!!) -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From hag@linnaean.org Tue Feb 12 19:45:04 2002 Received: from perdition.linnaean.org (h00d0b71b83ad.ne.mediaone.net [65.96.132.240]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id TAA10911; Tue, 12 Feb 2002 19:45:02 -0800 Received: by perdition.linnaean.org (Postfix, from userid 31013) id BF62E1BA14; Tue, 12 Feb 2002 22:44:39 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Steve Traugott Cc: infrastructures@terraluna.org Subject: [Infrastructures] Industry Perspectives, and a Comparison In-Reply-To: <20020212015027.B26037@scramjet.TerraLuna.Org> References: <20020212015027.B26037@scramjet.TerraLuna.Org> X-Mailer: VM 6.34 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty Date: Tue, 12 Feb 2002 22:44:39 -0500 Message-Id: <20020213034439.BF62E1BA14@perdition.linnaean.org> Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > From: Steve Traugott > Date: Tue, 12 Feb 2002 01:50:27 -0800 > > Anyone interested in drawing some maps? > Sign me up (or just keep talking here). > While these misconceptions should be no surprise to anyone on this > list, it's still a little disconcerting to face them head-on as part > of an intentional research project. We've hit these day after day, > while talking with otherwise smart people from whom we're seeking > business advice. It's a delicate task, convincing advisers that > they really know nothing about how systems administration is done in > most shops. > > We as an industry have done a really lousy job of raising awareness of > what we do. Our managers, customers, and clients only see the most > superficial surface evidence. In some ways, we are *too* good at > making what we do look easy. We're rugged individuals, mostly, and as > a group tend to seek refuge in this weird passive-aggressive, BOFH > culture that only we understand. And we're damn proud of it... > > We lack most of the normal attributes of a complex technological field > -- standards, certifications, licenses, lobbying groups, regulatory > agencies. We have no standards body at all equivalent to IETF. While > I'm as much of a devout individualist as anyone, Joyce and I together > have finally come to a tentative conclusion that, in order for this > industry to mature, there needs to be a more visible body of common > knowledge supporting it. Your words bring many thoughts, tho it's difficult to express most of them coherently. I'll take a few stabs. At least at the infrastructure level, a comparison to the lack of an IETF isn't fair. Much of what you think of as "needed to build an infrastructure" is predicated on the existence of open networking protocols. I would think that SAGE would fit here somewhere. I've never been terribly active, so I can't actually say anything about the reality of it. It is very difficult to establish a set of standards, best common practice, etc when many basic terms in system administration seem ill-defined/non-compareable. I've dealt with several large scale, semi-planned/organically grown "infrastructures" (to be generous), and every one of them was managed very differently in the aggregate. Many of the same themes would show up, but culture and business needs were different, and so no two systems expressed anything "infrastructure level" in a congruent fashion. So you almost seem to have a chicken and egg problem. Much of what you want to be able to express with standards will be difficult without a well established set of design patterns in the systems we deploy, and as yet, we don't seem to be there. As I remember you saying elsewhere, you've had a hard time getting "infrastructure style thinking" out there. Seems we need to keep trying, because there still doesn't seem to be a lot of awareness of it, even in the system admin community. I just pointed another person at infrastructures.org today. Anyway, please keep thinking and talking. I've got a lot of infrastructure type stuff burbling in my brain that just hasn't found a way out yet, and it's beginning to bug me. From jblaine@mitre.org Thu Feb 14 08:39:46 2002 Received: from smtpproxy1.mitre.org (smtpproxy1.mitre.org [129.83.20.90]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA30264 for ; Thu, 14 Feb 2002 08:39:44 -0800 Received: from avsrv1.mitre.org (avsrv1.mitre.org [129.83.20.58]) by smtpproxy1.mitre.org (8.11.3/8.11.3) with ESMTP id g1EGcw803402 for ; Thu, 14 Feb 2002 11:38:59 -0500 (EST) Received: from linus.mitre.org (linus.mitre.org [129.83.10.1]) by smtpsrv1.mitre.org (8.11.3/8.11.3) with ESMTP id g1EGcvk24523 for ; Thu, 14 Feb 2002 11:38:57 -0500 (EST) Received: from jblaine-pc.MITRE.ORG (jblaine-pc [129.83.10.132]) by linus.mitre.org (8.9.3/8.9.3) with ESMTP id LAA03937; Thu, 14 Feb 2002 11:38:56 -0500 (EST) Date: Thu, 14 Feb 2002 11:38:56 -0500 From: Jeff Blaine To: infrastructures@TerraLuna.Org cc: jblaine@linus.mitre.org Message-ID: <74454873.1013686736@jblaine-pc.MITRE.ORG> In-Reply-To: <200202122001.MAA31917@roton.terraluna.org> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: [Infrastructures] Re: Infrastructures digest, Vol 1 #14 - 3 msgs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hmm. I have questions about this week's messages, specifically yours Steve. I hope you find them to be questions worth thinking about for a bit before answering. I don't mean to offend anyone, but the comments are pretty frank. Why do you care about the wide dissemination and acceptance of these ideas? What are you gaining from it? Money? Recognition? Peace of mind? Happiness? -- I'm very confused about this. What 'external parties' do you want show these best practices to and why? Have you considered (surely you have) writing a book about this? Do you realize that there have been shelves of "best practice" Software Engineering books available, yet it really hasn't changed the way people write code (because the majority don't "get it" and/or feel it's really worth the effort (again, they don't get it))? IMHO, the knowledge of these best practices is small for a reason. Most sysadmins will hear everything you say, understand it, agree with it, and nod. Then they'll turn around and be themselves, whatever that is. Somewhere in all of that, you reached that 1% who is of the current mindset/openness/brightness to realize it for the gold it is. Your automobile factory automation example is an interesting (yet very dangerous) one. Going a little "out there" now (please think about this separately from the items above :)... Something that just crossed my mind: have you considered the 100-year impact of these concepts? I haven't. What do you think of self-curing machines? When has automation gone too far? Will there _be_ a you or me 100 years from now (I don't believe so upon first instinct). If not, is it because of the control we gave up? Do you care? (I'm not sure if I do yet). From luke@madstop.com Thu Feb 14 08:40:04 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA30360; Thu, 14 Feb 2002 08:39:55 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g1EGWUTs018421; Thu, 14 Feb 2002 10:32:31 -0600 (CST) Date: Thu, 14 Feb 2002 10:32:30 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Steve Traugott cc: infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] Industry Perspectives, and a Comparison In-Reply-To: <20020212015027.B26037@scramjet.TerraLuna.Org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Tue, 12 Feb 2002, Steve Traugott wrote: > More recently we're finding ourselves spending more time with Silicon > Valley area startup teams, venture capitalists, project managers, and > management consultants, to get their perspectives on how to bring such > a beast to market. This, added to my own experience as a V.P. at > Chase, has been educational, to say the least -- and I think we had it > easy at Chase. This is something that needs to be done by our people more and more. The more time sysadmins spend with business people, the more sysadmins will realize that they don't understand us and the more that the business people will realize what a fledgling industry ours is. > Most of the non-sysadmin folks we've talked with tend to carry some > (often conflicting) combination of these misconceptions: [SNIP] The only thing that surprises me about those is the extent to which sysadmins are surprised by them. The thing that frustrates me the most about sysadmins is that they actually believe that the current situation is a good situation. > While these misconceptions should be no surprise to anyone on this > list, it's still a little disconcerting to face them head-on as part > of an intentional research project. We've hit these day after day, > while talking with otherwise smart people from whom we're seeking > business advice. It's a delicate task, convincing advisers that > they really know nothing about how systems administration is done in > most shops. At least they aren't paying your salary and writing your performance reviews, which is the usual relationship sysadmins have with those people. > We as an industry have done a really lousy job of raising awareness of > what we do. Our managers, customers, and clients only see the most > superficial surface evidence. In some ways, we are *too* good at > making what we do look easy. We're rugged individuals, mostly, and as > a group tend to seek refuge in this weird passive-aggressive, BOFH > culture that only we understand. And we're damn proud of it... Individualism is great, and something to be proud of, but I agree that it's interfering with the industry now. In fact, one of the things I don't like about SAGE is that it appears to be just codifying that individualism, rather than kind of stamping it out through standardization. > have finally come to a tentative conclusion that, in order for this > industry to mature, there needs to be a more visible body of common > knowledge supporting it. Yeah, I definitely have to agree with this one. :) This is exactly the online resource I am trying to build, and all of the reasons you list, plus many more, are exactly the reasons why I want this resource. The only place where I see my planned resource and your email diverging are in their respective formalities. One of my main goals is to have a semi-formal way of rating sysadmins WRT eachother, and I am also planning on having formal methods of maintaining all of the information on the site, but it does certainly make sense to attempt to have something like an RFC-style formalization process for fundamentals. I'll have to think about this a bit; I know I'm behind on getting this site up, but I hope to have a mock-up this weekend. -- "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?" --Larry Wall in <1992Aug26.184221.29627@netlabs.com> From jblaine@mitre.org Thu Feb 14 08:59:37 2002 Received: from smtpproxy1.mitre.org (smtpproxy1.mitre.org [129.83.20.90]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA08390 for ; Thu, 14 Feb 2002 08:59:35 -0800 Received: from avsrv1.mitre.org (avsrv1.mitre.org [129.83.20.58]) by smtpproxy1.mitre.org (8.11.3/8.11.3) with ESMTP id g1EGwu808205 for ; Thu, 14 Feb 2002 11:58:56 -0500 (EST) Received: from linus.mitre.org (linus.mitre.org [129.83.10.1]) by smtpsrv1.mitre.org (8.11.3/8.11.3) with ESMTP id g1EGwtk28759 for ; Thu, 14 Feb 2002 11:58:55 -0500 (EST) Received: from jblaine-pc.MITRE.ORG (jblaine-pc [129.83.10.132]) by linus.mitre.org (8.9.3/8.9.3) with ESMTP id LAA05039 for ; Thu, 14 Feb 2002 11:58:54 -0500 (EST) Date: Thu, 14 Feb 2002 11:58:54 -0500 From: Jeff Blaine To: infrastructures@TerraLuna.Org Message-ID: <75652977.1013687934@jblaine-pc.MITRE.ORG> In-Reply-To: <74454873.1013686736@jblaine-pc.MITRE.ORG> X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: [Infrastructures] Re: Infrastructures digest, Vol 1 #14 - 3 msgs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > Will there _be_ a you or me 100 years from now (I don't believe so > upon first instinct). What I meant by that is "Will there be any need for system architects or sysadmins?" not "Will humans exist on this planet still?" That's a whole other mailing list. From duprec@scorec.rpi.edu Thu Feb 14 14:28:59 2002 Received: from mail.rpi.edu (root@mail.rpi.edu [128.113.22.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA17375 for ; Thu, 14 Feb 2002 14:28:58 -0800 Received: from smtp.scorec.rpi.edu (smtp.scorec.rpi.edu [128.113.123.156]) by mail.rpi.edu (8.11.3/8.11.3) with ESMTP id g1EMSnJ113970 for ; Thu, 14 Feb 2002 17:28:49 -0500 Received: from mastermind.scorec.rpi.edu (mastermind.scorec.rpi.edu [10.0.5.17]) by smtp.scorec.rpi.edu (8.10.2/8.10.2) with ESMTP id g1EMSeK15757 for ; Thu, 14 Feb 2002 17:28:40 -0500 (EST) Date: Thu, 14 Feb 2002 17:28:40 -0500 (EST) From: Christophe Dupre To: In-Reply-To: <200202142001.MAA05918@roton.terraluna.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Subject: [Infrastructures] Re: Infrastructures digest, Vol 1 #16 - 3 msgs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Thu, 14 Feb 2002 infrastructures-request@TerraLuna.Org wrote: > Date: Thu, 14 Feb 2002 11:38:56 -0500 > From: Jeff Blaine > To: infrastructures@TerraLuna.Org > cc: jblaine@linus.mitre.org > Subject: [Infrastructures] Re: Infrastructures digest, Vol 1 #14 - 3 msgs > > Hmm. > > I have questions about this week's messages, specifically yours Steve. > I hope you find them to be questions worth thinking about for a bit > before answering. I don't mean to offend anyone, but the comments > are pretty frank. > > Why do you care about the wide dissemination and acceptance of these > ideas? What are you gaining from it? Money? Recognition? Peace > of mind? Happiness? -- I'm very confused about this. I don't know if the questions abive are directed to Steve, Luke, or the community, but here's my take on it. Wide dissemination of these ideas can lead to more productive work. When working with other admins on certain projects, so much time is lost because of communication problems. There are so many terms used, with various meanings. Without going into details, when I talk about an infrastructure server, I'm thinking of a jumpstart server, with CVS for config files and scripts to control the installation of software. Others will think DNS, LDAP, etc. As a parallel, when doing tech support over the phone, how many time you say to the dummy user : 'is there a such and such directory in the window?' and he'll reply 'no, but there's a folder'. The same thing can be called a folder, file or directory, depending on your background. I sure hope most people on this list are not doing such low-level support, but the concept applies. Now, if those leading the effort get money, recognition, etc - great for them. My personal benefit is some common terminology and some recognized standards to give my PHB to support my budget requirements :-) > Do you realize that there have been shelves of "best practice" > Software Engineering books available, yet it really hasn't changed the > way people write code (because the majority don't "get it" and/or > feel it's really worth the effort (again, they don't get it))? And yet some use them, and benefit from them. > Going a little "out there" now (please think about this separately > from the items above :)... Something that just crossed my mind: > have you considered the 100-year impact of these concepts? I > haven't. What do you think of self-curing machines? When has > automation gone too far? Will there _be_ a you or me 100 years > from now (I don't believe so upon first instinct). If not, is it > because of the control we gave up? Do you care? (I'm not sure > if I do yet). I have no idea what's going to happen in 100 years. I don't expect to be around, so I don't really care. The past 50 years have seen some huge advances in the ease of installation/maintenance/use of computers. And yet, they still require trained/experienced people to properly care for them. I expect this to still be the case in the next 50 years, with probably the ability to manage a greater number of computers by myself, with hopefully more time to do more long-term planning. A single computer used to require an entire priesthood to care for it. Now a single admin can manage tens of computers (hundreds with the proper infrastructure in place). Maybe that will become thousands, but the need will probably still exist. Cheers! -- Christophe Dupre System Administrator, Scientific Computation Research Center Rensselaer Polytechnic Institute Troy, NY USA Phone: (518) 276-2578 - Fax: (518) 276-4886 From jblaine@linus.mitre.org Fri Feb 15 09:30:09 2002 Received: from smtpproxy1.mitre.org (smtpproxy1.mitre.org [129.83.20.90]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id JAA04725 for ; Fri, 15 Feb 2002 09:30:08 -0800 Received: from avsrv1.mitre.org (avsrv1.mitre.org [129.83.20.58]) by smtpproxy1.mitre.org (8.11.3/8.11.3) with ESMTP id g1FHTL813793; Fri, 15 Feb 2002 12:29:21 -0500 (EST) Received: from linus.mitre.org (linus.mitre.org [129.83.10.1]) by smtpsrv1.mitre.org (8.11.3/8.11.3) with ESMTP id g1FHTIk23232; Fri, 15 Feb 2002 12:29:19 -0500 (EST) Received: from localhost (jblaine@localhost) by linus.mitre.org (8.9.3/8.9.3) with ESMTP id MAA14372; Fri, 15 Feb 2002 12:29:18 -0500 (EST) Date: Fri, 15 Feb 2002 12:29:18 -0500 (EST) From: Jeff Blaine To: Christophe Dupre cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: Infrastructures digest, Vol 1 #16 - 3 msgs In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > I don't know if the questions abive are directed to Steve, Luke, or the > community, but here's my take on it. Anyone. > Wide dissemination of these ideas can lead to more productive work. When > working with other admins on certain projects, so much time is lost > because of communication problems. There are so many terms used, with > various meanings. Without going into details, when I talk about an > infrastructure server, I'm thinking of a jumpstart server, with CVS for > config files and scripts to control the installation of software. Others > will think DNS, LDAP, etc. > > As a parallel, when doing tech support over the phone, how many time you > say to the dummy user : 'is there a such and such directory in the > window?' and he'll reply 'no, but there's a folder'. The same thing can > be called a folder, file or directory, depending on your background. > > I sure hope most people on this list are not doing such low-level support, > but the concept applies. So, you want "Gold OS Server" to become globally defined as 1 thing that sysadmins know (or something similar)? As long as implementation details are left out of that definition, I am all for the benefits of that. The ideas are what count the most. The specifics of an implementation of such ideas are bonus proof of concept case studies. > > Do you realize that there have been shelves of "best practice" > > Software Engineering books available, yet it really hasn't changed the > > way people write code (because the majority don't "get it" and/or > > feel it's really worth the effort (again, they don't get it))? > > And yet some use them, and benefit from them. Absolutely. > > Going a little "out there" now (please think about this separately > > from the items above :)... Something that just crossed my mind: > > have you considered the 100-year impact of these concepts? I > > haven't. What do you think of self-curing machines? When has > > automation gone too far? Will there _be_ a you or me 100 years > > from now (I don't believe so upon first instinct). If not, is it > > because of the control we gave up? Do you care? (I'm not sure > > if I do yet). > > I have no idea what's going to happen in 100 years. I don't expect to be > around, so I don't really care. Do you care about the Earth's environment at all? If so, why do you care about that if you're not going to be around if it all crumbles? I'm not saying _I_ necessarily care... or don't care. > The past 50 years have seen some huge advances in the ease of > installation/maintenance/use of computers. And yet, they still require > trained/experienced people to properly care for them. I expect this to > still be the case in the next 50 years, with probably the ability to > manage a greater number of computers by myself, with hopefully more time > to do more long-term planning. > > A single computer used to require an entire priesthood to care for it. Now > a single admin can manage tens of computers (hundreds with the proper > infrastructure in place). Maybe that will become thousands, but the need > will probably still exist. Agreed, but my point is, who says that 1 guy maintaining the 500 boxes is you in 5 years? If they only need 1 guy instead of 5 or 10, then 4 or 9 are out of jobs. Just like auto workers and just like the guy who used to sit and monitor factory systems via analog meters/gauges. Hey, I don't mean to give the impression that I'm one of those sleazy people who goes around creating havoc quietly on systems then fixing it for job security. I'm on your side. I've been working toward these same (almost exact) infrastructure ideas for some time now. I do plenty every year to decrease the amount of hands-on "dumb" effort involved in my job. I've only just begun to consider the ramifications of such things though and it's interesting to me. It's something to think hard about. 20 years is an eternity in the computing world as you all well know. Sure, hardware failures (and someone to take care of them) will be an issue for quite some time, but I wouldn't be surprised to see the demand for our field drastically reduced in 20 years due to OS stability, self-monitoring/adjusting administration portion to OSes, etc. Unless you're 45 or over, that might be a reality while you're in the workforce still. Take care -- thanks for the reply. From partain@dcs.gla.ac.uk Sat Feb 16 04:41:32 2002 Received: from iona.dcs.gla.ac.uk (iona.dcs.gla.ac.uk [130.209.240.35]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id EAA22654 for ; Sat, 16 Feb 2002 04:41:15 -0800 Received: from slicker.dcs.gla.ac.uk ([130.209.242.51] helo=dcs.gla.ac.uk) by iona.dcs.gla.ac.uk with esmtp (Exim 3.13 #1) id 16c49D-0004iT-00; Sat, 16 Feb 2002 12:40:59 +0000 To: infrastructures@terraluna.org cc: couch@eecs.tufts.edu X-Mailer: mh-e 6.0; nmh 1.0.4; Emacs 21.4 Date: Sat, 16 Feb 2002 12:40:58 +0000 From: Will Partain Message-Id: Subject: [Infrastructures] sysadmin: "problem" or "network" field? [mostly from Agre/RRE] Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Folks, a little weekend reading for the bath, below... By Phil Agre, from his Red Rock Eater (RRE) mailing list. The article may throw light on the question, "what kind of field (of knowledge) is `system administration'"? In particular, is sysadmin a "problem" field, or a "network" field (terms defined below)? Conjecture: it's a "network" field. This has biggish implications, e.g.: (a) most of our can't-get-no-respect difficulties follow from living/working cheek-by-jowl with a "problem" field, namely computer science; (b) efforts to have us grow up into a proper ("problem") research field are, um, doomed; (d) we're more like library science than computer science; (d) we probably need some "network"-y way of structuring our body of knowledge; and (e) probably lots more :-) Comments, etc., welcome. Will ===================================== Date: Sun, 27 Jan 2002 22:54:17 -0800 Message-Id: <200201280654.WAA68962@alpha.oac.ucla.edu> From: Phil Agre To: "Red Rock Eater News Service" Subject: [RRE]notes and recommendations Some notes about distributed objects, technology-driven change, and the diversity of knowledge. [snips -will] ** Networks and problems. Different fields produce different kinds of knowledge. The idea of a diversity of knowledge, however, intimidates many people; it sounds to them like relativism, as if *anything* can count as knowledge if someone simply says so. That's silly; no such thing follows. Even so, it *is* a hard problem to understand how knowledge functions in society if knowledge is diverse, for example how to tell the difference between quality-control and censorship. The scholars who have argued for the diversity of knowledge, despite the quality of their research, have often been unconcerned with the public-relations problem that their insights suffer. They can win the argument about relativism when they are arguing with people equally as erudite as themselves, but they have historically not done a good job of translating the arguments into a rhetoric that wins public debates. That's partly because they are so concerned to defeat the mythology of unitary knowledge that they emphasize heterogeneity more than they emphasize the limits to heterogeneity. That's too bad, because the diversity of knowledge actually turns out to be related to the Internet's place in society. Let me suggest an intuitive way to think about the differences between different kinds of knowledge. To simplify, I'll stick with academic fields. Every academic field, I will suggest, has two dimensions: problem and network. By the "problem" dimension of knowledge I mean the ways in which research topics are framed as discrete and separable, so that researchers -- whether individuals or teams -- can dig into them and produce publishable results without enaging in far-flung collaborations. By the "network" dimension of knowledge I mean the ways in which researchers organize themselves across geographical and organizational boundaries to integrate experience from many different sites. Every field has its own complexity in both of these dimensions, but often the emphasis is on one dimension or another. As a result, we can roughly and provisionally categorize academic fields as "problem" fields and "network" fields. The prototype of a "problem" field is mathematics. Think of Andrew Wiles, who disappeared into his study for several years to prove Fermat's Last Theorem. The hallmark of "problem" fields is that a research topic has a great deal of internal depth and complexity. The math in Wiles' proof may seem like vast overkill for something so simple as the statement of Fermat's Last Theorem, but you can think of it as an engineering project that finished building a bridge over a conceptual canyon. Publicity value aside, the mathematicians value the bridge because they hope that it's going to carry heavier traffic in the future. Even so, it's not clear that Wiles' type of math represents the future. Math papers are more likely to be coauthored than in the old days, as mathematicians work increasingly by bringing different skills together. This is partly a legacy of the major math project of the 20th century, which aimed at the grand unification of fields rather than producing heavier theorems in a single area. That unification project opened up many seams of potential results along the edges between different areas of math. The increasing practical applicability of even very abstruse areas of math (e.g., in cryptography) didn't hurt either. Even so, math is still weighted toward the "problem" dimension. Math people do form professional networks like anyone else, but the purpose of these networks is not so much to produce the knowledge as to ensure a market for it. The same thing is true in computer science, where professional networks also help with funding. And those are not the only problem fields. Cultural anthropology is a good example. The anthropologist goes to a distant island, spends two years learning the culture, and writes a book that uses it as raw material to explore a particular theoretical problem in depth. The "problem" nature of cultural anthropology is partially an artefact of technology; if long-distance communication is hard then it's easier to uphold the myth that humanity comes sorted into discrete cultures, and a fieldworker who travels great distances to study a culture has no choice but to define a large, solitary research project. But that doesn't change the fact that the best anthropology (and there's a lot of good anthropology being written) has intellectual depth to rival anything being done in computer science, even if the conceptual and methodological foundations of the research could hardly be more different. Contrast these fields to some others: medicine, business, and library science. Medicine, business, and library science may not seem similar on the surface, but they have something important in common: they are all network-oriented. Because they study something that is complex and diverse (illnesses, businesses, and information), they build their knowledge largely by comparing and contrasting cases that arise in professional practice. Physicians don't make their careers by solving deep problems or having profound ideas; they make their careers by building networks that allow them to gather in one central location the phenomenology of a syndrome that has not yet been systematically described. Medical knowledge is all about experience-based patterns. It says, we've seen several hundred people with this problem, we've tried such-and-such treatments on them, and this is what happens. Business is the same way: we've investigated such-and-such an issue in the context of several businesses, and this is the pattern we've discerned. Library science, likewise, is concerned to bring order to the diversity of information as it turns up in the collections of library institutions worldwide. When mathematicians look at business or computer scientists look at library science, they often scoff. They have been taught to value "problems", and they are looking for the particular kind of "depth" that signifies "good work", "real results", and so on. When they don't find what they are looking for, they often become disdainful. The problem is that they are looking in the wrong place. The don't realize that the "problems" that they are familiar with are largely artificial constructions. To fashion those kinds of problems, you need to take several steps back from reality. You're abstracting and simplifying, or more accurately someone else is abstracting and simplifying for you. Many job categories are devoted to suppressing the messy details that threaten to falsify the abstractions of computer science, starting with the clerks whose computer terminals demand that they classify things that refuse to be classified. The dividing-line between computer science and the business-school discipline of "MIS" is especially interesting from this point of view, since the MIS managers are much closer to the intrinsic complexity and diversity of day-to-day business. Computer scientists, as a broad generalization, have little feeling for the complexity and diversity of the real world. That's not to say that they are bad people or defective intellects, only that the field of computer science frames its knowledge in certain ways. It takes all kinds to make a world, and that goes for knowledge as well. We should encourage the creative tension between problem field and network fields, rather than arguing over who is best. Medicine is an interesting case for another reason. Even though problem fields are higher-status than network fields as a broad generalization, medicine is an exception to the rule. If my theory is right, then, why doesn't medicine fall into the same undeservedly low-status bin as business and library science? The reasons are obvious enough. Medicine is a business unto itself -- at UCLA it's half the university's budget -- and it brings money in through patient fees, insurance reimbursements, and Medicare, as well as through research grants and student tuition. Money brings respect, all things being equal, although the increasingly problematic finances of teaching hospitals will test this dynamic in the near future. Medicine is also very aggressive in the way it wields symbols -- it's hard to beat life and death for symbolic value. What's more, business and library schools have stronger competitors than medical schools, so they have a greater incentive to speak in plain English. Precisely because they rely so heavily on symbols, medical schools have never had to explain how their knowledge works in ways that normal people can understand. Professional schools in general tend to produce knowledge that is more network-like than problem-like, but historically they have very often responded to the disdain of the more problem-oriented fields by trying to become more problem-oriented themselves. This strategy is very old; in fact Merton described it perhaps fifty years ago. Unfortunately, it doesn't always work. You end up with professional schools whose faculties are trained in research methods that are disconnected from the needs of their students, or else you end up with factionalized schools that are divided between the scientists and the fieldworkers, or with people whose skills lie in network methods trying to solve problems because that's what the university wants. I think this is all very unfortunate. I'm not saying that every field should be homogenous, and even if everyone does the research they ought to be doing we'll still have the problem of how scholars with incommensurable outlooks can get along. Still, the asymmetry of respect between network knowledge and problem knowledge is most unfortunate. I think the world would be better off if network knowledge were just as venerated as problem knowledge. Before this can happen, we need better metaphors. We are full of metaphors for talking about the wonders if problem knowledge, as we ought to be. When Andrew Wiles can go off in his room and prove Fermat's Last Theorem, that's a good thing, and there's nothing wrong with using the metaphor of "depth" to describe it. It's just that we need metaphors on the other side. So here's a metaphor. I propose that we view the university as the beating heart of the knowledge society. The heart, as we all know, pulls in blue blood from all over the body, sends it over to the lungs until it's nice and red with oxygen, and then pumps it back out into the body. The university does something similar, and the predominant working method of business schools can serve as a good way to explain it. If you read business journals, especially journals such as the Harvard Business Review that are largely aimed at a practitioner audience, you will often see two-by-two matrices with words written in them. These sorts of simple conceptual frameworks (which I've talked about before) are a form of knowledge, but it's not widely understood what form of knowledge they are. Once we understand it, we'll be able to see how the university is like a heart. So let's observe that there are at least two purposes that knowledge can serve: call them abstraction and mediation. Abstraction is the type of knowledge that the West has always venerated from Plato's day forward. It is something that rises above concrete particulars; in fact, it carries the implicit suggestion that concrete particulars are contaminants -- "accidents" is the medieval word -- compared to the fixed, permanent, perfect, essentially mathematical nature of the abstractions. Abstractions generalize; they extract the essence from things. They are an end in themselves. In Plato's theory we were all born literally knowing all possible knowledge already, since access to the ideals (as he called them) was innate. That made questions of epistemology (i.e., the study of the conditions of knowledge) not so urgent as they became subsequently, as the West began to recognize the absurdity of a conception of knowledge that is so completely detached from the material world. But if knowledge can abstract, it can also mediate. The purpose of the two-by-two matrices in the business journals is not to embody any great depth in themselves, the way a theorem or an ethnnography might. Instead, their purpose is to facilitate the creation of new knowledge in situ. Choose a simple conceptual framework (transaction costs, core competencies, structural holes, portfolio effects), and take it out into real cases -- two or more, preferably more. Study what each conceptual framework "picks out" in each case; that is, use the conceptual framework to ask questions, and keep asking questions until you can construct a story that makes sense within the logic of that particular case. That's important: each case has its details, and each case is filled with smart people who have a great deal of practical knowledge of how to make a particular enterprise more or less work. So work up a story that makes sense to them, that fits with their understandings, yet that is framed in terms of the concepts you've brought in. Of course, that might not be possible; your new concepts may not pick out anything real in a particular case, in which you need to get new concepts. But once you've found concepts that let you make sense of several cases, now you can compare and contrast. And that's where the real learning happens. Even with the concepts held constant, each case will tend to foreground some issues while leaving others in the background. Take the issues that are foreground in case A, and translate those issues over to cases B, C, D, and E, asking for each of them what's going on that might correspond to the issue from case A. It doesn't matter whether the other cases are all directly analogous to case A; even if the issue sorts out differently in those other cases, the simple fact that you've thought to ask the question will provoke new thoughts that may never have occurred to anybody before. That's what I mean by the mediating role of knowledge: it mediates the transfer of ideas back and forth between situations in the real world that might not seem at all comparable on the surface. And that's the beating heart: what the university does is fashion concepts that allow ideas to be transferred from one setting to another. Each setting has its own language, so the university invents a lingua franca that gets conversation started among them. At first the ideas will pass through the doors of the university. A researcher will go out to several different sites, gather ideas, bring them home, think about them, and then scatter them in other sites. Eventually the concepts themselves will be exported, so that students who graduate into companies or consulting firms will become beating hearts on their own account. (That's a place where the analogy falters: maybe the university is more like a manufacturer of hearts.) We in modern society take for granted something remarkable: that nearly every site of practice is on both the donating and the receiving end of these mediated transfers of ideas. Often we don't realize it because the people who import ideas by mediation from other fields will often present them full-blown, without bothering to explain where they got them. Other times, a kind of movement will get going whereby researchers and practitioners unite across disciplinary lines around a particular metaphor that they find useful for mediating transfers among themselves: self-organization is one of the fashionable metaphors of the moment. Mediating concepts can be used in various ways, but in general what you see is a mixture of two approaches: explicit comparing/contrasting of particular cases and something that looks more like abstraction. The resulting abstractions, however, usually have no great depth in themselves; their purpose is simply to summarize all of the issues and ideas and themes that have come up in the various cases, so that all of them can be transferred to new situations en masse. This is what "best practices" research is. It's also what physicians do when they codify the knowledge in a particular area of medicine; the human body is too complicated, variable, and inscrutable to really understand in any great depth, and so codified medical knowledge seeks to overwhelm it with a mass of experience loosely organized within some operational concepts and boiled down into procedures that can be taught, and whose results can be further monitored. This is the important thing about network knowledge: it really does operate in networks -- meaning both social networks and infrastructures -- and networks are institutions that have to be built and maintained. In a sense, network knowledge is about surveillance, and mediating concepts exist to render the results of surveillance useful in other places. The mediating role of concepts can help us to explain many things. It is a useful exercise, for example, to deliberately stretch the idea of mediation to situations where its relevance is not obvious. Philosophy, for example, has long been understand as the ultimate abstraction, something very distant from real practice. This is partly a side-effect of the unfortunate professionalization of philosophy that led to the hegemony of analytical philosophy in the English-speaking world perhaps a century ago, but really it dates much further back into the ancient Greek mythologies of ancient times. The popular conception of philosophy as the discipline of asking questions with profound personal meaning is almost completely unrelated to the real practice of philosophy at any time or place in history. There are exceptions. One of Heidegger's motivations, especially in his earliest days, was to reconstruct philosophy around the kinds of profound meanings that he knew from Catholic mysticism. Some political philosophers have tried to make themselves useful to actual concrete social movements. But for the most part, philosophy has been terribly abstract from any real practice. Yet, if we take seriously the mediational role of concepts, then maybe the situation is more complicated. One role of the university is precisely to create concepts that are so abstract that they can mediate transfers of ideas between fields that are very distant indeed. Perhaps we could go back and write a history of the actual sources of scholars' ideas, and maybe we would find that the very abstract concepts that scholars learned in philosophy often helped them to notice analogies that inspire new theories. Analogies have long been recognized as an important source of inspiration for new discoveries, especially in science but in other fields as well, and nothing facilitates the noticing of analogies so efficiently as an abstract idea that can be used to describe many disparate things. I would like to see the university take the mediating role of concepts more seriously. I would like every student to be taught a good-sized repertoire of abstract concepts that have historically proven useful for talking about things in several disparate fields -- examples might include positive and negative feedback, hermeneutics, proof by contradiction, dialectical relationships, equilibrium concepts from physics, evolution by natural selection, and so on -- and teach them not as knowledge from particular fields, but as schemata that help in noticing analogies and mediating the transfer of ideas from one topic to another. The students would be drilled on the use of these concepts to analyze diverse cases, and on comparing and contrasting whatever the analyses turn up, and then they be sent off to take classes in their chosen majors. After a while we could do some intellectual epidemiology to see which of the concepts actually prove useful to the students, and we could gradually evolve the curriculum until we've identified the most powerful concepts. I do realize the problem with this proposal: it is bound to set off power struggles along political lines, and between the sciences and humanities, over the best repertoire of concepts to teach. But that's life. The mediating role of concepts, and network knowledge generally, are also a useful way to re-understand fields that we normally understand mostly in terms of their problem knowledge. (You'll recall that my classification of fields as "network fields" and "problem fields" is a heuristic simplification, and that every field has both dimensions.) What is the network-knowledge dimension of math or computer science? I've already described one role of professional networking in each field, which is to provide an audience for one's work. All research depends on peer review, so it's in your interest to get out there and explain the important of your research to everyone who might be asked to evaluate it. Likewise, if you need funding for your research then you'll probably want to assemble a broad coalition of researchers who explain the significance of their proposed research in similar ways, so that you can approach NSF or the military with a proposition they can understand. But none of that speaks to the network nature of the knowledge itself. What is network-like about knowledge in math and computing? It's true that neither field employs anything like the case method. But they do have something else, which is the effort to build foundations. Much of math during the 20th century, as I mentioned, was organized by the attempt to unify different fields, and that means building networks of people with deep knowledge in different areas. Only then can proposed foundations be tested for their ability to reconstruct the existing knowledge in each area. In computing, the search for foundations takes the form of layering: designing generic computer functionality that can support diverse applications. In that kind of research, it's necessary to work on applications and platforms simultaneously, with the inevitable tensions that I also mentioned above. So in that sense math and computer science have a network dimension, and I think that each field would profit by drawing out and formalizing its network aspects more systematically. Even though anthropology is built on deep case studies, the network nature of its knowledge becomes clearer as you speak with the more sophisticated of its practitioners. Anyone who engages seriously with the depths of real societies is aware that theoretical categories apply differently to different societies, and that there's a limit to how much you can accomplish by spinning theories in abstraction from the particulars of an ethnographic case. I am basically a theorist myself, but I realize that my research -- that is, the theoretical constructs I describe -- is only valuable for the sense it makes of particular cases. So I read case studies, and I try to apply my half-formed concepts to those cases, or else I draw on concepts that have emerged from particular cases, and then I try to do some useful work with them. My work is also influence by personal experience, usually in ways that I don't write about. But I can only go so far before it's time to start testing the concepts against real cases again, and that's why I often move from one topic to another, contributing what I can until I feel like I'm out on a limb, beyond what I can confidently say based on existing case studies and common knowledge. It *is* possible to useful things without being directly engaged with cases, for example pointing out internal inconsistencies in existing theories, sketching new areas of research that other people haven't gotten around to inventing concepts for, noticing patterns that have emerged in the cases so far, or comparing and contrasting theoretical notions that have arisen in different contexts. But if you believe that theory can blast off into space without any mooring in real cases then you're likely to do the sort of pretentious big-T Theory that gives us all a bad name. Anthropologists are thoroughly infused with that understanding, and so the best ones really do refuse abstraction. They see their theoretical constructs very much as ways of mediating between different sites. Their concern is not practical, so they are not interested in moving ideas from one site to another on a material level. They are usually not trying to help the people they study. Rather, they are interested in describing the fullness of the social reality they find in a given place, and like the business people they understand that the real test is the extent to which their story about a particular case makes internal sense. Granted, they are less concerned than the business people to be understandable to the people they are studying, although that too is changing as the "natives" become more worldly themselves, and as it becomes more acceptable by slow degrees to study "us" as well as "them". In any case, I think that the anthropologists' relationship to theory is healthy, and I wish I could teach it to people in other fields. Anthropology is also becoming more network- like as reality becomes more network-like, and as the myth of discrete cultures becomes more and more of an anachronism, but that's a topic for another time. Knowledge is diverse because reality is diverse. In fact, reality is diverse on two levels. A field like medicine, business, or library science derives knowledge by working across the diversity of illnesses, businesses, and information, gathering more or less commensurable examples of each under relatively useful headings that can be used as to codify and monitor practice. And then the various fields themselves are diverse: they are diverse in diverse ways. Fields that pride themselves on abstraction operate by suppressing and ignoring diversity. That can be okay as a heuristic means of producing one kind of knowledge -- knowledge that edits the world in one particular way, and that can be useful when recombined with knowledge that edits the world in other ways. But it's harmful when abstraction is mistaken for truth, and when fields that refuse to abstract away crucial aspects of reality are disparaged as superficial compared to the artificial depth at the other end of campus. Let's keep inventing metaphors that make network-oriented fields sound just as prestigious and heroic as problem-oriented fields. The point, of course, is not just to mindlessly praise the work, since bad research can be done anywhere. The point, rather, is to render intuitive the standards that can and should guide us in evaluating research of diverse types. If we don't, then we will disserve ourselves by applying standards that don't fit, or else no standards at all. end From partain@dcs.gla.ac.uk Sat Feb 16 05:14:44 2002 Received: from slicker.dcs.gla.ac.uk (slicker.dcs.gla.ac.uk [130.209.242.51]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id FAA10884 for ; Sat, 16 Feb 2002 05:14:37 -0800 Received: (from partain@localhost) by slicker.dcs.gla.ac.uk (8.8.8+Sun/8.8.8) id NAA25047; Sat, 16 Feb 2002 13:14:28 GMT X-Authentication-Warning: slicker.dcs.gla.ac.uk: partain set sender to partain@dcs.gla.ac.uk using -f To: infrastructures@TerraLuna.Org References: <20020212015027.B26037@scramjet.TerraLuna.Org> From: Will Partain Date: 16 Feb 2002 13:14:28 +0000 In-Reply-To: <20020212015027.B26037@scramjet.TerraLuna.Org> Message-ID: Lines: 71 User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.4 (Common Lisp) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Infrastructures] Re: Industry Perspectives, and a Comparison Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Steve Traugott writes: > Most of the non-sysadmin folks we've talked with tend to carry some > (often conflicting) combination of these misconceptions: > > [interesting list! -will] > > We as an industry have done a really lousy job of raising awareness of > what we do. Our managers, customers, and clients only see the most > superficial surface evidence. In some ways, we are *too* good at > making what we do look easy. ... It's as with the Maytag repairman of yore: if you're *really* good, no-one even knows you exist. This seems to be what Murphy is on about in http://www.winface.com/blurb.html (? has anyone seen/read the whole book?): The Unix Guide to Defenestration This book explains that most commercial systems work disappoints because the incentives favor exactly the kind of continuous low level failure we usually see. Systems management careers are enhanced by budget growth and staff expansion, both of which are maximized by maintaining a level of non performance that teeters on the edge of catastrophe. Correspondingly, systems budgets and staffing levels, and therefore management careers, are diminished by successful execution of the cost reduction, or cost avoidance, mandates that normally go with the job. When companies pay for failure, they get failure. Throw away enough money as a CIO and you'll make sombody's annual "100 best IT users" list, but deliver services with the effectiveness and reliability of the phone system while meeting your mandate to cut costs and you won't get promoted; you won't be attending executive committee meetings; and most people --including recruiters, your colleagues, and your bosses-- will dismiss you as a loser whose budget and visibility are fading while theirs are growing. > Without at least a common set of generally accepted > practices, we've got very little to show external parties. I, for one, am always a bit jumpy about sets (well, usually one) of "accepted practices" or "standards" or whatever. * They very often turn into a political tool ("the reason you should give me lots of money and power is so I can `roll out' this very wonderful generally-accepted standard across the organization"). * When I'm trying to solve a problem, I don't *mind* seeing a general "best-practices" doc; but what I would *kill for* is a few concise utterly-unambiguous descriptions of how you *actually* solved the same-or-similar problem. * A sysadmin's goal should always be "competitive advantage through better systems". This may involve breaking with best practice and doing something weird. Back to politics again -- the dead weight of "industry standards" can then hold you back; cf. the dead weight of SEI CMM levels holding back people who want to be doing great software, and get to fill in forms instead :-) (Steve, obviously I am not against what you're describing. But these are some worries.) Will From luke@madstop.com Sun Feb 17 17:48:29 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA02194 for ; Sun, 17 Feb 2002 17:48:27 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g1I1XfCI026636; Sun, 17 Feb 2002 19:33:41 -0600 (CST) Date: Sun, 17 Feb 2002 19:33:41 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Jeff Blaine cc: Christophe Dupre , infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: Infrastructures digest, Vol 1 #16 - 3 msgs In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, 15 Feb 2002, Jeff Blaine wrote: > > As a parallel, when doing tech support over the phone, how many time you > > say to the dummy user : 'is there a such and such directory in the > > window?' and he'll reply 'no, but there's a folder'. The same thing can > > be called a folder, file or directory, depending on your background. [SNIP] > > So, you want "Gold OS Server" to become globally defined as 1 thing that > sysadmins know (or something similar)? It's not so much that particular term as a group of terms specifically developed or accepted for dealing with this environment. For example, it's taken my team hours of discussion to finally agree on terms for defining how we will talk about our infrastructure. We've decided on the following definitions (and by "definition", I mean we are adding meaning to a phrase, no relying on existing meaning): Management Infrastructure The services required to manage and maintain the rest of the infrastructure. This is basically the CVS, rsync, and gold servers currently, but is not necessarily restricted to those. Foundation Infrastructure The RFC-compliant services which all other services rely on, such as DNS, LDAP, and NTP. Infrastructure Integration Process The process by which an existing service is integrated into the newly managed infrastructure. These definitions are basically from memory, and aren't the wording we used at work, but they serve. Until we came up with these defnitions, we continually confused each other because we'd use terms that weren't specifically developed for infrastructure work. > > > Do you realize that there have been shelves of "best practice" > > > Software Engineering books available, yet it really hasn't changed the > > > way people write code (because the majority don't "get it" and/or > > > feel it's really worth the effort (again, they don't get it))? >> > > And yet some use them, and benefit from them. One big difference between programming and system administration is that there are classes on programming, but you certainly can't get a degree in system administration. There are thousands of sysadmins out there who have had basically no training on how to do their jobs; they'll continue to struggle along if we give them no help, but if we provide a good source of information for them, not only will they do their jobs better and hopefully bring better visibility to our industry, there's a good chance they'll actually give something back in the way of documentation, code, or just plain experience. There are a lot of ways in which system administration and programming are similar, but until we have a similaryly structured teaching process, there will be some fundamental differences. > Agreed, but my point is, who says that 1 guy maintaining the 500 boxes > is you in 5 years? If they only need 1 guy instead of 5 or 10, then > 4 or 9 are out of jobs. Just like auto workers and just like the guy > who used to sit and monitor factory systems via analog meters/gauges. Frankly, that doesn't bother me. I'd rather have a good car than a car which employed 5 times as many people (in both building and maintaining) but cost me much more in initial cost and in maintenance. I don't really notice a shortage of jobs in the car industry. It may look like it employs fewer people, but really, cars would not have proliferated to the extent it has if they were still being hand-built. The main skills that the top sysadmins have to have are design and coding. Given that, if everything becomes much more automated, the people are who currently the best sysadmins will transition to being those who design and code those infrastructure automation tools. Those who are not great at design or coding will transition instead to implementors. I don't think the number of jobs will really decrease; instead, the jobs will just specialize much more. Yeah, a bit more boring, but frankly, if that makes computers suck less, I'm all for it. And yes, apparently I'm one of those few people who love computers but think that the state of computing is incredibly embarassing. I think unix is the best we have right now, but I'm pretty embarassed that an OS developed 30 years ago is the best we have. I'm also incredibly embarassed that our industry doesn't just absolutely require that all (corporate) computers be maintained with automated tools--hell, I'm embarassed that those tools aren't even available. > It's something to think hard about. 20 years is an eternity in the > computing world as you all well know. Sure, hardware failures (and > someone to take care of them) will be an issue for quite some time, > but I wouldn't be surprised to see the demand for our field drastically > reduced in 20 years due to OS stability, self-monitoring/adjusting > administration portion to OSes, etc. > > Unless you're 45 or over, that might be a reality while you're in > the workforce still. I really don't think this will be a problem; instead, I think the problem of incompetent sysadmins who don't know code or design will kind of go away, and the sysadmins who can do both very well will instead be paid to actually do them. I really believe that if we can make this all work, there will be more jobs, not fewer, in system administration. They'll just be in different tiers. You'll have design organizations, code organizations, mechanic organizations, and many more. -- "Reality is that which, when you stop believing in it, doesn't go away." -- Philip K. Dick, "How to Build a Universe" From smichaels@houston.rr.com Fri Mar 1 06:21:29 2002 Received: from sm11.texas.rr.com (sm11.texas.rr.com [24.93.35.42]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id GAA24701 for ; Fri, 1 Mar 2002 06:21:27 -0800 Received: from computer (cs6669173-20.houston.rr.com [66.69.173.20]) by sm11.texas.rr.com (8.12.1/8.12.0) with SMTP id g21EI6KW026693; Fri, 1 Mar 2002 08:18:08 -0600 Message-ID: <005d01c1c12c$5a97bca0$14ad4542@houston.rr.com> Reply-To: "Steve Michaels" From: "Steve Michaels" To: Cc: "Dee Dee Brantley" , "Louise Coffelt" Date: Fri, 1 Mar 2002 08:21:15 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: [Infrastructures] Questions regarding a VP, Infrastructure & Operations job posting Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Infrastructure Professionals, Futurestep is currently seeking a VP, Infrastructure & Operations professional for a client in Indianapolis, and due to the lack of qualified candidates submitting their resumes, I am hoping you can lend me some pearls of wisdom on how I and Futurestep's client can improve our chances with the wording and expectations stated in this req? When doing so, could you also let me know if any of the requirements may be a roadblock for getting quality, qualified candidates, other than the obvious "fluent in Spanish" requirement. Scroll down below my contact information to see the posting. Your help will be greatly appreciated. Best Regards, Steve Michaels, CPC / AIRS CIR Senior Consultant Futurestep (a Korn/Ferry Company) 281-651-9184 (Office) 832-563-9349 (Mobile) smichaels@houston.rr.com http://www.futurestep.com JOB TITLE: VP, Infrastructure & Operations LOCATION: Indianapolis, IN RELOCATION: Employer paid COMPENSATION: Highly Competitive Base + up to a 30% Bonus SPECIAL REQUIREMENT: MUST be fluent in SPANISH and English Our client wants to be the employer of choice for diversity candidates and is at the forefront of an initiative to hire bright people who have diverse sets of experiences, ideas and backgrounds. The company is incredibly energetic and dynamic. They want their people to succeed. Our client, a $10B diversified firm, is one of the foremost providers of travel, real estate, vehicle, and financial services in the world. Their leadership is based on strong brands or market positions in fee-for-service businesses. Futurestep is working with our client to recruit strong people who have the experience and the capability to be promoted after 18 months. They offer significant career and on-the-job training and experiences for individuals who are the best and the smartest at what they do and who can "hit the ground running". The company is looking for a Vice President of Infrastructure and Operations. The successful candidate will be principally responsible for all IT technology infrastructure and related operations. VP of Infrastructure & Operations will direct and oversee multiple groups including 1) Data Center Operation, 2) Technical Services for All Platforms, 3) Networks, 4) Help Desk, 5) Desktop Services and International coordination. They are a global company with diverse and complex technical environments. The following Experience is a MUST * Minimum of 10 years related experience * Bilingual Fluent English & SPANISH is a MUST * International experience * Demonstrated experience in managing global computing infrastructure and operations (Fortune 500 experience is a plus), including IBM mainframe, Unix and NT platform, voice and data Network, security and information protection * BA or BS in Information Systems or a related area at a minimum. (MBA or equivalent desired.) For consideration please email a Word or Text formatted resume as an attachment. In the Subject Line please type Code: CSM8986, and in the text box let me know a good time/date when I may contact you. Email your resume to Steve Michaels at smichaels@houston.rr.com From RJGARRIS@worthingtonindustries.com Fri Mar 15 11:06:00 2002 Received: from ftp1.wthg.com (ftp1.wthg.com [170.103.142.2]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id LAA22156 for ; Fri, 15 Mar 2002 11:05:59 -0800 Received: from gw.wthg.com (gw.wthg.com [10.200.1.64]) by ftp1.wthg.com (8.9.3 (PHNE_25183)/8.9.3) with SMTP id OAA15794 for ; Fri, 15 Mar 2002 14:05:54 -0500 (EST) Received: from WII_PDO-Message_Server by gw.wthg.com with Novell_GroupWise; Fri, 15 Mar 2002 14:05:53 -0500 Message-Id: X-Mailer: Novell GroupWise Internet Agent 5.5.5.1 Date: Fri, 15 Mar 2002 14:05:33 -0500 From: "Russell Garrison" To: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-7 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by roton.terraluna.org id LAA22156 Subject: [Infrastructures] isconf and package systems Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I am involved with a systems administration project at my company to create a computational infrastructure for our Linux/UNIX systems. We started working on a scripted install of Debian GNU/Linux and after a visit to infrastructures.org we decided to scale up the project to include an infrastructure. I have been reading through the list archives and found some valuable review of the makefiles and state engine concepts, but I am still stumped about packaging. I have seen use of packages on the list and in the isconf code, but my question centers a little more around Debian and APT. APT offers a very nice and organized basis for much of our software space on the linux systems, but we have a problem with the configure scripts. When we add a package APT wants to run configuration options that we would much rather create make targets for so the actions are audited and repeatable. We have come up with five options (with objections) to battle this: -not use apt at all (not desirable ¯ lose a strong point of Debian) -download just the .deb files and extract from there manually (unknown) -improve non-interactive package mode (currently a manual dialog appears) -re-create packages to suit automation (undesired overhead) -re-write apt client to provide further automation (undesired overhead) The objections aren't hard and fast, just negative aspects that need to be considered in light of the benefits. I'm just wondering what is a good infrastructures way of handling packages and if anyone on this list has experience they can offer to help me sort this out. I thank you in advance and in post for the wonderful work this project is doing to improve systems. This philosophy strikes me as the only sane way of running more than one system in production. From jrstear@sandia.gov Fri Mar 15 12:07:34 2002 Received: from mm01snlnto.sandia.gov (mm01snlnto.sandia.gov [132.175.109.20]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id MAA20920 for ; Fri, 15 Mar 2002 12:07:34 -0800 Received: from 132.175.109.1 by mm01snlnto.sandia.gov with ESMTP ( Tumbleweed MMS SMTP Relay (MMS v4.7)); Fri, 15 Mar 2002 13:07:26 -0700 X-Server-Uuid: 95b8ca9b-fe4b-44f7-8977-a6cb2d3025ff Received: from mercy.sandia.gov (mercy.sandia.gov [134.253.242.218]) by sass165.sandia.gov (8.12.1/8.12.1) with ESMTP id g2FK7SCw012291; Fri, 15 Mar 2002 13:07:28 -0700 (MST) Received: from jrstear by mercy.sandia.gov with local (Exim 3.33 #1 ( Debian)) id 16lxz5-0007Ke-00; Fri, 15 Mar 2002 13:07:27 -0700 Date: Fri, 15 Mar 2002 13:07:27 -0700 To: "Russell Garrison" cc: infrastructures@roton.terraluna.org, kelbley@cs.unm.edu Subject: Re: [Infrastructures] isconf and package systems Message-ID: <20020315200727.GB27948@sandia.gov> References: MIME-Version: 1.0 In-Reply-To: User-Agent: Mutt/1.3.24i From: "Jon Stearley" X-Filter-Version: 1.8 (sass165) X-WSS-ID: 108C8BF42018207-01-01 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, Mar 15, 2002 at 02:05:33PM -0500, Russell Garrison wrote: > -not use apt at all (not desirable ? lose a strong point of Debian) never!!! :) some packages don't ask questions if the config files are in place, others do (ie- "you want your old, or new config files?" or sometimes just stupid questions :( ). in practice, if you put your config files in place first and then `/usr/bin/yes "" | apt-get upgrade` and `/usr/bin/yes "" | apt-get install $install_list` everything works (ie- "take all defaults!"). this is a functional equivalent to apt-get's --assume-yes option, which i found didn't quite do the job (can't remember why right now). it feels a little hackey, but it's worked for years on cs.unm.edu's infrastructure, `update`ing all debian hosts nightly (SO satisfying when security updates hit the mirrors). see ftp://ftp.cs.unm.edu/pub/jstear/example_infrastructure.tar.gz for a quite functional debian infrastructure (cfengine/potato/update-debs.pl therein has the apt stuff). i left there 8mo ago and that tarball is even older, but george kelbley will know any updated scoops on the debian infrastructure there. > -improve non-interactive package mode (currently a manual dialog appears) > -re-create packages to suit automation (undesired overhead) > -re-write apt client to provide further automation (undesired overhead) if you do improve the code, please do pass patches to pkg maintainers :) gd'day! -- +--------------------------------------------------------------+ | Jon Stearley (505) 845-7571 (FAX 844-2067) | | Compaq Federal LLC High Performance Solutions | | Sandia National Laboratories Scalable Systems Integration | +--------------------------------------------------------------+ From docx@io.com Fri Mar 15 13:45:42 2002 Received: from hiram.io.com (root@hiram.io.com [199.170.88.27]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA07809 for ; Fri, 15 Mar 2002 13:45:41 -0800 Received: from eris.io.com (eris.io.com [199.170.88.11]) by hiram.io.com (8.11.2/8.11.2) with ESMTP id g2FLjbI07044; Fri, 15 Mar 2002 15:45:37 -0600 Received: from localhost (docx@localhost) by eris.io.com (8.11.6/8.11.6) with ESMTP id g2FLjR623154; Fri, 15 Mar 2002 15:45:27 -0600 X-Authentication-Warning: eris.io.com: docx owned process doing -bs Date: Fri, 15 Mar 2002 15:45:27 -0600 (CST) From: Dylan Northrup To: Russell Garrison cc: infrastructures@roton.terraluna.org Subject: Re: [Infrastructures] isconf and package systems In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: An infinite number of monkeys in the guise of Russell Garrison wrote: [snippage about APT] :=I have been reading through the list archives and found some valuable review of the makefiles and state engine concepts, but I am still stumped about packaging. I have seen use of packages on the list and in the isconf code, but my question centers a little more around Debian and APT. APT offers a very nice and organized basis for much of our software space on the linux systems, but we have a problem with the configure scripts. When we add a package APT wants to run configuration options that we would much rather create make targets for so the actions are audited and repeatable. We have come up with five options (with objections) to battle this: := := -not use apt at all (not desirable ¯ lose a strong point of Debian) From what I hear, apt is a very nice thing. However, depending on your requirements you may need to lose this. If you require any sort of certification of the applications and tools you install on your host you may need to forgo the use of apt. If operational certification is not a hard and fast requirement you can let this slide. := -download just the .deb files and extract from there manually (unknown) If apt can be run in download-only mode (never used apt, but I do have experience with debian packages) you can use this method to grab the files once, put them on a shared volume accessible to all your necessary machines and run dpkg on each host (naturally in an automated fashion). := -improve non-interactive package mode (currently a manual dialog appears) You could try using expect to get around this, but a better option is to not have to deal with the interactive dialogs to beghin with. := -re-create packages to suit automation (undesired overhead) In my previous and former lifetimes I've found that rarely do packages not require a) specific tweaks or b) some form of interactive installation that could or should be automated away. Re-creating the packages with these defaults solved this problem nicely. Obviously if you're updating your software semi-regularly doing the package creation becomes a non-trivial time sink. Whether the time spent here saves time on the back end is generally site dependent. := -re-write apt client to provide further automation (undesired overhead) You could write a patch to apt to improve automation and contribute it back to the Debian team. I do not know if your timeline and resources allow for this. Perhaps a way to provide a script to answer the dialogs that would be read in by dpkg and react appropriately. . . kinda like a specialized version of expect. Hope this helps. -- Dylan Northrup <*> docx@io.com <*> http://www.io.com/~docx/ "Easy to bitch, easy to whine, easy to moan, easy to cry, easy to feel like there ain't nothing in your life. Harder to work, harder to strive, hard to be glad to be alive, but it's really worth it if you give it a try." -- Cowboy Mouth, 'Easy' From abfrankffm@web.de Fri Mar 15 20:01:22 2002 Received: from smtp.web.de (smtp01.web.de [194.45.170.210]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id UAA06061 for ; Fri, 15 Mar 2002 20:01:21 -0800 Received: from dsl-213-023-052-094.arcor-ip.net ([213.23.52.94] helo=web.de) by smtp.web.de with asmtp (WEB.DE(Exim) 4.41 #2) id 16m5Nb-0007L8-00 for infrastructures@roton.terraluna.org; Sat, 16 Mar 2002 05:01:16 +0100 Message-ID: <3C92C37F.2090707@web.de> Date: Sat, 16 Mar 2002 05:01:03 +0100 From: Albrecht Frank Reply-To: infrastructures@TerraLuna.Org Organization: A+B Frank Frankfurt am Main User-Agent: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:0.9.7) Gecko/20020203 X-Accept-Language: de, en, fr MIME-Version: 1.0 To: infrastructures@roton.terraluna.org Subject: Re: [Infrastructures] isconf and package systems References: Content-Type: text/plain; charset=ISO-8859-7; format=flowed Content-Transfer-Encoding: 7bit Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Russell Garrison wrote: > I am involved with a systems administration project at my company to > create a computational infrastructure for our Linux/UNIX systems. > We started working on a scripted install of Debian GNU/Linux and > after a visit to infrastructures.org we decided to scale up the > project to include an infrastructure. > [...] Hello there, perhaps you are searching something that is already written. Look at the FAI-page at http://www.informatik.uni-koeln.de/fai Regards Albrecht From partain@dcs.gla.ac.uk Sat Mar 16 02:56:00 2002 Received: from slicker.dcs.gla.ac.uk (slicker.dcs.gla.ac.uk [130.209.242.51]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id CAA25048 for ; Sat, 16 Mar 2002 02:55:58 -0800 Received: (from partain@localhost) by slicker.dcs.gla.ac.uk (8.8.8+Sun/8.8.8) id KAA14905; Sat, 16 Mar 2002 10:55:52 GMT X-Authentication-Warning: slicker.dcs.gla.ac.uk: partain set sender to partain@dcs.gla.ac.uk using -f To: infrastructures@terraluna.org Cc: partain@dcs.gla.ac.uk References: From: Will Partain Date: 16 Mar 2002 10:55:52 +0000 In-Reply-To: Message-ID: Lines: 54 User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.4 (Common Lisp) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Infrastructures] ARK packaging (was Re: isconf and package systems) Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: "Russell Garrison" writes: > ... I'm just wondering what is a good infrastructures way > of handling packages ... FWIW, the Arusha Project (http://ark.sf.net/) [ARK] way [technically: Sidai team only] of dealing with "packages" is (a) to assume you're going to have to deal with multiple types of packages eventually (e.g.: even if Debian is your normal thing, what are you going to do when asked to install "Cadence LDV version 3.4 or greater"?), and (b) to put a uniform ARK veneer across all packaging types, including commercial we-took-lots-of-bad-drugs-before-inventing-this-packaging-scheme packages. If you get the current ARK code and type 'ark package reveal ALL', you'll see it do "methods" beginning with 'get-source', and proceeding to 'configure', 'compile', 'check', and on to 'install', and beyond. It ain't rocket science to see how this "flow" maps onto building a standard GNU package from source :-) The important thing is to see that the same "flow" can map onto other packaging schemes. For example, let's say you want to install rsync on your HP-UX boxes by using the "depot" on hpux.connect.org.uk... You make an ARK "package" in which 'get-source' does a wget to hpux.connect.org.uk, 'configure', 'compile', etc., are no-ops, and 'install' runs "swinstall -s rsync-whatever.depot '*'", and so on. Similarly, for debian, you'd make many steps in the normal "flow" be no-ops, and some critical one would be 'apt-get rsync'. The point is: 'ark package install rsync' can do *different* things on different hosts, as appropriate. If there are bits of stuff to be run 'round about the crucial 'apt-get rsync', that's easily done. We do not currently have a do-things-the-Debian-way substrate (which would do the heavy lifting for all packages that "inherit" from it), but we would *love* to have such. Should be comparatively easy, once you understand the ARK religion :-) (The very inadequate documentation of Sidai-style package management is at http://ark.sourceforge.net/sidai-pkg-mgmt.html) Hope this is of some use, Will From luke@madstop.com Sat Mar 23 17:18:14 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA06281 for ; Sat, 23 Mar 2002 17:18:10 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g2O1YamP020444; Sat, 23 Mar 2002 19:34:37 -0600 (CST) Date: Sat, 23 Mar 2002 19:34:36 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org cc: Tim Quinlan , Geoffrey Parsons Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] Web site is up Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Well, I finally built a website running slashcode. For those who remember the thread I started a couple months ago, I finally got the site built, and I have posted my email there as the first article. I have no idea where it's going yet, or if slashcode is going to work for what I have in mind, but if anyone is interested in participating, the url is http://madstop.com. If anyone would like to help me with the site, please email me and I will set you up with administrative capabilities. As I mentioned, I have started a thread on the site about what the site is for, so if you have any comments on that fact, please take it there. BTW, at this point anonymous cowards are allowed to post, but I don't think I'm going to leave it that way for very long. -- Get with the program, jeffrey. No one is "wrong" on Usenet. They are either 100% totally correct, or they are "a lying, scum sucking weasel." There is no in between. -- Garrett Johnson, in talk.politics.misc From luke@madstop.com Mon Apr 1 19:53:11 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id TAA06674 for ; Mon, 1 Apr 2002 19:53:06 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g323quU28352 for ; Mon, 1 Apr 2002 21:52:57 -0600 (CST) Date: Mon, 1 Apr 2002 21:52:56 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] public, shared CVS repository Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hello all, (I'm sending this to the infrastructures list, but it's directly related to my madstop.com website; I haven't gotten any feedback on madstop.com yet, but I've gotten good feedback here. I'll probably double-post until I start to see enough traffic on madstop.com to make the switch worth it.) I'm trying to come up with a method of creating a public, shared, and hopefully secure CVS repository. Here is what I want: Web-based configuration (because this is for a web site) The ability for a single person to do CVS checkins and checkouts using a CLI, e.g., cvs over ssh The ability for a person to give someone else the ability to do CVS checkins The ability for everyone to do cvs checkouts I think I could do this with normal user accounts, but it would be a PITA, and I don't really like the idea of giving everyone a normal user account. I think I can set up CVS users pretty securely, especially for anon, but I'm uncomfortable with having a direct linkage between website users and CVS users. Is that the only way to do it? I've thought about having some sort of wrapper which was run on initial login, instead of the cvs binary; it could do things like verify that the user has the right to do the checkins s/he is trying to do. However, that would require actually modifying the cvs binary itself, since things like the user and the repository are communicated after cvs is run--when a user logs in to use cvs, the exact command they run is 'cvs server', with no other arguments. If I have to, I'll probably have a direct correlation between web site users who have a CVS repository and local users, but I'd like to avoid it. From the other direction, if I do end up having to do this, I'm going to have to modify slashcode to create local users with very specific characteristics (i.e., they can only run a single command, their passwords always have to be in sync with MySQL, they probably need the ability to set up ssh authorized_keys files, that kind of thing). I'm also thinking about attempting to chroot all of the users; this might give me what I want, in that the users would not be literal local users, but in order for it to work I would have to (1) write a SUID wrapper that su's to the correct user after the chroot, (2) somehow authenticate a crapload of unique users to one local user, then chroot/su to the correct unique website user, and (3) maintain all chrooted permissions in the CVS repositories. None of those sound very fun, and considering that I hope to allow both password and SSH key authentication, I don't really think I'll be able to rely just on chrooting. I probably would chroot the users either way, but I think I'm going to have to have a real local user, do password/SSH key authentication, then chroot to the same user but in a more protected space. I'm giving a talk at Rubi-con this weekend (anyone going?), but what time I won't be devoting to that, I'll be using to try to solve this problem. Once I've got the technology figured out, I still have to integrate it all with slashcode, which I think is going to suck quite a bit. If anyone has any great ideas or would like to help, I'd love to hear it. -- "True Terror is to wake up one morning and discover that your high school class is running the country." -- Kurt Vonnegut From luke@madstop.com Mon Apr 1 20:25:10 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id UAA22634 for ; Mon, 1 Apr 2002 20:25:09 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g324P8R28500 for ; Mon, 1 Apr 2002 22:25:08 -0600 (CST) Date: Mon, 1 Apr 2002 22:25:08 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] Re: public, shared CVS repository Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Here are a couple of features that I want for my public CVS repository: (Because of my laziness in differentiating between repositories, cvsroots, and modules, I use them relatively interchangeably here.) Each module or repository must have an owner; I'd prefer to allow multiple people marked as owner, but that won't work according to unix file permissions. An owner can give write access to the repository to others, essentially creating a group of people who have the ability to modify a repository. Preferably, you would want owners to be able to create groups, and give those groups permission to modify files, groups of files, or trees of files; anything less would encourage people to give no access, because they would usually end up giving too much access. Anyone with write permission to a file or repository must be able to do web-based CVS checkouts, at least such that s/he can view the file and its history, along with CLI/cvs-server-over-ssh checkouts. I consider cvs-over-ssh sufficient, because if you can get CVS, you can probably get ssh. This may turn out not to be the case once we get into it, but I think it's an acceptable starting point. Anyone at all must be able to do CLI- or web-based checkouts. Anyone at all must be able to submit patches to the owner of the repository, preferably using a web page, I suppose. Owners should be able to accept or reject those patches; it would be kind of nifty if the CVS system would let the owner auto-apply patches, do a checkout, and see if it works, but I suppose it's about as reasonable to expect the owner to do a manual checkout, manual patch application, and then a test. All of this should be relatively integrated into the web page--you go to a user's page, and you can see his/her CVS repository; you do a search for something on the site, and CVS entries show up; some sort of section which allows you to browse tools; repository owners having the ability to file a file, directory, or tree under a specific category (probably based on directory, since CVS seems to be very fond of directories and not so happy about files); and probably many more. At some point, there needs to be some kind of person-independent repository, containing things like RFC-equivalent. Here, people can check out an RFC, make their own modification, and then submit the patch, and leave it up to the community (using that rating system I have yet to invent) as to whether the patch gets accepted. Or, they can fork it, and if it gets popular enough, submit it as an RFC.something, or something like that. Unless we get some consistent, simple method of maintaining version across code and white-papers this tool will never really be of use. What can we do, and how should we do it? (This comment available at http://madstop.com/comments.pl?sid=5&op=&threshold=0&commentsort=0&mode=nested&cid=6). -- "Anyone who considers arithmatical methods of producing random digits is, of course, in a state of sin." -- John Von Neumann From mfw@wyle.org Mon Apr 1 21:08:14 2002 Received: from wyle.org (m206-106.dsl.tsoft.com [198.144.206.106]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA13298 for ; Mon, 1 Apr 2002 21:08:12 -0800 Received: from wyle.org (monch.wyle.org [127.0.0.1]) by wyle.org (8.12.1/8.12.1) with ESMTP id g3258BRk030600; Mon, 1 Apr 2002 21:08:11 -0800 Received: (from mfw@localhost) by wyle.org (8.12.1/8.12.0.Beta12) id g3258BET030599; Mon, 1 Apr 2002 21:08:11 -0800 Date: Mon, 1 Apr 2002 21:08:11 -0800 From: Mitch Wyle To: "Luke A. Kanies" , infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: public, shared CVS repository Message-ID: <20020401210811.A30508@wyle.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi Luke, I think you can implement the policy requirements at which you hint by the technical requirements you state without resorting to an implementation of arbitrary access control lists (ACLs) -- which are different from Unix file / group permissions. There are many ways to implement ACLs if your technical requirements really do map isomorphically to the policies they imply. Here is one idea: Establish a (heaven forefend!) shared "role" Unix account to which owners by policy, not technical restriction, "su." The owners decide among themselves who will "write" to the account under what circumstances. Again the policies are set manually according to the module ownership team. This account adds one more level of accountability since no one ever logs in to the account and their RLOG environment variable is always set to themselves. Module owners who do not trust their teammates can simply not give out the role account's password and will prefer to apply the patches submitted by the other team members and also to perform check-ins themselves. When they are on vacation or other leave, they change the role account password, give it to the stand-in, then change it back when they return. Teams whose module owners ignore patches complain to Luke or "The Higher Authority[sm]." Luke then settles the disputes or allows a module to branch. CVS over ssh works fine on windoze (cygwin or native), macOS-X and Unix, so yes, go with CVS/ssh. > Anyone at all must be able to submit patches to the owner of the > repository, preferably using a web page, I suppose. Web pages are crap for this purpose; email is better. BTW: The cvsweb stuff with all those groovy colors helps more than most "merge" tools to track changes before or after an update. CVS is awesome. http://stud.fh-heilbronn.de/~zeller/cgi/cvsweb.cgi/ It's like... gnarly, man. Your "person-independent respository" idea will not work. You need ownership. Otherwise things will languish or get perverted. There can exist no rating system that prevents concerted electioneering attack by a minority. > Unless we get some consistent, simple method of maintaining version > across code and white-papers this tool will never really be of use. > What can we do, and how should we do it? What is the problem with using sourceforge? Who authors the whitepapers? Who reviews them? Can the tech writers use the (shudder) office-2k webdav stuff with mod_dav? or (kill me now) the ms-sharepoint portal? What is the application for these tools? Best, -Mitch On Monday, April 01, 2002 at 20:04 "Luke A. Kanies" wisely wrote: > Here are a couple of features that I want for my public CVS repository: > > (Because of my laziness in differentiating between repositories, cvsroots, > and modules, I use them relatively interchangeably here.) > > Each module or repository must have an owner; I'd prefer to allow multiple > people marked as owner, but that won't work according to unix file > permissions. > > An owner can give write access to the repository to others, essentially > creating a group of people who have the ability to modify a repository. > > Preferably, you would want owners to be able to create groups, and give > those groups permission to modify files, groups of files, or trees of > files; anything less would encourage people to give no access, because > they would usually end up giving too much access. > > Anyone with write permission to a file or repository must be able to do > web-based CVS checkouts, at least such that s/he can view the file and its > history, along with CLI/cvs-server-over-ssh checkouts. I consider > cvs-over-ssh sufficient, because if you can get CVS, you can probably get > ssh. This may turn out not to be the case once we get into it, but I > think it's an acceptable starting point. > > Anyone at all must be able to do CLI- or web-based checkouts. > > Anyone at all must be able to submit patches to the owner of the > repository, preferably using a web page, I suppose. > > Owners should be able to accept or reject those patches; it would be kind > of nifty if the CVS system would let the owner auto-apply patches, do a > checkout, and see if it works, but I suppose it's about as reasonable to > expect the owner to do a manual checkout, manual patch application, and > then a test. > > All of this should be relatively integrated into the web page--you go to a > user's page, and you can see his/her CVS repository; you do a search for > something on the site, and CVS entries show up; some sort of section which > allows you to browse tools; repository owners having the ability to file a > file, directory, or tree under a specific category (probably based on > directory, since CVS seems to be very fond of directories and not so happy > about files); and probably many more. > > At some point, there needs to be some kind of person-independent > repository, containing things like RFC-equivalent. Here, people can check > out an RFC, make their own modification, and then submit the patch, and > leave it up to the community (using that rating system I have yet to > invent) as to whether the patch gets accepted. Or, they can fork it, and > if it gets popular enough, submit it as an RFC.something, or something > like that. > > Unless we get some consistent, simple method of maintaining version across > code and white-papers this tool will never really be of use. What can we > do, and how should we do it? > > (This comment available at > http://madstop.com/comments.pl?sid=5&op=&threshold=0&commentsort=0&mode=nested&cid=6). > > -- > "Anyone who considers arithmatical methods of producing random digits > is, of course, in a state of sin." -- John Von Neumann > > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures From Tarjei.Jensen@kvaerner.com Tue Apr 2 22:28:44 2002 Received: from kcpgb-mime02.kvaerner.com (kcpgb-mime02.kvaerner.com [193.132.79.9]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id WAA23410 for ; Tue, 2 Apr 2002 22:28:43 -0800 From: Tarjei.Jensen@kvaerner.com Received: from kcpgb-bb02.messaging.kvaerner.com (unverified) by kcpgb-mime02.kvaerner.com (Content Technologies SMTPRS 4.2.5) with ESMTP id for ; Wed, 3 Apr 2002 07:24:56 +0100 Received: by kcpgb-bb02.messaging.kvaerner.com with Internet Mail Service (5.5.2653.19) id ; Wed, 3 Apr 2002 07:28:04 +0100 Message-ID: To: infrastructures@roton.terraluna.org Date: Wed, 3 Apr 2002 07:28:10 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Infrastructures] public, shared CVS repository Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: CVS might be awesome, but there is a problem with it: It does not use http. There is no guarantee that the company firewall have socks available. Which means that there might be a large class of users which cannot use your nice and shiny CVS enabled site. greetings, From cwcjr@bestweb.net Thu Apr 11 10:32:44 2002 Received: from pimout4-int.prodigy.net (pimout4-ext.prodigy.net [207.115.63.103]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA11289 for ; Thu, 11 Apr 2002 10:32:42 -0700 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout4-int.prodigy.net (8.11.0/8.11.0) with SMTP id g3BHWL159940; Thu, 11 Apr 2002 13:32:26 -0400 Message-ID: <009601c1e17e$d935dfc0$c5c0a8c0@prodigy.com> From: "Will Cox" To: "Luke A. Kanies" , References: Date: Thu, 11 Apr 2002 13:32:21 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 Subject: [Infrastructures] madstop.com, or sysadmin knowledge management Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Luke, et al., On Monday, I ran across a site[1] because of a link from Jon Udell's site[2]. I finally got around to reading it today. Oddly enough, this site, "s l a m," intersects with your intent with madstop.com[3]. There's quite a significant community, covering many subjects, building around Userland's Radio Userland[4] application, which is basically stitching together syndication, aggregation, and writing, to make them all straightforward. /cwc [1] http://radio.weblogs.com/0104487/ [2] http://radio.weblogs.com/0100887/ [3] http://www.madstop.com/ [4] http://radio.userland.com/ -- From jcarro10@sprintspectrum.com Fri Apr 12 13:30:21 2002 Received: from smtpgw6.sprintspectrum.com (smtpgw6.sprintspectrum.com [207.40.188.14]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA08520 for ; Fri, 12 Apr 2002 13:30:20 -0700 Received: from pkcex004.sprintspectrum.com (pkcex004.sprintspectrum.com [208.10.75.139]) by smtpgw6.sprintspectrum.com (8.11.2/8.11.3) with ESMTP id g3CKTv503612 for ; Fri, 12 Apr 2002 15:29:57 -0500 (CDT) Received: by pkcex004.sprintspectrum.com with Internet Mail Service (5.5.2654.89) id ; Fri, 12 Apr 2002 15:29:57 -0500 Message-ID: From: "Carroll, Jim" To: "Infrastructures (E-mail)" Date: Fri, 12 Apr 2002 15:29:56 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2654.89) Content-Type: text/plain; charset="iso-8859-1" Subject: [Infrastructures] IS monitoring Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: After reviewing the Infrastructures web site, in particular the monitoring aspect, these questions: 1. It looks like the recommended tool, NetSaint, focuses on the "something's broken, spring into action" type of monitoring. But is there a recommended tool for the trending of system and network performance? I'm familiar with MRTG, but a) this focuses only on the network (unless you also get MRTG-PME), and b) it looks like it might be superceded by the likes of RRD Tool in combination with something else. Of course, there's always c) that magic something I've completely overlooked. My question is, does anyone have any recommendations with regards to infrastructure performance monitoring/trending (and not just the network side of things)? 2. The environment I recently joined already has Big Brother in place. Problem is, it's a bit like a yappy, high-strung dog, overreacting to the slightest blip on the radar screen. The fact that our BB host is on one side of a WAN connection which occasionally has connectivity dropouts, and the hosts being monitored are on the other side of that WAN connection, doesn't help. (It was done this way, because the monitoring host has access to the paging system.) A couple of us here are already impressed with the promise that NetSaint looks to offer, but another admin (the one who installed BB) seems keen on BB, as well as pointing out that NetSaint is only version 0.0.7. I'd like to get some input to help sway us in one direction or the other. If nothing else, I just want to minimize expending any energy on something which will create extra work for us down the road (over and above another, preferred solution). Suggestions/comments? I feel I have another question to ask, but it hasn't quite distilled itself. If it comes, I'll ask it. I realize these are the classic, "What is the best open source tool to do XYZ?" questions, but I feel like I'm starting to spin my wheels. TIA, Jim Carroll From mfw@wyle.org Fri Apr 12 14:46:55 2002 Received: from wyle.org (m206-106.dsl.tsoft.com [198.144.206.106]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA16340 for ; Fri, 12 Apr 2002 14:46:55 -0700 Received: from wyle.org (monch.wyle.org [127.0.0.1]) by wyle.org (8.12.1/8.12.1) with ESMTP id g3CLl2Rk001748 for ; Fri, 12 Apr 2002 14:47:02 -0700 Received: (from mfw@localhost) by wyle.org (8.12.1/8.12.0.Beta12) id g3CLl1xi001747 for infrastructures@terraluna.org; Fri, 12 Apr 2002 14:47:01 -0700 Date: Fri, 12 Apr 2002 14:47:01 -0700 From: Mitch Wyle To: "Infrastructures (E-mail)" Subject: Re: [Infrastructures] IS monitoring Message-ID: <20020412144701.B1619@wyle.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Jim, It is true the netsaint "focusses" on alerting you that something has exceeded the threshold of wrongness. However it is also the case that netsaint can successfully be (ab)used for trending and reporting on system performance. Netsaint does keep stats and logs and delivers web reports with detailed history. Netsaint has a flexible "plug in" architecture that lets you write your own modules for the performance measurement about which you care. Netsaint will do the trending and web-reporting for you. It is also likely that someone else has written the netsaint module for the infrastructure element you want to trend. On Friday, April 12, 2002 at 13:04 "Carroll, Jim" wisely wrote: > After reviewing the Infrastructures web site, in particular the monitoring > aspect, these questions: > > 1. It looks like the recommended tool, NetSaint, focuses on the "something's > broken, spring into action" type of monitoring. But is there a recommended > tool for the trending of system and network performance? I'm familiar with > MRTG, but a) this focuses only on the network (unless you also get > MRTG-PME), and b) it looks like it might be superceded by the likes of RRD > Tool in combination with something else. Of course, there's always c) that > magic something I've completely overlooked. My question is, does anyone > have any recommendations with regards to infrastructure performance > monitoring/trending (and not just the network side of things)? > > 2. The environment I recently joined already has Big Brother in place. > Problem is, it's a bit like a yappy, high-strung dog, overreacting to the > slightest blip on the radar screen. The fact that our BB host is on one > side of a WAN connection which occasionally has connectivity dropouts, and > the hosts being monitored are on the other side of that WAN connection, > doesn't help. (It was done this way, because the monitoring host has access > to the paging system.) A couple of us here are already impressed with the > promise that NetSaint looks to offer, but another admin (the one who > installed BB) seems keen on BB, as well as pointing out that NetSaint is > only version 0.0.7. I'd like to get some input to help sway us in one > direction or the other. If nothing else, I just want to minimize expending > any energy on something which will create extra work for us down the road > (over and above another, preferred solution). Suggestions/comments? > > I feel I have another question to ask, but it hasn't quite distilled itself. > If it comes, I'll ask it. > > I realize these are the classic, "What is the best open source tool to do > XYZ?" questions, but I feel like I'm starting to spin my wheels. > > TIA, > > Jim Carroll > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures From Brian.Dunbar@Plexus.com Fri Apr 12 14:51:46 2002 Received: from aspen.plexus.com (aspen.plexus.com [64.215.193.225]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA18751 for ; Fri, 12 Apr 2002 14:51:45 -0700 Received: from exchange1.admin.plx.plexus.com (exchange1.admin.plx.plexus.com [206.209.243.203]) by aspen.plexus.com (8.11.3nb1/8.11.3) with ESMTP id g3CLpod07596; Fri, 12 Apr 2002 16:51:50 -0500 (CDT) Received: by exchange1.admin.plx.plexus.com with Internet Mail Service (5.5.2650.21) id <2F1G981W>; Fri, 12 Apr 2002 16:51:50 -0500 Message-ID: <93EF56FF0BD1234E8D1C74B9813E0EA02886C3@neen-mail-003.na.plexus.com> From: Brian Dunbar To: "Carroll, Jim" , "Infrastructures (E-mail)" Subject: RE: [Infrastructures] IS monitoring Date: Fri, 12 Apr 2002 16:51:49 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: We're using BB here, no problems with false alerts. You might talk the BB admin into mondifying some threseholds, it it's too yappy. brian -----Original Message----- From: Carroll, Jim [mailto:jcarro10@sprintspectrum.com] Sent: Friday, April 12, 2002 3:30 PM To: Infrastructures (E-mail) Subject: [Infrastructures] IS monitoring After reviewing the Infrastructures web site, in particular the monitoring aspect, these questions: 1. It looks like the recommended tool, NetSaint, focuses on the "something's broken, spring into action" type of monitoring. But is there a recommended tool for the trending of system and network performance? I'm familiar with MRTG, but a) this focuses only on the network (unless you also get MRTG-PME), and b) it looks like it might be superceded by the likes of RRD Tool in combination with something else. Of course, there's always c) that magic something I've completely overlooked. My question is, does anyone have any recommendations with regards to infrastructure performance monitoring/trending (and not just the network side of things)? 2. The environment I recently joined already has Big Brother in place. Problem is, it's a bit like a yappy, high-strung dog, overreacting to the slightest blip on the radar screen. The fact that our BB host is on one side of a WAN connection which occasionally has connectivity dropouts, and the hosts being monitored are on the other side of that WAN connection, doesn't help. (It was done this way, because the monitoring host has access to the paging system.) A couple of us here are already impressed with the promise that NetSaint looks to offer, but another admin (the one who installed BB) seems keen on BB, as well as pointing out that NetSaint is only version 0.0.7. I'd like to get some input to help sway us in one direction or the other. If nothing else, I just want to minimize expending any energy on something which will create extra work for us down the road (over and above another, preferred solution). Suggestions/comments? I feel I have another question to ask, but it hasn't quite distilled itself. If it comes, I'll ask it. I realize these are the classic, "What is the best open source tool to do XYZ?" questions, but I feel like I'm starting to spin my wheels. TIA, Jim Carroll _______________________________________________ Infrastructures mailing list Infrastructures@mailman.terraluna.org http://mailman.terraluna.org/mailman/listinfo/infrastructures From oetiker@ee.ethz.ch Fri Apr 12 14:57:28 2002 Received: from tardis.ee.ethz.ch (tardis-delek-fast.ee.ethz.ch [129.132.2.199]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA21694 for ; Fri, 12 Apr 2002 14:57:27 -0700 Received: from nova.ee.ethz.ch (nova.ee.ethz.ch [129.132.2.203]) by tardis.ee.ethz.ch (Postfix) with ESMTP id EA7833C58E; Fri, 12 Apr 2002 23:57:32 +0200 (MET DST) Received: by nova.ee.ethz.ch (Postfix, from userid 510) id 4D24120D9B; Fri, 12 Apr 2002 23:57:32 +0200 (MEST) Received: from localhost (localhost [127.0.0.1]) by nova.ee.ethz.ch (Postfix) with ESMTP id 38B5D18A7A; Fri, 12 Apr 2002 23:57:32 +0200 (MEST) Date: Fri, 12 Apr 2002 23:57:31 +0200 (MEST) From: Tobias Oetiker To: "Carroll, Jim" Cc: "Infrastructures (E-mail)" Subject: Re: [Infrastructures] IS monitoring In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Today Carroll, Jim wrote: > After reviewing the Infrastructures web site, in particular the monitoring > aspect, these questions: > > 1. It looks like the recommended tool, NetSaint, focuses on the "something's > broken, spring into action" type of monitoring. But is there a recommended > tool for the trending of system and network performance? I'm familiar with > MRTG, but a) this focuses only on the network (unless you also get > MRTG-PME), and b) it looks like it might be superceded by the likes of RRD > Tool in combination with something else. Of course, there's always c) that > magic something I've completely overlooked. My question is, does anyone > have any recommendations with regards to infrastructure performance > monitoring/trending (and not just the network side of things)? > > 2. The environment I recently joined already has Big Brother in place. > Problem is, it's a bit like a yappy, high-strung dog, overreacting to the > slightest blip on the radar screen. The fact that our BB host is on one > side of a WAN connection which occasionally has connectivity dropouts, and > the hosts being monitored are on the other side of that WAN connection, > doesn't help. (It was done this way, because the monitoring host has access > to the paging system.) A couple of us here are already impressed with the > promise that NetSaint looks to offer, but another admin (the one who > installed BB) seems keen on BB, as well as pointing out that NetSaint is > only version 0.0.7. I'd like to get some input to help sway us in one > direction or the other. If nothing else, I just want to minimize expending > any energy on something which will create extra work for us down the road > (over and above another, preferred solution). Suggestions/comments? Hi Jim, We recently created gossibps. the idea there is that normally everything works fine, and our problem is that we get tired of looking at a working system and tend to be not looking in the rare instances where things do not work right ... go to http://isg.ee.ethz.ch/tool cheers tobi -- ______ __ _ /_ __/_ / / (_) Oetiker, ETZ J97, ETH, 8092 Zurich, Switzerland / // _ \/ _ \/ / phoneto:+41(0)1-632-5286 faxto:+41(0)1-632-1517 /_/ \.__/_.__/_/ oetiker@ee.ethz.ch http://google.com/search?q=tobi From luke@madstop.com Sun Apr 14 14:43:13 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA24038 for ; Sun, 14 Apr 2002 14:43:05 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g3ELh4v08882; Sun, 14 Apr 2002 16:43:04 -0500 (CDT) Date: Sun, 14 Apr 2002 16:43:04 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: Will Cox cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] madstop.com, or sysadmin knowledge management In-Reply-To: <009601c1e17e$d935dfc0$c5c0a8c0@prodigy.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Thu, 11 Apr 2002, Will Cox wrote: > Luke, et al., > > On Monday, I ran across a site[1] because of a link from Jon Udell's > site[2]. I finally got around to reading it today. Oddly enough, this site, > "s l a m," intersects with your intent with madstop.com[3]. There's quite a > significant community, covering many subjects, building around Userland's > Radio Userland[4] application, which is basically stitching together > syndication, aggregation, and writing, to make them all straightforward. I'm still checking this out, but I'd have to say that I'm pretty hesitant to use anything that doesn't run pretty much everywhere; from what I can tell from the Radio Userland site, the software doesn't run on any Unix except OS X, which pretty much leaves out my entire target audience. I'll look more into the site, though, and contact the guy who runs it, because there does appear to be overlap. I'm not nearly up-to-date enough on modern web technologies, and it's really starting to hit me how much that impacts my ability to make the modern, useful site I want. I have yet to delve into weblogs, but there seems to be a significant amount of important work going on there. Anyway, thanks for the link. Luke -- Tobacco stocks have taken a big tumble," says Jay Leno. "Phillip Morris fell 6 points. They lost so much money they may have to lay off two senators." From luke@madstop.com Sun Apr 14 18:21:56 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id SAA08508 for ; Sun, 14 Apr 2002 18:21:51 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g3F1EJO10099; Sun, 14 Apr 2002 20:14:19 -0500 (CDT) Date: Sun, 14 Apr 2002 20:14:19 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: Mitch Wyle cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: public, shared CVS repository In-Reply-To: <20020401210811.A30508@wyle.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi Mitch, Sorry for the late reply. I always espouse laziness as a virtue, but apparently I've been practicing just a little bit much of it. On Mon, 1 Apr 2002, Mitch Wyle wrote: > Here is one idea: > > Establish a (heaven forefend!) shared "role" Unix account to which > owners by policy, not technical restriction, "su." The owners decide > among themselves who will "write" to the account under what > circumstances. Again the policies are set manually according to the > module ownership team. This account adds one more level of > accountability since no one ever logs in to the account and their RLOG > environment variable is always set to themselves. > > Module owners who do not trust their teammates can simply not give out > the role account's password and will prefer to apply the patches > submitted by the other team members and also to perform check-ins > themselves. When they are on vacation or other leave, they change the > role account password, give it to the stand-in, then change it back when > they return. The problem I have with this basic model is that fundamentally what most people will be doing is adding their own code in small snippets to their repositories, at least at first. In other words, a person may have 15 or 20 different scripts, and 2 or 3 white paperas, and most of them aren't really going to modified by anyone else or be considered part of a larger module. I basically plan on each person getting the equivalent of a single module (so you would have /cvsroot/~user or something like that), and they can determine how things are organized under there. In other words, there won't be a direct map between modules and rights. For instance, image the following module path: /cvsroot/luke/homeconfig/ /cvsroot/luke/perlmods/ /cvsroot/luke/certmgr/ /cvsroot/luke/bin/ /cvsroot/luke/papers/ I have my normal .profile-like config files in homeconfig, I have my independent but not CPAN-quality perl modules in perlmods, I have a specific certificate manager app in certmgr, I have about 100 different scripts in multiple languages in bin, and I have my various white papers in papers. In this case, I probably wouldn't want to give anyone the ability to modify homeconfig, but I definitely would want to give people the ability to modify certmgr as part of a dev team. And under bin, some scripts I would like to be able to add people to, and some I wouldn't. Yes, the burden could be on me as the user to make sure that everything is organized in such a way that rights are easy to delegate, but the reality is that if the web site doesn't make rights easy to use and delegate, then most people won't use them. I've been thinking more seriously about wrapping the cvs binary itself, because that's one of the few ways I can think of to get what I want. If I allowed key-based logins only (which I think is reasonable), then I could basically just do a quick check on permissions before I passed everything off to the actual CVS binary. I'm going to have to look more into how CVS works, but... Yes, I'm rambling, and I don't really know what I'm talking about, but I'm trying. I'm currently trying to get the Wiki plugin to work with SlashCode, also. > Teams whose module owners ignore patches complain to Luke or "The Higher > Authority[sm]." Luke then settles the disputes or allows a module to > branch. I don't really like this model; I think that branches and forks are bound to happen, but it seems to not be as large a problem as one would think. One of the reasons I think it is so important to be able to rate the content is so that when we do have forks, they can be rated relative to each other and thus try to pick the "best" version. > CVS over ssh works fine on windoze (cygwin or native), macOS-X and Unix, > so yes, go with CVS/ssh. Good; I've only used it over Unixes. > > Anyone at all must be able to submit patches to the owner of the > > repository, preferably using a web page, I suppose. > > Web pages are crap for this purpose; email is better. They are crap, but I think it's kind of important for the facility to at least be there. At the very least it could be implemented as a "send a message to the author about this file" and the patch could be the message. As much as I think the web is currently a limited communication medium, I do believe that it functions effectively as a kind of least-common-denominator. > BTW: The cvsweb stuff with all those groovy colors helps more than > most "merge" tools to track changes before or after an update. CVS is > awesome. > > http://stud.fh-heilbronn.de/~zeller/cgi/cvsweb.cgi/ > > It's like... gnarly, man. So, when you say colors, what do you mean? Do you mean syntax highlighting? I don't get it; when I look at the code there, all I see is normal text. The python 'viewcvs.cgi' claims the ability to do something or other with colors, too, but the only place I see colors is in the title bars and such. > Your "person-independent respository" idea will not work. You need > ownership. Otherwise things will languish or get perverted. There can > exist no rating system that prevents concerted electioneering attack by > a minority. I guess a better way to say it is that there would have to be a repository which could be considered to be of the site, not of a person, even if a single person controls some aspects of that repository. Yes, you probably would always want someone to be checking the work, but you don't want that person to have "ownership" of the repository, or even pieces of it. > > Unless we get some consistent, simple method of maintaining version > > across code and white-papers this tool will never really be of use. > > What can we do, and how should we do it? > > What is the problem with using sourceforge? I hadn't really thought about it, because it seemed like it was more of a storage than a communications medium. I've looked around at it a bit, and I can't seem to find a way to access files directly; I also don't know what the developer interface looks like. I really want something that looks mostly the same to developers and users, except that developers have write access. I like the way that viewcvs and cvsweb work; I'd like my site to look like that but wrapped for branding and nav menus. > Who authors the whitepapers? Who reviews them? Can the tech writers > use the (shudder) office-2k webdav stuff with mod_dav? or (kill me now) > the ms-sharepoint portal? Me, you, and any other sucker we can con into thinking this site will help everyone. As to webdav, I'm not really familiar with it. Someone at Rubi-con mentioned it (BTW, my talk is linked off of madstop.com, if anyone is interested--it's very bare bones, because I actually presented it), but I haven't had a chance to look into it. Do you fear all WebDAV, or just M$'s? > What is the application for these tools? This isn't about specific applications, it's about overall providing a collaboration locus for sysadmins. What do you do as a sysadmin? Then I want there to be tools to address that. If you already have the tools and don't need this site, then I would love you to contribute your expertise and code. That's my basic model. The rich choose to share with everyone, including the poor, in hopes that the other rich will sufficiently enhance everyone's tool base that it will be worth it. By rich, of course, I mean information-, tools-, and experience-rich. From jcarro10@sprintspectrum.com Tue Apr 16 16:28:25 2002 Received: from smtpgw6.sprintspectrum.com (smtpgw6.sprintspectrum.com [207.40.188.14]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id QAA10585 for ; Tue, 16 Apr 2002 16:28:24 -0700 Received: from pkcex004.sprintspectrum.com (pkcex004.sprintspectrum.com [208.10.75.139]) by smtpgw6.sprintspectrum.com (8.11.2/8.11.3) with ESMTP id g3GNS3518343 for ; Tue, 16 Apr 2002 18:28:04 -0500 (CDT) Received: by pkcex004.sprintspectrum.com with Internet Mail Service (5.5.2654.89) id <27HGJJMB>; Tue, 16 Apr 2002 18:28:04 -0500 Message-ID: From: "Carroll, Jim" To: "Infrastructures (E-mail)" Date: Tue, 16 Apr 2002 18:28:01 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2654.89) Content-Type: text/plain; charset="iso-8859-1" Subject: [Infrastructures] authentication & client file access Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: In trying to roll out a 'proper' infrastructure architecture, I'm running into the odd snag which usually gets me pondering about what constitutes the best approach. Environment: Solaris8 clients and servers, new Linux clients, possibly Linux on laptops. Need to work with/around entrenched Windows environment and a handful of user-owned Linux systems. I'm looking for input on the following: 1. Authentication servers I know the Infrastructures web site espouses the use of NIS over all other options. After reading about the security advantages of NIS+ over NIS, as well as being urged to implement NIS+, I've set up a master and replica NIS+ server. I had to get the NIS+ source code for Linux (as well as the appropriate PAM module) and get that working. So far, everything (including automounter) seems to be working well, with the possible question surrounding why changing one's password on a Linux client seems to fail to update the credentials. (Sometimes it works, usually it doesn't. Works fine on the Solaris side.) If anyone could give me any warm fuzzies with regards to NIS not being the security concern that some make it out to be, let me know. Otherwise, it looks like I'll have to make this work with NIS+. Having said that, it was brought to my attention today that some users may eventually be running Linux on a laptop. Naturally this introduces the problem of being able to authenticate while disconnected from the network. I recall that the Infrastructures site suggests making every client a slave (with the added overhead associated with pushing the maps, and BTW, what happens if you try to push those maps when half the machines (laptops) are disconnected for the night?), but I'm not particularly smitten with that approach. The workaround was suggested to create a local entry in /etc/passwd for that user's laptop, eg, Joe would have an entry for joe01, Mary would have an entry mary01, etc. This is to permit the user to login and work while disconnected from the network. I pointed out that it introduces the problem we were trying to solve by using NIS/NIS+, ie, managing separate logins and UIDs. I even suggested having a master passwd/shadow file which incorporated all of the 'standalone' accounts, but was told that there was no real need for that, that each machine could be administered on an as-needed basis. Again, this doesn't fill me with warm fuzzies; I see it scaling alright... scaling for more/new problems. So here I am pondering/reconsidering NIS, as well as wondering how best to deal with disconnected operations. 2. Client File Access As mentioned above, I intend to roll out automounter for home directories, at the very least. First problem is, how to deal with the need for disconnected computing? Assuming the problem of user authentication has been dealt with, it would be great if the user automounted their home directory. Great for enabling a user to login to any workstation, having their desktop available, anywhere. But disconnect that laptop, and... limbo. Another problem is that there are already a handful of Linux workstations on the network, set up by a few developers who are Linux-savvy and are rather fond of having root. The Linux systems we're pushing out won't grant that privilege (by design). However, we have a concern. Presently the users' home directories are shared from one server, read-only. In order for the automounted home directory to work well with Gnome, it's desirable to make that share read-write. Problem is, anyone who already has root on a Linux box could conceivably compromise anyone's home directory, simply by creating a local account and su'ing to the user in question. We've toyed with the idea of restricting access based on IP address, but also have to deal with the problem that the subnet in question is served by DHCP (Windows), a process we don't own or control. Granted, we could set the workstation up with a static IP address, but then that prevents the developer from being as portable across subnets as their manager envisions them as being (something which is currently not an issue for the DHCP-served Windows clients). So we're back to restricting NFS client access based on subnet, which of course opens us up to a root-enabled user doing as he pleases on any read-write share. Solaris supports Secure NFS; Linux does not (appear to). I have a cursory awareness of CodaFS. Great for Linux, but I'm not sure how stable it is on the Solaris side. I can't help but wonder if I'm simply looking at the wrong approach/tools to solve the problem. Perhaps something like rsync for keeping home directories up to date...? Maybe NIS slaves on every workstation *is* the answer...? Maybe CodaFS is the answer for home directories, providing we set those users up with a Linux-based CodaFS server? Does anyone have any feel on how "ready for prime time" CodaFS is? .... Side question: The development team already has a CVS server in place. Would it be best to just take advantage of this server for infrastructure management purposes, or is there a reasonable reason why we should set up our own? TIA, Jim Carroll From hakanson@cse.ogi.edu Tue Apr 16 17:05:55 2002 Received: from church.cse.ogi.edu (root@cse.ogi.edu [129.95.20.2]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA31162 for ; Tue, 16 Apr 2002 17:05:54 -0700 Received: from cse.ogi.edu (hepburn.cse.ogi.edu [129.95.40.57]) by church.cse.ogi.edu (8.9.3/8.9.3) with ESMTP id RAA18837; Tue, 16 Apr 2002 17:06:02 -0700 (PDT) Message-Id: <200204170006.RAA18837@church.cse.ogi.edu> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: "Carroll, Jim" Cc: "Infrastructures (E-mail)" Subject: Re: [Infrastructures] authentication & client file access In-reply-to: Your message of "Tue, 16 Apr 2002 18:28:01 CDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 16 Apr 2002 17:06:02 -0700 From: Marion Hakanson Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Jim Carroll wrote: > I know the Infrastructures web site espouses the use of NIS over all other > options. After reading about the security advantages of NIS+ over NIS, as > well as being urged to implement NIS+, I've set up a master and replica NIS+ It seems to me that NIS security isn't so terrible if your clients (and slaves) don't rely on broadcasts to find a server. Just out of curiousity, why not use LDAP? Solaris-8 supports it natively, on the client side, and it allegedly works with Linux as well. Sun has NIS-to-LDAP gateway code, too, for older clients. I'm not sure what options LDAP has for disconnected mode, except that I just read that you can configure whether the master pushes changes to replicas, or the "slaves" do a "pull". Regarding disconnected NIS authentication, if you run NIS in "compatibility mode" (with "compat" in nsswitch.conf), you can have a local passwd file with (e.g.) a laptop owner's account in it, and at the end a "+" (or some variation) which appends NIS accounts to it. We generally keep the root account in the local passwd file anyway, for example. > 2. Client File Access > . . . > that share read-write. Problem is, anyone who already has root on a Linux > box could conceivably compromise anyone's home directory, simply by creating > a local account and su'ing to the user in question. We've toyed with the > idea of restricting access based on IP address, but also have to deal with > . . . > share. Solaris supports Secure NFS; Linux does not (appear to). With NFS, you really only have host-based authentication (i.e. IP address). We dealt with the "self-managed root" issue here by "exporting" the home directories according to customer group, so GroupA's machines would not be able to clobber homedirs owned by GroupB. The centrally-maintained machines get the top-level homedir area exported to them, since they don't run the risk of self-managed uid clashes (accidental or otherwise). Perhaps you could architect the DHCP ranges to that GroupA's wandering machines get addresses in a particular range, regardless of subnet, and you could still export based on trust-level even for nomadic systems. Just some quick thoughts, and looking forward to further discussion.... Regards, -- Marion Hakanson CSE Computing Facilities From dert@utter.net Tue Apr 16 21:54:32 2002 Received: from seer.utter.net (h0030ab09c36e.ne.client2.attbi.com [66.30.111.217]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA18413 for ; Tue, 16 Apr 2002 21:54:30 -0700 Received: by seer.utter.net (Postfix, from userid 13132) id 65BD6EFCB7; Wed, 17 Apr 2002 00:54:25 -0400 (EDT) Date: Wed, 17 Apr 2002 00:54:25 -0400 From: Andy Davidoff To: infrastructures@terraluna.org Cc: "Carroll, Jim" Subject: Re: [Infrastructures] authentication & client file access Message-ID: <20020417005425.A2835@eecs.tufts.edu> Reply-To: andy+infrastructures@cs.tufts.edu References: <200204170006.RAA18837@church.cse.ogi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200204170006.RAA18837@church.cse.ogi.edu>; from hakanson@cse.ogi.edu on Tue, Apr 16, 2002 at 05:06:02PM -0700 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: [ Are you on the mailing list, Jim? A Cc: just in case... ] We're using NIS+ on Solaris (clients, master server, replicas) and Linux. To my knowledge, there is no NIS+ server for Linux, which means that one cannot apply replicas to Linux mobiles for disconnected authentication.[0] We are pretty heavily invested in dynamic reconfiguration via tools such as Maelstrom and CFengine, but this level of infrastructure is not at all recommended for machines on the fringe of an administrative domain[1]. We run NIS+ for security.[2] NIS+ gives us NFS security which stretches beyond a simple IP-based lookup table to host and user DES credentials. I support the "NIS security isn't" camp. 'Nuff said regarding NIS. :-) Has anyone messed with NYS? I heard a rumor that this offers NIS+ support on Linux with the possibility of running a server. Can anyone confirm? It seems that solid documentation on the Linux side is actually diminishing... Has anyone actually deployed a robust LDAP-NIS+ gateway in production with heterogeneous clients? I'd settle for one flavor of SunOS and one of Linux! Looking forward to another NIS/NIS+/LDAP BoF at LISA, [0] Running NIS+ in NIS-compatibility mode cripples NIS+ security features. [1] By this I mean hosts which benefit from your network's resources but over which you do not maintain sole administrative control. This is a distinction of /policy/ and not technical feasibility. [2] University environment, CompSci students, cracker-magnet resources. -- Andy Davidoff Sen. Unix SysAdmin Tufts University From joelh@cyberzod.com Thu Apr 18 00:06:49 2002 Received: from cyberzod.com (12-253-25-167.client.attbi.com [12.253.25.167]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id AAA25859 for ; Thu, 18 Apr 2002 00:06:48 -0700 Received: from saber (saber.nabs.com [192.168.128.7]) by cyberzod.com (8.11.2/8.11.2) with SMTP id g3I76wm25949 for ; Thu, 18 Apr 2002 01:06:58 -0600 From: "Joel Huddleston" To: "Infrastructures \(E-mail\)" Subject: RE: [Infrastructures] authentication & client file access Date: Thu, 18 Apr 2002 01:06:58 -0600 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) In-reply-to: X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 Importance: Normal Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: The use of NIS vs. NIS+ vs. Flatfiles vs. 3rd party vs. roll-your-own raises its head again. Well, let me tell you my whys and whynots: Before I go any further, I'd like to point out that the Infrastructure site "describes" the use of NIS, but I don't think I'd go as far as saying that it "espouses" it. My personal complaint with NIS/NIS+ is that they REPLACE or AUGMENT the functionality of services other than passwd and group text. NIS is a poor replacement for BIND or a decent /etc/services file. (Imagine the paradox of that last service: I want to connect to the NIS server, so I need its port from rpc, but I need the rpc port number from /etc/services, but I am configure to get this entry from the NIS services map, so I need to connect to the NIS server...) It doesn't really happen this way; the rpc port is hard-coded in the NIS code to avoid the problem. It just irks me when people won't use the Internet established solution, BIND, with the lame-ass excuse "We use NIS." ---NIS Why: Easy, Just about everyone supports it, easy to integrate into local apps Why not: Easy to hack, cleartext passwords available to crackers, no client-side authentication, poor logging, NIS server is a passwd text server and not an authentication server, still limited to 8 character passwords ---NIS+ Why: Sun only environment, OK--some functionality in hetero environments but servers limited to sun, easy to integrate into local apps Why not: NIS+ server uses high-level authentication (Kerberos 4) to deliver (my favorite part:) passwd text entries, Kerberos 4 implemented but DOES NOT authenticate users, just NIS+ access, still limited to 8 character passwords, still poor logging, Sun doesn't even use NIS+ internally Hard lessons learned: When NIS+ got integrated into the last environment I saw, it was used for EVERYTHING, even databases that could have been implemented better using other tools. When the headaches of managing NIS+ finally got to us, we found that we could not go back because too many tools had been rolled around NIS+ by well-meaning, but misguided developers. This was especially annoying when those tools had nothing whatsoever to do with authentication/authorization and instead used NIS+ as a cheap-ass database tool. I strongly urge you, based on this experience, to avoid using NIS/NIS+ for anything other than authentication/authorization. Get a real database (even MySQL) to solve the other problems. ---LDAP Why: Gaining acceptance, Lengthy passwords Why not: hard to get going "out of the box", limited vendor support, many vendors claiming to "do" LDAP with few of them recommending it, may be tricky to integrate into local apps ---Kerberos 4 Why: No good reasons with K5 out. Why not: design flaws in protocol introduce security risks, no current code support ---Kerberos 5 Why: Good authentication system, Single Sign On (not just common password) reduces the number of times a user must authenticate and therefor the likelyhood that their passwords will be shoulder-surfed, Versatile password construction rules, Password aging, Lengthy passwords, Great logging, interoperates with "The Bill Gates Operating System Which Must Not Be Named", easy to integrate into local apps (if you read the documentation, that is) Why not: Does not solve the problem of serving passwd entries (authentication without authorization), Limited vendor support, Locally maintained sources must be compiled and distributed, Hard to get management buy-in for an Opensource solution, no maintenance tools to speak of ---Flat Files Why: Comes this way out of the box, shadows can be used to "increase" security of hashes, some systems support MD5 hashed passwords Why not: obviously a support nightmare ---3rd Party Why: Usually comes with comprehensive management tools Why not: Limited vendor support, Limited platform support, Algorithms have probably not been subjected to peer review and may have protocol or implementation bugs that reduce security, Highly variable success stories, may be difficult to integrate into local apps --------------------------------------------------- My last really successful deployment of a security infrastructure consisted of: Roll-my-own scripts to deliver (via CMUSup) passwd text to clients with NO HASHES. Kerberos 5 SSO compiled and run "straight out of the box" to authenticate users. SecurID/ACE one-time passcodes for "sensitive" systems. A bunch of scripts that integrated these three services. A bunch of scripts that integrated the Cisco IOS systems written by co-worker. A PAM module for Solaris (written by me with generous supplies of code from the Kerb5 sources.) Kerberos for Linux worked "out of the box." Note that this security infrastructure was used in a mixed environment and authenticated users on the following hardware/systems: Solaris 2.6, 2.7, 8 Cisco IOS (11 I think) RedHat Linux 6.0, 6.1, 7.0 ApacheSSL Web Sites on Linux Windows NT SSH v1 Secure FTP sudo There may have been others, but these were the major systems. Problems I never solved with this: o Disconnected laptops. o Xwindows. o SSO support to web pages. (could authenticate with Kerberos password, but no GSSAPI automated stuff) I had a solution prepared for the disconnected Laptop problem but never got it coded. My solution was: o A user tries to authenticate to the server. o IF server sends a token: o If the user decrypts the token, then CACHE THE ENCRYPTED TOKEN and user is authenticated. o else (the server is not available): o Look for a cached encrypted token for this user. o If found and it decrypts with user password, user is authenticated. Note that this second type of authentication will not grant the user a current TGT. So no Kerberized services can be used, only the local box. The user will have to kinit later if the server was just unreachable. I still have the source for my Solaris PAM module for Kerberos 5. If anyone wants it, I can supply. It had some features that Sun may not have put into theirs. For X, I had thought that I might have the networking guys disable port 6000 through the routers and demand that people use SSH tunnels to partially solve the Xwindows problem. This is, however, a "half-assed" (tm) solution. I tried to demonstrate the vulnerability of Xwindows to a manager by printing his X screen from his locked office, but he thought I had used some special adminy sort of power and didn't buy in. X is just a horrendous, bloated, slow security risk that everyone seems content to live with. Go figure. Wow, I typed a lot. I must have felt strongly about this. :) Jim also mentioned Client File Access. I have no solution for this problem that won't get me flamed to a crisp, so I'll just say "I am waiting for Coda." _______________________________________________ Infrastructures mailing list Infrastructures@mailman.terraluna.org http://mailman.terraluna.org/mailman/listinfo/infrastructures From partain@dcs.gla.ac.uk Thu Apr 18 04:26:06 2002 Received: from slicker.dcs.gla.ac.uk (slicker.dcs.gla.ac.uk [130.209.242.51]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id EAA31689 for ; Thu, 18 Apr 2002 04:26:00 -0700 Received: (from partain@localhost) by slicker.dcs.gla.ac.uk (8.8.8+Sun/8.8.8) id MAA15533; Thu, 18 Apr 2002 12:26:07 +0100 (BST) X-Authentication-Warning: slicker.dcs.gla.ac.uk: partain set sender to partain@dcs.gla.ac.uk using -f To: "Carroll, Jim" Cc: "Infrastructures (E-mail)" , paul@dcs.ed.ac.uk Subject: Re: [Infrastructures] authentication & client file access References: From: Will Partain Date: 18 Apr 2002 12:26:07 +0100 In-Reply-To: Message-ID: Lines: 27 User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.4 (Common Lisp) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: "Carroll, Jim" writes: > In trying to roll out a 'proper' infrastructure architecture, I'm running > into the odd snag which usually gets me pondering about what constitutes the > best approach. > > I'm looking for input on the following: > > 1. Authentication servers > 2. Client File Access Jim, et al. -- The "big picture" you're looking at smells a lot like the one that Paul Anderson's group at Edinburgh has been working on for while (their post-LCFG work). I'm pretty sure someone from that group is on this list -- could someone give a synopsis, if only of the issues? Moreover, I can't believe that there aren't *lots* of people looking at much the same "big picture". If you are such a person/group, could you please post a few lines of "how it looks to us"? (The other messages in this thread have been interesting! Keep it up. Jim, if you got private replies, could you summarize?) Will From partain@dcs.gla.ac.uk Thu Apr 18 12:56:37 2002 Received: from iona.dcs.gla.ac.uk (iona.dcs.gla.ac.uk [130.209.240.35]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA10056 for ; Thu, 18 Apr 2002 12:56:36 -0700 Received: from slicker.dcs.gla.ac.uk ([130.209.242.51] helo=dcs.gla.ac.uk) by iona.dcs.gla.ac.uk with esmtp (Exim 3.13 #1) id 16yI1N-0003a9-00 for infrastructures@terraluna.org; Thu, 18 Apr 2002 20:56:45 +0100 From: Will Partain To: infrastructures@terraluna.org X-Mailer: mh-e 6.1; nmh 1.0.4; Emacs 21.4 Date: Thu, 18 Apr 2002 20:56:44 +0100 Message-Id: Subject: [Infrastructures] good "network design" books/references? Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Lest anyone tire of big-picture things-to-think-about... here's another one: Know of any good books/articles/web-pages/references for doing "network design" for a (not-too-big) organization? My initial interest is of the high-level "what are the right conceptual ideas/models I should be using?" variety. Grumbles with what I've bumped into so far: * It is written by the security-obsessed, with no apparent thought to what would be good/useful/helpful to those trying to use the thing. Anything more balanced? * It is of the conventional firewall/NAT/VPNs/intranet/etc school of thought. I would prefer something closer to old-fashioned "end-to-end" thinking (see http://www.reed.com/Papers/EndtoEnd.html and perhaps also http://www.reed.com/Papers/endofendtoend.html), but am prepared to be educated :-) I will summarize back to the list if useful info comes my way. Thanks, Will From stevegt@roton.terraluna.org Fri Apr 19 10:38:11 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id KAA09001; Fri, 19 Apr 2002 10:38:11 -0700 Date: Fri, 19 Apr 2002 10:38:10 -0700 From: Steve Traugott To: infrastructures@TerraLuna.Org Cc: luke@madstop.com, labrown@nc.rr.com Message-ID: <20020419103809.A4927@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] The Turing Synthesis and Automated Administration Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All! Woke up this morning at 3:00 a.m., with a paper full-blown in my head, fighting to get out. Some of you have heard parts of this before, and credit for the original idea goes to Lance Brown, from a discussion we had at LISA last fall. I'd welcome debate and feedback. Nutshell: Self-administering hosts are in effect Turing machines; they do brain surgery on themselves. They don't behave according to strict Von-Neumann rules, but have complex feedback loops, recursion, and self-modifying code. *No* language or syntax can completely specify administrative actions to be applied to a machine without also specifying the complete history and order of previous changes. This is because the behavior of the interpreter or compiler for any language is itself dependent on the current disk contents, because the interpreter/compiler executes in the context of the current disk. Using a change management tool to detect current disk content and act accordingly is problematic for the same reason. This means that proactive ordering of actions is very important. This runs contrary to a recent trend in LISA papers, and this is the first time I've been able to demonstrate it in writing. The draft is at: http://www.infrastructures.org/papers/turing.html Can anyone see anything wrong with this argument so far? Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From jrstear@sandia.gov Fri Apr 19 11:46:29 2002 Received: from mm01snlnto.sandia.gov (mm01snlnto.sandia.gov [132.175.109.20]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id LAA12717; Fri, 19 Apr 2002 11:46:28 -0700 Received: from 132.175.109.1 by mm01snlnto.sandia.gov with ESMTP ( Tumbleweed MMS SMTP Relay (MMS v4.7)); Fri, 19 Apr 2002 12:46:16 -0600 X-Server-Uuid: 95b8ca9b-fe4b-44f7-8977-a6cb2d3025ff Received: from mercy.sandia.gov (mercy.sandia.gov [134.253.242.218]) by sass165.sandia.gov (8.12.3/8.12.3) with ESMTP id g3JIkafC007558; Fri, 19 Apr 2002 12:46:36 -0600 (MDT) Received: from jrstear by mercy.sandia.gov with local (Exim 3.33 #1 ( Debian)) id 16ydP0-0001ND-00; Fri, 19 Apr 2002 12:46:34 -0600 Date: Fri, 19 Apr 2002 12:46:34 -0600 To: "Steve Traugott" cc: infrastructures@terraluna.org, luke@madstop.com, labrown@nc.rr.com Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration Message-ID: <20020419184634.GD4514@sandia.gov> References: <20020419103809.A4927@scramjet.TerraLuna.Org> MIME-Version: 1.0 In-Reply-To: <20020419103809.A4927@scramjet.TerraLuna.Org> User-Agent: Mutt/1.3.24i From: "Jon Stearley" X-Filter-Version: 1.8 (sass165) X-WSS-ID: 10DEBA72493881-01-01 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > Self-administering hosts are in effect Turing machines; they do brain > surgery on themselves. They don't behave according to strict > Von-Neumann rules, but have complex feedback loops, recursion, and > self-modifying code. > > *No* language or syntax can completely specify administrative actions > to be applied to a machine without also specifying the complete > history and order of previous changes. > > This is because the behavior of the interpreter or compiler for any > language is itself dependent on the current disk contents, because > the interpreter/compiler executes in the context of the current disk. > Using a change management tool to detect current disk content and act > accordingly is problematic for the same reason. true only when the robot's behavior ("change management tool" in this context) is affected by the changes it is making, correct? while _not_null_, in practice i believe this is _generally_negligable_. when needed (and it is sometimes no doubt!), press the reset button on your virtual-host (ie- "auto-reinstall" your infrastructure) and you've eliminated the requirement of having to maintain all the nodes and edges of your state graph - a seductive goal indeed! if above is correct (?), school one needs to be updated to: "order doesn't matter, as long as a reset mechanism is available when necessary". > This means that proactive ordering of actions is very important. This > runs contrary to a recent trend in LISA papers, and this is the first > time I've been able to demonstrate it in writing. excellent stuff!!! definately write it up for that looming deadline... ;) -- +--------------------------------------------------------------+ | Jon Stearley (505) 845-7571 (FAX 844-2067) | | Compaq Federal LLC High Performance Solutions | | Sandia National Laboratories Scalable Systems Integration | +--------------------------------------------------------------+ From mcarter@lanl.gov Fri Apr 19 11:59:18 2002 Received: from mailrelay1.lanl.gov (mailrelay1.lanl.gov [128.165.4.101]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id LAA18171; Fri, 19 Apr 2002 11:59:17 -0700 Received: from nis-mail.lanl.gov (localhost.localdomain [127.0.0.1]) by mailrelay1.lanl.gov (8.11.6/8.11.6/(ccn-5)) with ESMTP id g3JIwqe01308; Fri, 19 Apr 2002 12:58:52 -0600 Received: from guglielmo.lanl.gov (guglielmo.lanl.gov [128.165.207.66]) by nis-mail.lanl.gov (8.11.6/8.11.6/(ccn-5)) with ESMTP id g3JIwtI00899; Fri, 19 Apr 2002 12:58:55 -0600 Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration From: "Michael J. Carter" To: Jon Stearley Cc: Steve Traugott , infrastructures@terraluna.org, luke@madstop.com, labrown@nc.rr.com In-Reply-To: <20020419184634.GD4514@sandia.gov> References: <20020419103809.A4927@scramjet.TerraLuna.Org> <20020419184634.GD4514@sandia.gov> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.3 Date: 19 Apr 2002 12:58:55 -0600 Message-Id: <1019242735.32636.69.camel@guglielmo> Mime-Version: 1.0 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, 2002-04-19 at 12:46, Jon Stearley wrote: > > Self-administering hosts are in effect Turing machines; they do brain > > surgery on themselves. They don't behave according to strict > > Von-Neumann rules, but have complex feedback loops, recursion, and > > self-modifying code. > > > > *No* language or syntax can completely specify administrative actions > > to be applied to a machine without also specifying the complete > > history and order of previous changes. > > > > This is because the behavior of the interpreter or compiler for any > > language is itself dependent on the current disk contents, because > > the interpreter/compiler executes in the context of the current disk. > > Using a change management tool to detect current disk content and act > > accordingly is problematic for the same reason. > > true only when the robot's behavior ("change management tool" in this > context) is affected by the changes it is making, correct? while > _not_null_, in practice i believe this is _generally_negligable_. > when needed (and it is sometimes no doubt!), press the reset button on > your virtual-host (ie- "auto-reinstall" your infrastructure) and > you've eliminated the requirement of having to maintain all the nodes > and edges of your state graph - a seductive goal indeed! > > if above is correct (?), school one needs to be updated to: > "order doesn't matter, as long as a reset mechanism is available > when necessary". > I think what Steve is driving at is that while 'a reset mechanism' may be available, who decides when it needs to be used? I don't see how a self-administering host can know that it has deviated from the norm (presumably due to an "order error") and needs to start at square one. If the admin has to push reset, then the host isn't self-administering. Cheers, mjc > > > > This means that proactive ordering of actions is very important. This > > runs contrary to a recent trend in LISA papers, and this is the first > > time I've been able to demonstrate it in writing. > > excellent stuff!!! definately write it up for that looming > deadline... ;) > > -- > +--------------------------------------------------------------+ > | Jon Stearley (505) 845-7571 (FAX 844-2067) | > | Compaq Federal LLC High Performance Solutions | > | Sandia National Laboratories Scalable Systems Integration | > +--------------------------------------------------------------+ > > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures -- Michael J. Carter | Mason's First Law of Synergism: The one IT Team Leader | day you'd sell your soul for something, Space Data Systems (NIS-3) | souls are a glut. Los Alamos National Laboratory | From jrstear@sandia.gov Fri Apr 19 13:13:42 2002 Received: from mm02snlnto.sandia.gov (mm02snlnto.sandia.gov [132.175.109.21]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id NAA24756; Fri, 19 Apr 2002 13:13:41 -0700 Received: from 132.175.109.1 by mm02snlnto.sandia.gov with ESMTP ( Tumbleweed MMS SMTP Relay (MMS v4.7)); Fri, 19 Apr 2002 14:13:23 -0600 X-Server-Uuid: 95b8ca9b-fe4b-44f7-8977-a6cb2d3025ff Received: from mercy.sandia.gov (mercy.sandia.gov [134.253.242.218]) by sass165.sandia.gov (8.12.3/8.12.3) with ESMTP id g3JKDofC014282; Fri, 19 Apr 2002 14:13:51 -0600 (MDT) Received: from jrstear by mercy.sandia.gov with local (Exim 3.33 #1 ( Debian)) id 16yelQ-0001PS-00; Fri, 19 Apr 2002 14:13:48 -0600 Date: Fri, 19 Apr 2002 14:13:48 -0600 To: "Michael J. Carter" cc: "Steve Traugott" , infrastructures@terraluna.org, luke@madstop.com, labrown@nc.rr.com Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration Message-ID: <20020419201348.GF4514@sandia.gov> References: <20020419103809.A4927@scramjet.TerraLuna.Org> <20020419184634.GD4514@sandia.gov> <1019242735.32636.69.camel@guglielmo> MIME-Version: 1.0 In-Reply-To: <1019242735.32636.69.camel@guglielmo> User-Agent: Mutt/1.3.24i From: "Jon Stearley" X-Filter-Version: 1.8 (sass165) X-WSS-ID: 10DEA5E9521869-01-01 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > be available, who decides when it needs to be used? > > I don't see how a self-administering host can know that it has deviated > from the norm (presumably due to an "order error") and needs to start at > square one. > > If the admin has to push reset, then the host isn't self-administering. absolutely true. but is fully/truely/super-duper self-administering infrastructures really the goal? there are precious few (*if any*) human creations for which periodic human resets are not necessary and accepted practice. steve's points (and the whole discussion) is extremely valueable, don't get me wrong. industry chooses something along the lines of maximum effectiveness/(cost/benfit), that's all. my 2c. -jon From stevegt@roton.terraluna.org Fri Apr 19 18:11:34 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id SAA18921; Fri, 19 Apr 2002 18:11:34 -0700 Date: Fri, 19 Apr 2002 18:11:33 -0700 From: Steve Traugott To: Jon Stearley Cc: "Michael J. Carter" , infrastructures@terraluna.org, luke@madstop.com, labrown@nc.rr.com Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration Message-ID: <20020419181133.A22600@scramjet.TerraLuna.Org> References: <20020419103809.A4927@scramjet.TerraLuna.Org> <20020419184634.GD4514@sandia.gov> <1019242735.32636.69.camel@guglielmo> <20020419201348.GF4514@sandia.gov> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020419201348.GF4514@sandia.gov>; from jrstear@sandia.gov on Fri, Apr 19, 2002 at 02:13:48PM -0600 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi Jon! On Fri, Apr 19, 2002 at 02:13:48PM -0600, Jon Stearley wrote: > > be available, who decides when it needs to be used? > > > > I don't see how a self-administering host can know that it has deviated > > from the norm (presumably due to an "order error") and needs to start at > > square one. > > > > If the admin has to push reset, then the host isn't self-administering. > > absolutely true. but is fully/truely/super-duper self-administering > infrastructures really the goal? there are precious few (*if any*) > human creations for which periodic human resets are not necessary and > accepted practice. steve's points (and the whole discussion) is > extremely valueable, don't get me wrong. industry chooses something > along the lines of maximum effectiveness/(cost/benfit), that's all. > my 2c. I think this is a key place where my point has been missed a few times over the years. Self-administering can't be done halfway -- doing some things automatically and some things manually is the *most* expensive way to go, because you have the overhead of the automation as well as the indeterminate behavior of human error. It's the worst of both worlds. Fully self-administering is what I do. It's all I do. To me, it's not a "goal" any more -- it's instead been a practice for 8 years. Things are getting better, but for most of that 8 years I've felt like a 1905-era pilot being told that what he does is impossible because 'that thang can't never fly'. ;-) At the other extreme, I've seen some really skull-cracking analysis of how self-administering systems *might* be done at some point in the future. These are important from a research perspective, and I've gotten good ideas from them, but I've tried very hard to stay right in the middle, always sticking with tools and techniques usable in current production -- as a consultant, I can't get too far ahead or I don't get paid. As most of you know by now, all I've usually ever used is makefiles and some wrapper scripts that run at boot -- the sysadmin equivalent of cloth wings and piano wire. We can work together on building aluminum wings and jet engines, but in order for us to do that we first need to finally agree that human flight is feasible and desireable. I don't rely on "if all else fails, re-install". I can see how the "virtual machine" idea in the bootstrapping paper may have given this misimpression -- at one point I think I even use the phrase "rebuilding a virtual machine". Re-installing is a disaster-recovery response. It's expensive in labor, expensive in political capital unless there's actually been a disaster, and can be horribly expensive in terms of reliability and unscheduled downtime. From my perspective, jumpstarting machines to get them back into a known state is a worst-case scenario. Once you've crashed the airplane, you can rebuild it from scratch, but the damage you've done to the passengers by that point is irreversible. "Hey, I've just fubared all 2000 of the machines on the trading floor; can someone stop the markets for a few hours while I re-install them?" That's a situation I wanted to avoid at all costs. ;-) It's been 8 years, thousands of machines, and several organizations since I started doing deterministic, ordered changes, and I can't recall a time when I've ever re-installed the same operating system on a managed production desktop or server, short of hardware failure. Joel, can you think of anything? I suspect deterministic ordering is the airfoil of self-administering systems. If I'm right, then purposely randomized ordering, while fascinating, might be roughly equivalent to wing-flapping machanisms. It took centuries for aerodynamics experimenters to figure out and prove to each other that the flapping wasn't important, but the airfoil was. I think we may be in a similar situation here. I know that to encourage things to get out of sequence is to risk breaking production machines in such a way that users will notice. This is exactly what we're supposed to prevent. It's a sysadmin's version of the hippocratic oath; "First, do no harm". "But ordering isn't as important as long as you test your changes before putting them in production." Sure. But to do that, the production machines have to have been built *exactly* the same way that the test environment was, or else the test is invalid. The only way to ensure that test and production are built the same way is to, well, build them the same way, applying changes in the same order in each case. "But you can just test in production, and if the change broke something, you can always back it out. If the backout fails, then just re-install". If anyone still feels this way, then they should re-read the above paragraphs about reliability and downtime. Testing in production, and relying on scheduled downtime and backout windows, eats into uptime numbers and precludes 24x7 operation. A global economy has no "off hours". I think about this every time I'm waiting in line at the Hertz counter after arriving in Tampa at 2 a.m. on a Friday night, right smack in the middle of the Hertz scheduled maintenance downtime. Does any of this make sense? Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From docx@io.com Fri Apr 19 19:29:51 2002 Received: from solomon.io.com (root@solomon.io.com [199.170.88.24]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id TAA27193; Fri, 19 Apr 2002 19:29:49 -0700 Received: from eris.io.com (IDENT:root@eris.io.com [199.170.88.11]) by solomon.io.com (8.11.2/8.11.2) with ESMTP id g3K2TO225240; Fri, 19 Apr 2002 21:29:24 -0500 Received: from localhost (docx@localhost) by eris.io.com (8.11.6/8.11.6) with ESMTP id g3K2TOD10377; Fri, 19 Apr 2002 21:29:24 -0500 X-Authentication-Warning: eris.io.com: docx owned process doing -bs Date: Fri, 19 Apr 2002 21:29:23 -0500 (CDT) From: Dylan Northrup To: Steve Traugott cc: Jon Stearley , "Michael J. Carter" , , , Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration In-Reply-To: <20020419181133.A22600@scramjet.TerraLuna.Org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: An infinite number of monkeys in the guise of Steve Traugott wrote: :=I think this is a key place where my point has been missed a few times :=over the years. Self-administering can't be done halfway -- doing :=some things automatically and some things manually is the *most* :=expensive way to go, because you have the overhead of the automation :=as well as the indeterminate behavior of human error. It's the worst :=of both worlds. I disagree. It's a law of diminishing returns. The closer you get to fully automating your system, the harder it becomes to completely automate away the rest of it. It's the computer version of Xeno's Paradox where you really *can* only get half the way through what's left. Also, if you really *can* automate *everything* then you're not solving interesting problems, or your problems aren't evolving, or your customers are very boring and completely non-standard indeed. I find that once you've conquered/automated a particular problem, there's always one more problem to solve . . . a wrinkle I hadn't thought of or a new requirement I didn't know about. You can never fully automate a sufficiently complex system. . . especially when users are involved ;-) Now, this isn't to say you shouldn't automate as much as possible. Fix a problem once and either solve the problem completely or automate the work around. Your toolkit should always be expanding. But the tools won't do the job without proper guidance from the mechanic. [rest of comments snipped as I agree with them and can add nothing else meaningful] :=Does any of this make sense? Yep. -- Dylan Northrup <*> docx@io.com <*> http://www.io.com/~docx/ "Easy to bitch, easy to whine, easy to moan, easy to cry, easy to feel like there ain't nothing in your life. Harder to work, harder to strive, hard to be glad to be alive, but it's really worth it if you give it a try." -- Cowboy Mouth, 'Easy' From dmasten@piratelabs.org Fri Apr 19 23:16:49 2002 Received: from localhost.localdomain (m198-157.dsl.rawbw.com [198.144.198.157]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id XAA14928 for ; Fri, 19 Apr 2002 23:16:48 -0700 Received: (from dmasten@localhost) by localhost.localdomain (8.11.6/8.11.6) id g3K6Fw301838; Fri, 19 Apr 2002 23:15:58 -0700 X-Authentication-Warning: localhost.localdomain: dmasten set sender to dmasten@piratelabs.org using -f Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration From: David Masten To: Dylan Northrup Cc: infrastructures@terraluna.org In-Reply-To: References: Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.3 Date: 19 Apr 2002 23:15:57 -0700 Message-Id: <1019283357.1807.24.camel@localhost.localdomain> Mime-Version: 1.0 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, 2002-04-19 at 19:29, Dylan Northrup wrote: > I disagree. It's a law of diminishing returns. The closer you get to fully > automating your system, the harder it becomes to completely automate away > the rest of it. It's the computer version of Xeno's Paradox where you > really *can* only get half the way through what's left. > Touching a production machine invariably means the machine is not doing five nines uptime. Production machines must be fully automated. In the many manufacturing industries the line person has no thought in the making of the product. Everything is done in a very specific order, deviation results in scrap at best, and death and liability suits at worst. I think this is what Steve is talking about in "automating" everything. The interesting problems are solved in the test facilities, not the customer's hands. On the trading floor, in the data center, and in the cubicles is not where we should handle interesting problems. > Also, if you really *can* automate *everything* then you're not solving > interesting problems, or your problems aren't evolving, or your customers > are very boring and completely non-standard indeed. I find that once you've > conquered/automated a particular problem, there's always one more problem to > solve . . . a wrinkle I hadn't thought of or a new requirement I didn't know > about. You can never fully automate a sufficiently complex system. . . > especially when users are involved ;-) And hopefully you have found and solved the interesting problems in the test lab, not the trading floor. ;-) Dave From Tarjei.Jensen@kvaerner.com Sat Apr 20 03:22:57 2002 Received: from kcpgb-mime03.kvaerner.com ([193.132.79.11]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id DAA13014 for ; Sat, 20 Apr 2002 03:22:56 -0700 Received: from kcpgb-bb01.messaging.kvaerner.com (unverified) by kcpgb-mime03.kvaerner.com (Content Technologies SMTPRS 4.2.5) with ESMTP id for ; Sat, 20 Apr 2002 11:19:37 +0100 Received: by KCPGB-BB01 with Internet Mail Service (5.5.2653.19) id ; Sat, 20 Apr 2002 11:20:41 +0100 Message-ID: From: "Jensen, Tarjei KOGAS" To: "'infrastructures@TerraLuna.Org'" Date: Sat, 20 Apr 2002 11:21:02 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Infrastructures] RE: Infrastructures digest, Vol 1 #31 - 6 msgs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Steve Traugott wrote: > Self-administering hosts are in effect Turing machines; they do brain > surgery on themselves. They don't behave according to strict > Von-Neumann rules, but have complex feedback loops, recursion, and > self-modifying code. > > *No* language or syntax can completely specify administrative actions > to be applied to a machine without also specifying the complete > history and order of previous changes. > > This is because the behavior of the interpreter or compiler for any > language is itself dependent on the current disk contents, because > the interpreter/compiler executes in the context of the current disk. > Using a change management tool to detect current disk content and act > accordingly is problematic for the same reason. > > This means that proactive ordering of actions is very important. This > runs contrary to a recent trend in LISA papers, and this is the first > time I've been able to demonstrate it in writing. > > The draft is at: > > http://www.infrastructures.org/papers/turing.html > > Can anyone see anything wrong with this argument so far? You would expect a expert system to be able to handle history. You won't get a good system without storing a history and having some sort of expert system to interpret that. I don't think it is absolutely neccessary to track the changes to the system configuration apart from a evaluation standpoint. It is nice to know what worked and what did not work at all. What is neccessary, is to track is what sort of loads the system have had. What the resource bottlenecks were. VMS has a expert system which configure the machine, but at the time I used it, it did not use history, so it was neccessary to include some pretty strong hints in a configuration file. Apart from that it seemed to work reasonably well. greetings, From matt@loudcloud.com Sat Apr 20 09:25:39 2002 Received: from listproc.corp.loudcloud.com (olly.loudcloud.com [66.54.20.10]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id JAA11338 for ; Sat, 20 Apr 2002 09:25:39 -0700 Received: from loudcloud.com (mattm.corp.loudcloud.com [192.168.32.234]) by listproc.corp.loudcloud.com (8.11.6/8.11.6) with ESMTP id g3KGS7f25362; Sat, 20 Apr 2002 09:28:07 -0700 (PDT) Message-ID: <3CC19800.96D93B9B@loudcloud.com> Date: Sat, 20 Apr 2002 09:32:00 -0700 From: Matt Machczynski X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: David Masten CC: Dylan Northrup , infrastructures@terraluna.org Subject: Re: [Infrastructures] The Turing Synthesis and AutomatedAdministration References: <1019283357.1807.24.camel@localhost.localdomain> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: David Masten wrote: > On Fri, 2002-04-19 at 19:29, Dylan Northrup wrote: > > > I disagree. It's a law of diminishing returns. The closer you get to fully > > automating your system, the harder it becomes to completely automate away > > the rest of it. It's the computer version of Xeno's Paradox where you > > really *can* only get half the way through what's left. > Touching a production machine invariably means the machine is not doing > five nines uptime. Production machines must be fully automated. > > In the many manufacturing industries the line person has no thought in > the making of the product. Everything is done in a very specific order, > deviation results in scrap at best, and death and liability suits at > worst. I think this is what Steve is talking about in "automating" > everything. The interesting problems are solved in the test facilities, > not the customer's hands. On the trading floor, in the data center, and > in the cubicles is not where we should handle interesting problems. > I think you're both right to a degree. As the amount of change is introduced into the system goes up, the ability to automate it completely decreases. If you're dealing with a set of requirements n which changes with a standard build cycle every 6 months then close to total automation can occur. As the amount of change increases (and the types of change increase...also important), it becomes more difficult to automate it all and meet customer schedule. > > > Also, if you really *can* automate *everything* then you're not solving > > interesting problems, or your problems aren't evolving, or your customers > > are very boring and completely non-standard indeed. I find that once you've > > conquered/automated a particular problem, there's always one more problem to > > solve . . . a wrinkle I hadn't thought of or a new requirement I didn't know > > about. You can never fully automate a sufficiently complex system. . . > > especially when users are involved ;-) > > And hopefully you have found and solved the interesting problems in the > test lab, not the trading floor. ;-) > > Dave m. From p_james_carroll@yahoo.com Thu Apr 11 07:27:41 2002 Received: from web12704.mail.yahoo.com (web12704.mail.yahoo.com [216.136.173.241]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id HAA11677 for ; Thu, 11 Apr 2002 07:27:41 -0700 Message-ID: <20020411142747.48003.qmail@web12704.mail.yahoo.com> Received: from [208.24.179.207] by web12704.mail.yahoo.com via HTTP; Thu, 11 Apr 2002 07:27:47 PDT Date: Thu, 11 Apr 2002 07:27:47 -0700 (PDT) From: Jim Carroll To: Infrastructures MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Infrastructures] leveraging CVS Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: In my current environment, I'm slowly working on implementing an infrastructure, based on the methodologies on infrastructures.org. Looking at the CVS requirement, I'm told that the developers already have a CVS server in place (and have had for some time). Should I take advantage of this, or should I set up my own infrastructures-only CVS server? TIA, Jim Carroll __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/ From pincky@soon.com Sun Apr 14 04:33:18 2002 Received: from ws1-11.us4.outblaze.com (205-158-62-112.outblaze.com [205.158.62.112]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id EAA27102 for ; Sun, 14 Apr 2002 04:33:17 -0700 Received: (qmail 60268 invoked by uid 1001); 14 Apr 2002 11:32:51 -0000 Message-ID: <20020414113251.60267.qmail@mail.com> Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 7bit MIME-Version: 1.0 X-Mailer: MIME-tools 5.41 (Entity 5.404) Received: from [62.14.125.164] by ws1-11.us4.outblaze.com with http for pincky@soon.com; Sun, 14 Apr 2002 11:32:51 +0000 From: "Joel Pinckheard" To: "Carroll, Jim" , "Infrastructures \(E-mail\)" Date: Sun, 14 Apr 2002 11:32:51 +0000 Subject: RE: [Infrastructures] IS monitoring X-Originating-Ip: 62.14.125.164 X-Originating-Server: ws1-11.us4.outblaze.com Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: We just started to use Big Sister (http://bigsister.graeff.com/) for our site. It claims to be completly compatible with Big Brother clients and scripts. We choose it because it was Perl based with a fairly simple interface to setting up graphs with RDDtool. Another nice feature is that the dependancy checks are fairly flexible... Documentation is a bit lacking at the moment but so far we haven´t run into problems. That said we only installed it a couple of weeks ago! -- Joel Pinckheard pincky@pincky.com http://www.pincky.com From stevegt@roton.terraluna.org Sat Apr 20 11:11:12 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id LAA01442 for infrastructures@TerraLuna.Org; Sat, 20 Apr 2002 11:11:12 -0700 Date: Sat, 20 Apr 2002 11:11:12 -0700 From: Steve Traugott To: infrastructures@TerraLuna.Org Message-ID: <20020420111111.B24714@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Administrivia Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All, Speaking of automation, I just approved a few messages that had stacked up in the mailman admin queue because they were posted from non-member addresses. Could I convince those of you with multiple addresses to stick to using one, or else to subscribe all of them? (And does anyone know of a way to make mailman reject these automatically rather than holding them for review?) Thanks, Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From stevegt@roton.terraluna.org Sat Apr 20 14:44:34 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id OAA13647; Sat, 20 Apr 2002 14:44:34 -0700 Date: Sat, 20 Apr 2002 14:44:34 -0700 From: Steve Traugott To: Dylan Northrup Cc: Jon Stearley , "Michael J. Carter" , infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration Message-ID: <20020420144434.A6759@scramjet.TerraLuna.Org> References: <20020419181133.A22600@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from docx@io.com on Fri, Apr 19, 2002 at 09:29:23PM -0500 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, Apr 19, 2002 at 09:29:23PM -0500, Dylan Northrup wrote: > An infinite number of monkeys in the guise of Steve Traugott wrote: > > :=I think this is a key place where my point has been missed a few times > :=over the years. Self-administering can't be done halfway -- doing > :=some things automatically and some things manually is the *most* > :=expensive way to go, because you have the overhead of the automation > :=as well as the indeterminate behavior of human error. It's the worst > :=of both worlds. > > I disagree. It's a law of diminishing returns. The closer you get to fully > automating your system, the harder it becomes to completely automate away > the rest of it. It's the computer version of Xeno's Paradox where you > really *can* only get half the way through what's left. We have to be using different meanings of the term "automate". Automated systems administration is very straightforward. There is only one way to change the contents of disk or RAM in a running UNIX machine -- the syscall interface. The task of automated administration is simply to make sure that each machine's kernel gets the right system calls, in the right order, to make it be the machine you want it to be. There are no system calls used in the course of administration which can't be issued from a shell script, a perl script, or (in very rare cases) a piece of C code. This makes the process of putting automation in place itself very straightforward. In my case, rather than typing a command at a shell prompt, I put it into a makefile stanza. If it's a complex series of commands, with conditionals and loops etc., then it goes into a script, the script goes into CVS, and the script name goes into the makefile. Putting the CVS and makefile framework in place is itself also very straightforward, as we just proved at Caterpillar earlier this year -- Luke took 6 hours to go from scratch to a running installation of isconf, with me looking over his shoulder. Luke is an exceptionally smart guy, (Hi Luke!) ;-P But I think just about anyone on this list should be able to get similar results. I was there for a week; the first 4 days were spent on political buy-in, only the last on technology. This is important -- the only major hurdles I have ever run into for the last few years are political rather than technological. The major improvement I've had to make in myself is in learning how to show both management and sysadmins that this stuff can be easy. People look for ways to prove the viewpoint they hold. This can happen unconsciously. They may want to automate, but their own perception of the difficulty can block them, even to the point of convincing their own management that it's not worth trying. And so the industry consensus is that fully automated administration is hard. This consensus must be broken on a per-site basis, at least until we reach a critical mass of automated sites. In my experience, the more tasks you automate, the easier it gets to automate the rest. > Also, if you really *can* automate *everything* then you're not solving > interesting problems, or your problems aren't evolving, or your customers > are very boring and completely non-standard indeed. I find that once you've > conquered/automated a particular problem, there's always one more problem to > solve . . . a wrinkle I hadn't thought of or a new requirement I didn't know > about. You can never fully automate a sufficiently complex system. . . > especially when users are involved ;-) I separate problem-solving and corrective action from each other. Problem solving is an intelligence function. This is the job of humans. Corrective action is a manipulative function. This is the job of code. A systems administrator solves a problem and implements corrective action via a shell prompt, on one or more machines, making the corrective action more or less non-repeatable. An infrastructure architect solves a problem and implements corrective action in repeatable code, which then executes on all applicable machines. Execution is within a control framework (like make) and "applicable" is defined by configuration files within that framework (hosts.conf in isconf version 2i). > Now, this isn't to say you shouldn't automate as much as possible. Fix a > problem once and either solve the problem completely or automate the work > around. Your toolkit should always be expanding. But the tools won't do > the job without proper guidance from the mechanic. ISconf hasn't needed to expand very much over the last 8 years, unless you count the makefile stanzas themselves. It *has* been rewritten a few times though, as I got better at understanding what it does. I jotted some notes on my HP palmtop on the train into NYC in early 1994 and rolled a makefile and 'configure' script out to a few hundred machines globally that same year, thinking "this thing's a hack but it'll help me get a handle on the chaos". That "hack" worked real well compared to everything else I've tried before and since, and it's taken me a long time to figure out exactly why. I think I only really grokked it in full during the last year. It's funny -- I had looked at cfengine in 1994, and almost used it instead. It wasn't until years later, in 1998, that I *tried* to use cfengine as a replacement for isconf and realized that the two solve orthogonal problems and aren't interchangeable with each other. The only major addition I've needed to add so far is some perl networking code to solve the "barrier" problem of synchronizing administrative events on two or more machines at a time, such as when building nodes in an HA cluster. But the core mechanism is still just plain old, simple make. The only innovation there has been to use GNU make, to get conditionals into the makefiles themselves. The Perl rewrite might use Make.pm or a derivative. Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From atp@piskorski.com Sat Apr 20 15:52:10 2002 Received: from tehun.pair.com (tehun.pair.com [209.68.2.71]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id PAA17587 for ; Sat, 20 Apr 2002 15:52:09 -0700 Received: (qmail 3092 invoked by uid 3106); 20 Apr 2002 22:52:20 -0000 Date: Sat, 20 Apr 2002 18:52:20 -0400 From: Andrew Piskorski To: infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration Message-ID: <20020420225219.GA99329@piskorski.com> Mail-Followup-To: infrastructures@TerraLuna.Org References: <20020419181133.A22600@scramjet.TerraLuna.Org> <20020420144434.A6759@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20020420144434.A6759@scramjet.TerraLuna.Org> User-Agent: Mutt/1.3.25i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Sat, Apr 20, 2002 at 02:44:34PM -0700, Steve Traugott wrote: > And so the industry consensus is that fully automated administration > is hard. This consensus must be broken on a per-site basis, at least > until we reach a critical mass of automated sites. A top-notch BOOK on the subject might go a really, really long ways. I can't be the only one who thinks this... Thoughts on this, Steve? -- Andrew Piskorski http://www.piskorski.com From duprec@scorec.rpi.edu Sun Apr 21 14:07:47 2002 Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA01884 for ; Sun, 21 Apr 2002 14:07:46 -0700 Received: from smtp.scorec.rpi.edu (smtp.scorec.rpi.edu [128.113.123.156]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g3LL7iDc532334 for ; Sun, 21 Apr 2002 17:07:49 -0400 Received: from jumpgate.scorec.rpi.edu (jumpgate.scorec.rpi.edu [10.0.1.2]) by smtp.scorec.rpi.edu (8.10.2/8.10.2) with ESMTP id g3LL7Zd17810 for ; Sun, 21 Apr 2002 17:07:38 -0400 (EDT) Date: Sun, 21 Apr 2002 17:07:35 -0400 (EDT) From: Christophe Dupre To: In-Reply-To: <200204211901.MAA01482@roton.terraluna.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Subject: [Infrastructures] Re: The Turing Synthesis and Automated Administration Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Funny how this fits right in with a discussion I was having with a co-worker last week... We've been playing with various automation scripts to maintain our workstations. It starts with jumpstart and kickstart, then every maintenance action is done through scripts at bootup. A bit of a pain as it requires a reboot. I haven't looked at Isconf yet (I've meant to since last year's LISA), but how do other people handle this ? Is anyone doing it through regular cron or other mean ? Anyway, the infrastructure idea is really a no-brainer for situations where you have n (where n can get very large) workstations with a similar configuration. But I'm having a hard time convincing my boss tht spending the time to do the same thing for our one-of-a-kind servers is also worthwhile - after spending X hours to find a little problem, and fixing it, I need to spend more time to implement the fix in the scripts, in case we have to rebuild everything (for disaster recovery). Of course then, the scripts won't actually roll-out the fix, since it has already been manually applied during the discovery phase. And no, we don't have a test lab separate from our production machines - a small University research center is too poor to afford several extra enterprise-class machines. :-) > We have to be using different meanings of the term "automate". > > Automated systems administration is very straightforward. There is > only one way to change the contents of disk or RAM in a running UNIX > machine -- the syscall interface. The task of automated > administration is simply to make sure that each machine's kernel gets > the right system calls, in the right order, to make it be the machine > you want it to be. > > There are no system calls used in the course of administration which > can't be issued from a shell script, a perl script, or (in very rare > cases) a piece of C code. This makes the process of putting > automation in place itself very straightforward. In my case, rather > than typing a command at a shell prompt, I put it into a makefile > stanza. > > If it's a complex series of commands, with conditionals and loops > etc., then it goes into a script, the script goes into CVS, and the > script name goes into the makefile. > > Putting the CVS and makefile framework in place is itself also very > straightforward, as we just proved at Caterpillar earlier this year -- > Luke took 6 hours to go from scratch to a running installation of > isconf, with me looking over his shoulder. Luke is an exceptionally > smart guy, (Hi Luke!) ;-P But I think just about anyone on this > list should be able to get similar results. > > I was there for a week; the first 4 days were spent on political > buy-in, only the last on technology. This is important -- the only > major hurdles I have ever run into for the last few years are > political rather than technological. The major improvement I've had > to make in myself is in learning how to show both management and > sysadmins that this stuff can be easy. > > People look for ways to prove the viewpoint they hold. This can > happen unconsciously. They may want to automate, but their own > perception of the difficulty can block them, even to the point of > convincing their own management that it's not worth trying. > > And so the industry consensus is that fully automated administration > is hard. This consensus must be broken on a per-site basis, at least > until we reach a critical mass of automated sites. > > In my experience, the more tasks you automate, the easier it gets to > automate the rest. > > > Also, if you really *can* automate *everything* then you're not solving > > interesting problems, or your problems aren't evolving, or your customers > > are very boring and completely non-standard indeed. I find that once you've > > conquered/automated a particular problem, there's always one more problem to > > solve . . . a wrinkle I hadn't thought of or a new requirement I didn't know > > about. You can never fully automate a sufficiently complex system. . . > > especially when users are involved ;-) > > I separate problem-solving and corrective action from each other. > > Problem solving is an intelligence function. This is the job of > humans. > > Corrective action is a manipulative function. This is the job of > code. > > A systems administrator solves a problem and implements corrective > action via a shell prompt, on one or more machines, making the > corrective action more or less non-repeatable. > > An infrastructure architect solves a problem and implements corrective > action in repeatable code, which then executes on all applicable > machines. Execution is within a control framework (like make) and > "applicable" is defined by configuration files within that framework > (hosts.conf in isconf version 2i). > > > Now, this isn't to say you shouldn't automate as much as possible. Fix a > > problem once and either solve the problem completely or automate the work > > around. Your toolkit should always be expanding. But the tools won't do > > the job without proper guidance from the mechanic. > > ISconf hasn't needed to expand very much over the last 8 years, unless > you count the makefile stanzas themselves. It *has* been rewritten a > few times though, as I got better at understanding what it does. > > I jotted some notes on my HP palmtop on the train into NYC in early > 1994 and rolled a makefile and 'configure' script out to a few hundred > machines globally that same year, thinking "this thing's a hack but > it'll help me get a handle on the chaos". > > That "hack" worked real well compared to everything else I've tried > before and since, and it's taken me a long time to figure out exactly > why. I think I only really grokked it in full during the last year. > > It's funny -- I had looked at cfengine in 1994, and almost used it > instead. It wasn't until years later, in 1998, that I *tried* to use > cfengine as a replacement for isconf and realized that the two solve > orthogonal problems and aren't interchangeable with each other. > > The only major addition I've needed to add so far is some perl > networking code to solve the "barrier" problem of synchronizing > administrative events on two or more machines at a time, such as when > building nodes in an HA cluster. > > But the core mechanism is still just plain old, simple make. The only > innovation there has been to use GNU make, to get conditionals into > the makefiles themselves. The Perl rewrite might use Make.pm or a > derivative. -- Christophe Dupre System Administrator, Scientific Computation Research Center Rensselaer Polytechnic Institute Troy, NY USA Phone: (518) 276-2578 - Fax: (518) 276-4886 From duprec@scorec.rpi.edu Sun Apr 21 14:09:36 2002 Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id OAA01945 for ; Sun, 21 Apr 2002 14:09:35 -0700 Received: from smtp.scorec.rpi.edu (smtp.scorec.rpi.edu [128.113.123.156]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g3LL9kDc146980 for ; Sun, 21 Apr 2002 17:09:46 -0400 Received: from jumpgate.scorec.rpi.edu (jumpgate.scorec.rpi.edu [10.0.1.2]) by smtp.scorec.rpi.edu (8.10.2/8.10.2) with ESMTP id g3LL9fd17816 for ; Sun, 21 Apr 2002 17:09:41 -0400 (EDT) Date: Sun, 21 Apr 2002 17:09:41 -0400 (EDT) From: Christophe Dupre To: In-Reply-To: <200204201901.MAA27473@roton.terraluna.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Subject: [Infrastructures] Re: leveraging CVS Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Technically, CVS is CVS and so you could use your already existing CVS server. However, I like to keep all my infrastructure services on one well-backed up server, so I did set up a separate server from the developpers'. > In my current environment, I'm slowly working on implementing an > infrastructure, based on the methodologies on infrastructures.org. > > Looking at the CVS requirement, I'm told that the developers already > have a CVS server in place (and have had for some time). > > Should I take advantage of this, or should I set up my own > infrastructures-only CVS server? -- Christophe Dupre System Administrator, Scientific Computation Research Center Rensselaer Polytechnic Institute Troy, NY USA Phone: (518) 276-2578 - Fax: (518) 276-4886 From cavin@mit.edu Mon Apr 22 15:48:47 2002 Received: from lap1-wccf.mit.edu (LAP1-WCCF.MIT.EDU [18.42.2.105]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id PAA25909 for ; Mon, 22 Apr 2002 15:48:44 -0700 Received: (from cavin@localhost) by lap1-wccf.mit.edu (8.9.3) id SAA03651; Mon, 22 Apr 2002 18:48:55 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15556.37718.823408.23363@lap1-wccf.mit.edu> Date: Mon, 22 Apr 2002 18:48:54 -0400 From: Tom Cavin To: infrastructures@TerraLuna.Org Subject: [Infrastructures] Re: The Turing Synthesis and Automated Administration In-Reply-To: References: <200204211901.MAA01482@roton.terraluna.org> X-Mailer: VM 6.99 under Emacs 20.7.1 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi All, I've been following this discussion (and reading the papers from LISA XV) and I think I have a different perspective that may prove useful in these discussions. Steve and Joel's original infrastructures paper (LISA XII) deals with trading room systems in an environment where the systems and needs are both well defined and centrally controlled. At the other extreme, Mark Burgess is in an academic environment where the systems and needs are not well defined or are fairly volitile and where central control is very limited. These two different environments have lead to two different approaches to infrastructures. In a financial trading infrastructure, the users are mostly traders running very specific software to make their trades and view or forcast market conditions. There are probably also a few managers who would use a different set of programs, and then there are the central servers and various specialty systems. In this situation, reliability is absolutely required. Users are not allowed to add their own software to the system, only the sys-admins have the root password (or equivalent), and all changes get beta-tested on an isolated system before being deployed via the central infrastructure. All equipment is owned by the company. People who might try to make changes to the system without approval are likely to be reprimanded, fired, or prosecuted depending on who they are and what they tried to do. In an academic environment, the users are mostly students running software for classes, playing games, sending messages, playing music, surfing the web, downloading and writing viruses, etc. There are also faculty members, post-docs, administrators, and other staff doing their work -- usually in a slightly more relaxed environment than the corporate equivalent. And there are also the same sort of central servers and various specialty systems. In this situation, reliability would be a nice change, and flexibility is the major goal. Users are continually adding their own software to the system, any number of people have root access (or equivalent) to one or more systems, and many changes go live with no testing at all. Most of the central equipment is owned by the institution, but individual workstations can be owned by anyone, and fiscal ownership does not equate to practical control. People who make changes to the system without approval tend to be ignored unless they both cause problems for other users and are identified, and even though changes could lead to reprimands, firing/expulsion, or prosecution, there is also the possibility of a research paper or a thesis on system vulnerabilites or network instabilities. Needless to say, the tools needed to manage these two cases are likely to be rather different. While both tools are designed to manage the infrastructure, the goal of Isconf is to take systems from one known state to another in an orderly fashion, and the goal of Cfengine is to get a system to a partially known state from an unknown state. It is also the case that much of the published research since Steve and Joel's LISA XII paper has been done in the academic world. I suspect that once they read the paper, most of the sys-admins in the corporate world would be able to apply it more or less as laid out in the paper and there wouldn't be much to do in the way of research beyond adding details. The academic world really is different from the corporate world, even if both use the same type of systems. So these new papers are going to be trying to solve different aspects of the problem. The whole concept of an infrastructure is a really sweet idea, and that first paper was a real eye-opener for many folks, especially the academic sys-admins and CS faculty. The fact that there is a real, practical solution to the problems of maintaining large groups of systems in a scalable manner in one environment seems to have acted as a challenge to people to extend this solution to other environments. No one in academia seems to have looked at the problem since MIT's Project Athena (which gave us X windows, Hesiod - a precursor to LDAP, Zephyr - a precursor to instant messaging, simple unattended installation, and automatic unattended updates of systems) in the late 1980s and early 1990s. Athena, though still used at MIT, never caught on elsewhere. I think the reason for this is that there is a steep learning curve to get everything running all at once. The infrastructures paper introduced something that is less ambitious than Athena, and more comprehensible. I'm not quite sure what my point is in all this except to say that there is still much work to do in order to get the necessary infrastructures in place, and that different environments will probably need different tools to deploy and manage those infrastructures. First principles apply here: you need to know what you want to do before you choose or build the tools to do it. Cfengine, Isconf, and Maelstrom (and I'm sure there are other tools out there) all focus on different aspects of the problem of managing systems and sites. As an infrastructure architect, you need to know your options. It's not so much picking the best tool as picking the tool that best helps you in your particular situation. --Tom -- Tom Cavin Phone: (617) 258 - 7806 Computer Operations Manager Email: cavin@mit.edu MIT - Whitaker College Computer Facility or tec@ai.mit.edu From stevegt@roton.terraluna.org Tue Apr 23 20:15:16 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id UAA11727; Tue, 23 Apr 2002 20:15:16 -0700 Date: Tue, 23 Apr 2002 20:15:14 -0700 From: Steve Traugott To: Tom Cavin Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: The Turing Synthesis and Automated Administration Message-ID: <20020423201514.A23959@scramjet.TerraLuna.Org> References: <200204211901.MAA01482@roton.terraluna.org> <15556.37718.823408.23363@lap1-wccf.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <15556.37718.823408.23363@lap1-wccf.mit.edu>; from tec@ai.mit.edu on Mon, Apr 22, 2002 at 06:48:54PM -0400 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, Apr 22, 2002 at 06:48:54PM -0400, Tom Cavin wrote: > I've been following this discussion (and reading the papers from LISA XV) > and I think I have a different perspective that may prove useful in these > discussions. Steve and Joel's original infrastructures paper (LISA XII) > deals with trading room systems in an environment where the systems and > needs are both well defined and centrally controlled. At the other > extreme, Mark Burgess is in an academic environment where the systems and > needs are not well defined or are fairly volitile and where central control > is very limited. > > These two different environments have lead to two different approaches to > infrastructures. Exactly, Tom. The job of academic environments is to teach and do research. But academic computing environments rarely try to intentionally replicate the operating procedures needed for mission-critical business reliability. A student's time isn't considered as valuable as, say, a trader's. So student administrators are inadvertently taught to treat users poorly, and to create and administer ad-hoc systems with lower reliability and higher labor requirements than what a business would normally expect. Academic-based research focuses on fixing the inherent security and reliability problems of these same ad-hoc, indeterminate environments. As a result, university research efforts tend to try to solve the problems of university computing, not those of mission-critical business environments. This extends beyond universities, to government research labs, due to their academic roots. My favorite recent example is when a local lab performed extensive planning and analysis for the upgrade of a heavily-used login server. This planning incorporated the idea of a three-day shutdown during the upgrade. Horrified on general principle, I proposed some alternatives, but didn't expect to make a dent. The extended outage took place as planned. This sort of upgrade path would be unacceptable in a mission-critical business environment. The interns in that lab learn very bad habits. These habits will cut into their quality of life -- they simply aren't gaining the tools they need to be more effective. And so the systems administrators produced by our educational institutions must be retaught when they land in a business environment. I've long recognized that the "best" administrators often have no formal C.S. training at all -- they weren't formally taught ad-hoc habits. There is no strong feedback loop in place to correct this mismatch between education and workplace. For there to be one, academic computing would need to be more open to those of us on the receiving end of their efforts. A common response I get when I mention mission-critical reliability is "we can't afford that", as if I were talking about *spending* money. I'm talking about saving it. A well-managed computer costs a certain amount of money, and gives you a certain return on that investment. If you buy the same computer, and then manage it badly, it will cost you more in labor and provide a lower ROI. Which is cheaper? Sure, expensive hardware solutions like HA and SAN will help you squeeze out a few more hours of uptime per year. But in most cases, procedural changes will avoid most outages. > Needless to say, the tools needed to manage these two cases are likely to > be rather different. While both tools are designed to manage the > infrastructure, the goal of Isconf is to take systems from one known state > to another in an orderly fashion, and the goal of Cfengine is to get a > system to a partially known state from an unknown state. Yes. And "partially known" is not a comfortable foundation for business operations. There's enough uncertainty in venture financing, product development, marketing, and staffing -- why add to it by messing with the reliability and predictability of the underlying infrastructure? Years after launch is way too late to try to get a grip on a business -- this is true whether we're talking about infrastructure controls or financial controls. > It is also the case that much of the published research since Steve and > Joel's LISA XII paper has been done in the academic world. I suspect it will continue to be, until we ourselves do something about it. The fact is, most professionals just don't have the time or energy to write down what they're doing, and most employers will neither give permission to publish, nor pay the author to do it on company time. This heavily weights the ratio of academic versus professional authors who are able to publish at LISA, for instance. > The whole concept of an infrastructure is a really sweet idea, and that > first paper was a real eye-opener for many folks, especially the academic > sys-admins and CS faculty. The fact that there is a real, practical > solution to the problems of maintaining large groups of systems in a > scalable manner in one environment seems to have acted as a challenge to > people to extend this solution to other environments. No one in academia > seems to have looked at the problem since MIT's Project Athena (which gave > us X windows, Hesiod - a precursor to LDAP, Zephyr - a precursor to instant > messaging, simple unattended installation, and automatic unattended updates > of systems) in the late 1980s and early 1990s. Athena, though still used > at MIT, never caught on elsewhere. I think the reason for this is that > there is a steep learning curve to get everything running all at once. The > infrastructures paper introduced something that is less ambitious than > Athena, and more comprehensible. It's interesting you should mention Athena -- it was one of my own inspirations for the work we described in that paper. I agree; Athena was a wonderful exception. In our own lab we played with parts of Athena, like Hesiod or Zephyr, or of CMU's work, like SUP or AFS. We knew we couldn't roll most of these out to production, due to conflicting legacy and/or user requirements. SUP is the only one of these which actually made it into production, and we used it heavily (it's still a better technical fit than anon rsync.) We were intentionally making the time to do masters-equivalent research, exploring existing art and figuring out why it worked and why it didn't, and then building on what we'd learned. We *made* the opportunity to do this; on the org chart were were just a small admin group. It meant a lot of late nights. For several months, I carried a whole bundle of past LISA papers around, reading them on the train. The whole goal was to implement fully automated administration using open standards with as little complexity as possible. In hindsight, I think that we were lucky to hit on the makefile thing early on -- and I probably wouldn't have tried that if I hadn't just left USL. The UNIX source code makes extensive use of VPATH in makefiles, to handle different hardware platforms. I thought we could do something similar at the admin layer. That never worked out, but the makefiles themselves stayed and grew. * * * I think there's a larger story here, something more fundamental missing from systems administrator education: a business foundation. In today's businesses, the systems administrators daily make decisions which have a fundamental impact on the ability of the business to function. As clerical jobs have been automated and migrated into systems, systems administrators have become defacto business administrators, making sure that business information flows from source to sink. They usually do this in isolation from the executive team, and often don't have the background or authority to incorporate business requirements into these decisions. This translates into less effective businesses and limited career paths for IT professionals -- a CIO rarely makes CEO. Again, fixing this for the whole industry is going to require fundamental academic change, starting way down at the undergrad level. Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From stevegt@roton.terraluna.org Tue Apr 23 22:41:36 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id WAA25254; Tue, 23 Apr 2002 22:41:36 -0700 Date: Tue, 23 Apr 2002 22:41:35 -0700 From: Steve Traugott To: Tom Cavin Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: The Turing Synthesis and Automated Administration Message-ID: <20020423224135.B23959@scramjet.TerraLuna.Org> References: <200204211901.MAA01482@roton.terraluna.org> <15556.37718.823408.23363@lap1-wccf.mit.edu> <20020423201514.A23959@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020423201514.A23959@scramjet.TerraLuna.Org>; from stevegt@TerraLuna.Org on Tue, Apr 23, 2002 at 08:15:14PM -0700 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Tue, Apr 23, 2002 at 08:15:14PM -0700, Steve Traugott wrote: > On Mon, Apr 22, 2002 at 06:48:54PM -0400, Tom Cavin wrote: > > I've been following this discussion (and reading the papers from LISA XV) > > and I think I have a different perspective that may prove useful in these > > discussions. Steve and Joel's original infrastructures paper (LISA XII) > > deals with trading room systems in an environment where the systems and > > needs are both well defined and centrally controlled. At the other > > extreme, Mark Burgess is in an academic environment where the systems and > > needs are not well defined or are fairly volitile and where central control > > is very limited. > > > > These two different environments have lead to two different approaches to > > infrastructures. > > Exactly, Tom. > > The job of academic environments is to teach and do research. But > academic computing environments rarely try to intentionally replicate > the operating procedures needed for mission-critical business > reliability. Just came back from dinner. Joyce looked at this previous message and said that she thinks it's not too bad, but that a lot of people are still going to disagree with me. I admit that a lot of it is worded strongly. Let me try again: - Tom's original, very-well-written point was that business and academia are very different environments and, as such, need to be managed differently, with different toolsets and procedures. I agree. - But after reading Tom's message, I had to ask myself "Why? Why are they different?" - Should they be the same? Should business environments remain ad-hoc, like academia, the way they have been for years? Or should academic computing catch up to what we've started doing in business, and become more structured, to better represent the environment most students will graduate into? - Or should the two remain different, in order to each serve their own needs more completely? If so, how do we educate systems administrators for business environments? Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From duprec@scorec.rpi.edu Wed Apr 24 07:57:01 2002 Received: from mail.rpi.edu (mail.rpi.edu [128.113.22.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA28136 for ; Wed, 24 Apr 2002 07:57:00 -0700 Received: from smtp.scorec.rpi.edu (smtp.scorec.rpi.edu [128.113.123.156]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id g3OEvChQ158732 for ; Wed, 24 Apr 2002 10:57:12 -0400 Received: from mastermind.scorec.rpi.edu (mastermind.scorec.rpi.edu [10.0.5.17]) by smtp.scorec.rpi.edu (8.10.2/8.10.2) with ESMTP id g3OEv3d29084 for ; Wed, 24 Apr 2002 10:57:07 -0400 (EDT) Date: Wed, 24 Apr 2002 10:57:02 -0400 (EDT) From: Christophe Dupre To: In-Reply-To: <200204231901.MAA12850@roton.terraluna.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Subject: [Infrastructures] Re: The Turing Synthesis and Automated Administration Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Tue, 23 Apr 2002 infrastructures-request@TerraLuna.Org wrote: > In an academic environment, the users are mostly students running software > for classes, playing games, sending messages, playing music, surfing the > web, downloading and writing viruses, etc. There are also faculty members, > post-docs, administrators, and other staff doing their work -- usually in a > slightly more relaxed environment than the corporate equivalent. And there > are also the same sort of central servers and various specialty systems. > In this situation, reliability would be a nice change, and flexibility is > the major goal. Users are continually adding their own software to the > system, any number of people have root access (or equivalent) to one or > more systems, and many changes go live with no testing at all. Most of the > central equipment is owned by the institution, but individual workstations > can be owned by anyone, and fiscal ownership does not equate to practical > control. People who make changes to the system without approval tend to be > ignored unless they both cause problems for other users and are identified, > and even though changes could lead to reprimands, firing/expulsion, or > prosecution, there is also the possibility of a research paper or a thesis > on system vulnerabilites or network instabilities. I don't entirely agree with you on this. I work for a university, and from experience what you are describing is typical of a Computer Science department. There people will do research on the O/S itself, and need to crash the machines on a regular basis. Even then, such research is usually limited to a well-identified subset of machines. Almost all other departments and research centers use computers as a mere tool to speed-up computations. There computers are required to be reliable, as it often take several days on computation to get a single result. Having an uncontrolled environment lead to wasted time and money, and lots of frustration. I would say that 80% of the computers in a research environment have the same requirements as for a commercial environment - with the big difference that downtime doesn't cost as much, and so a lot less money is spent to have a high uptime: no generator, UPS for a small subset of machines, minimal test environment with students as guinea pigs, etc... Of course, we don't fire students for security breaches, they just get to have a serious talk with the dean. Now, I'm not saying that you have an invalid point, I just want people not to have the misconception that research environment are total chaos. :-) -- Christophe Dupre System Administrator, Scientific Computation Research Center Rensselaer Polytechnic Institute Troy, NY USA Phone: (518) 276-2578 - Fax: (518) 276-4886 From docx@io.com Wed Apr 24 10:03:18 2002 Received: from hiram.io.com (root@hiram.io.com [199.170.88.27]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA30625 for ; Wed, 24 Apr 2002 10:03:17 -0700 Received: from eris.io.com (IDENT:root@eris.io.com [199.170.88.11]) by hiram.io.com (8.11.2/8.11.2) with ESMTP id g3OH3JE27512; Wed, 24 Apr 2002 12:03:19 -0500 Received: from localhost (docx@localhost) by eris.io.com (8.11.6/8.11.6) with ESMTP id g3OH3Pd06246; Wed, 24 Apr 2002 12:03:25 -0500 X-Authentication-Warning: eris.io.com: docx owned process doing -bs Date: Wed, 24 Apr 2002 12:03:25 -0500 (CDT) From: Dylan Northrup To: Christophe Dupre cc: infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] Re: The Turing Synthesis and Automated Administration In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: An infinite number of monkeys in the guise of Christophe Dupre wrote: :=I don't entirely agree with you on this. I work for a university, and from :=experience what you are describing is typical of a Computer Science :=department. There people will do research on the O/S itself, and need to :=crash the machines on a regular basis. Even then, such research is usually :=limited to a well-identified subset of machines. Almost all other :=departments and research centers use computers as a mere tool to speed-up :=computations. There computers are required to be reliable, as it often :=take several days on computation to get a single result. Having an :=uncontrolled environment lead to wasted time and money, and lots of :=frustration. I would say that 80% of the computers in a research :=environment have the same requirements as for a commercial environment - :=with the big difference that downtime doesn't cost as much, and so a lot :=less money is spent to have a high uptime: no generator, UPS for a small :=subset of machines, minimal test environment with students as guinea pigs, :=etc... Having been on both sides of the fence, I agree that there are requirements for uptime in an academic environment (especially if the box you're working on holds the dean's e-mail!). But the primary issue is the "minimal test environment" that isn't available for most major changes. When the costs of downtime are low, the money spent to prevent downtime is also low. However, this doesn't invalidate the argument that most issues that can and do cause downtime are software/architecture problems and not hardware problems. Having a load balancer in front of multiple servers with back end DBs attached to RAID arrays is a good way to protect against hardware failure. But, it's very expensive and still won't give you good reliability unless you have a solid software architecture for your services. . . and building a solid software architecture/infrastructure (for the most part) just takes time and effort. :=Now, I'm not saying that you have an invalid point, I just want people not :=to have the misconception that research environment are total chaos. :-) You haven't seen some of the research labs I've seen ;-) -- Dylan Northrup <*> docx@io.com <*> http://www.io.com/~docx/ "Easy to bitch, easy to whine, easy to moan, easy to cry, easy to feel like there ain't nothing in your life. Harder to work, harder to strive, hard to be glad to be alive, but it's really worth it if you give it a try." -- Cowboy Mouth, 'Easy' From jblaine@mitre.org Wed Apr 24 10:45:17 2002 Received: from smtpproxy1.mitre.org (smtpproxy1.mitre.org [129.83.20.90]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA19532 for ; Wed, 24 Apr 2002 10:45:16 -0700 Received: from avsrv1.mitre.org (avsrv1.mitre.org [129.83.20.58]) by smtpproxy1.mitre.org (8.11.3/8.11.3) with ESMTP id g3OHj0i06688; Wed, 24 Apr 2002 13:45:00 -0400 (EDT) Received: from linus.mitre.org (linus.mitre.org [129.83.10.1]) by smtpsrv1.mitre.org (8.11.3/8.11.3) with ESMTP id g3OHiuh24610; Wed, 24 Apr 2002 13:44:58 -0400 (EDT) Received: from jblaine-pc.MITRE.ORG (jblaine-pc [129.83.10.132]) by linus.mitre.org (8.9.3/8.9.3) with ESMTP id NAA16163; Wed, 24 Apr 2002 13:44:56 -0400 (EDT) Date: Wed, 24 Apr 2002 13:44:56 -0400 From: Jeff Blaine To: Christophe Dupre cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: The Turing Synthesis and Automated Administration Message-ID: <1748647369.1019655896@jblaine-pc.MITRE.ORG> In-Reply-To: X-Mailer: Mulberry/2.0.8 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > Funny how this fits right in with a discussion I was having with a > co-worker last week... We've been playing with various automation scripts > to maintain our workstations. It starts with jumpstart and kickstart, then > every maintenance action is done through scripts at bootup. A bit of a > pain as it requires a reboot. I haven't looked at Isconf yet (I've meant > to since last year's LISA), but how do other people handle this ? Is > anyone doing it through regular cron or other mean ? When we load a box, our final "installation automation" runs cfengine. Via our cfengine config file, cfengine puts itself in the machine's root crontab (it's a call to a wrapper script that staggers the start time of the actual process by sleeping a "random" number of seconds. FWIW, our process currently looks like this for Solaris boxes (and is identical in logic, but not specifics, to our Linux kickstarts *grumble*): 1. add_install_client 2. boot net - install 3. OS loads 4. finish script uses Casper Dik's auto-install package to install the AFS client and a few other files needed to get the machine "happy enough" (consider it bootstrap phase 2) 5. Machine reboots. 6. As part of the finish script /etc/rc3.d/S99rcfify was put in place ;) At this point, it runs, patches the machine to our specs, and runs cfengine to get our real core site-isms into place, including a cron job to run itself early in the morning. 7. Machine reboots. 8. Every night skew is corrected via cfengine via cron. Our cfengine setup works (to our standards) because of 3 things: 1. AFS space is available on every machine after the 1st reboot at install-time. 2. Our netgroups drive 90% of our cfengine logic (yes, *gasp* we use NIS, like OMG!!! How 1995 and uncool!!! :P) 3. Our entire repository of site-isms exists in AFS space. This setup has worked without fail for 4 years now. Jeff Blaine From hakanson@cse.ogi.edu Wed Apr 24 11:09:57 2002 Received: from church.cse.ogi.edu (root@cse.ogi.edu [129.95.20.2]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id LAA32577 for ; Wed, 24 Apr 2002 11:09:56 -0700 Received: from cse.ogi.edu (hepburn.cse.ogi.edu [129.95.40.57]) by church.cse.ogi.edu (8.9.3/8.9.3) with ESMTP id LAA10425 for ; Wed, 24 Apr 2002 11:10:08 -0700 (PDT) Message-Id: <200204241810.LAA10425@church.cse.ogi.edu> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] Re: The Turing Synthesis and Automated Administration In-reply-to: Christophe Dupre's message of "Wed, 24 Apr 2002 10:57:02 EDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 24 Apr 2002 11:10:08 -0700 From: Marion Hakanson Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Folks, I find myself partially in agreement with Steve's assessment of the typical academic computing environment, and I'd like to add my views to the discussion. I've spent pretty much my entire system administration career working for one of two university Computer Science departments (starting about 1984). One was at a public (state) university, and the other (my current position) has been at a private university. The sample size is very small, but the amount of resources devoted to computer support, and the understanding of its importance, were quite different, so I think it's fair to say that different academic environments are, well, different. One used primarily student labor, and the small professional staff spent most of its time training students (high turnover). The other uses only professional staff. The cultures are quite disparate. Another comment: I earned a B.S. degree in Computer Science, and worked a couple of years towards a graduate degree. Yet, contrary to Steve's notion that such university training leads one away from the Infrastructures ideal, I find that I and many (not _all_!) of my colleagues have all along done as much automation as we could manage. Some of my co-workers over the years have chafed at the hoops they had to jump through to make changes, but those with engineering backgrounds seemed to understand the necessity. Our early efforts were along the lines of cloning hosts, using a single master machine (partly scripted installs, and partly hand-crafted, but systematically so) which was replicated using a combination of Make, m4, scripts, rdist, and NFS-shared filesystems. We did this for a few reasons: First, limited staff to manage a lot of machines (and a lot of different platforms -- as many as 11 different versions of Unix at one time). Second, mechanical replication made all machines the same for our users, giving a more stable environment for research and education. Third, I personally find doing the same danged task by hand again and again to be so tedious that I'll go to a tremendous amount of effort to avoid it -- sure, it was fun when I was young and energetic, but life is too short. Yes, we have our researchers who need to crash and burn their kernels, other system software, or even their network switch. They have their own self-managed systems for those purposes. And they don't use those boxes to read their email or write up their papers. But we also support researchers who need fairly high availability of their compute engines (and their underlying file services) in order to meet contract deadlines with their funding agencies. If they don't meet the deadlines, they're out of business. The tricky balancing act that we have here which is probably not as extreme in business nor even non-computer-science academic areas is the tension between providing rigidly and centrally controlled network services without interfering with the flexibility and rapid response required by our researchers, all without burning up too much money. Yes, my job is way less stressful than supporting a trading room floor, or a chip-manufacturing floor, or an e-business farm -- I'm not cut out for that level of intensity. But we still need cheap and reliable system administration. Personally, I found the original Infrastructures paper resulted in a big "Yes, finally!" And I have not been able to characterize why, but my years of experience have given me some discomfort at the cfengine approach of continually monitoring, tweaking, and re-tweaking things as necessary. My gut just tells me that a system constantly changing itself without some human approval mechanism is going to lead to a less reliable system, not a more reliable one. Perhaps Steve is onto something when he says that "order is important" -- in my experience, it's _repeatability_ that is important: Can I tear down this machine and make it back exactly the same as it was before? Or, when Box-A dies, can I make Box-B do what Box-A was doing while awaiting repairs? It turns out that I _can_ use cfengine in many ways to achieve my goals, but it takes a little effort to do so -- it's not as "natural" in cfengine as I'd like it to be. Another tool which may prove promising someday is Alva Couch's idea of using a Prolog back-end to examine a system's state and then go about finding a way to make it match a high-level description of its intended state (http://www.eecs.tufts.edu/~couch/ongoing.html). For the moment, I don't much care which tools I use to do the job, as long as they don't end up being more tedious to use than I can bear (:-). OK, that's enough of my babbling. Keep up the good work, folks.... Regards, -- Marion Hakanson CSE Computing Facilities From partain@dcs.gla.ac.uk Thu Apr 25 03:46:09 2002 Received: from slicker.dcs.gla.ac.uk (slicker.dcs.gla.ac.uk [130.209.242.51]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id DAA29585 for ; Thu, 25 Apr 2002 03:46:07 -0700 Received: (from partain@localhost) by slicker.dcs.gla.ac.uk (8.8.8+Sun/8.8.8) id LAA21499; Thu, 25 Apr 2002 11:46:21 +0100 (BST) X-Authentication-Warning: slicker.dcs.gla.ac.uk: partain set sender to partain@dcs.gla.ac.uk using -f To: infrastructures@TerraLuna.Org References: <200204211901.MAA01482@roton.terraluna.org> <15556.37718.823408.23363@lap1-wccf.mit.edu> From: Will Partain Date: 25 Apr 2002 11:46:21 +0100 In-Reply-To: <15556.37718.823408.23363@lap1-wccf.mit.edu> Message-ID: Lines: 15 User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.4 (Common Lisp) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: [Infrastructures] academic vs industrial (was: Re: The Turing Synthesis ...) Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Tom Cavin and others have been commenting (very sensibly) on the "two different environments" (academic and corporate) that "lead to two different approaches to infrastructures". I would suggest that it's more "spectrum" than "dichotomy". For example, there are "corporate" environments, often innovation-driven/led, where there's lots of "churn" of new software packages and tools, and perhaps hardware, too. *Way* more free-wheeling than what Steve has traditionally worried about, but still managed in some sensible way. Will Partain (Arusha Project) From stevegt@roton.terraluna.org Thu Apr 25 11:08:30 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id LAA05335; Thu, 25 Apr 2002 11:08:30 -0700 Date: Thu, 25 Apr 2002 11:08:29 -0700 From: Steve Traugott To: "Carroll, Jim" Cc: "Infrastructures (E-mail)" Subject: Re: [Infrastructures] authentication & client file access Message-ID: <20020425110829.A29662@scramjet.TerraLuna.Org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from jcarro10@sprintspectrum.com on Tue, Apr 16, 2002 at 06:28:01PM -0500 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Tue, Apr 16, 2002 at 06:28:01PM -0500, Carroll, Jim wrote: > I know the Infrastructures web site espouses the use of NIS over all other > options. After reading about the security advantages of NIS+ over NIS, as > well as being urged to implement NIS+, I've set up a master and replica NIS+ > server. I had to get the NIS+ source code for Linux (as well as the > appropriate PAM module) and get that working. So far, everything (including > automounter) seems to be working well, with the possible question > surrounding why changing one's password on a Linux client seems to fail to > update the credentials. (Sometimes it works, usually it doesn't. Works > fine on the Solaris side.) Hi Jim, Horrors! This is the problem with writing technical things down -- the words go obsolete. I've just modified all mentions of NIS on Infrastructures.Org, to now also mention LDAP. Nutshell: LDAP has been proven to be a better alternative than NIS, much more flexible, and is usable even on hosts which don't explicitly support it, by using Perl scripts as glue. When I was at Netscape Corp. in 2000 I only had one password at a time, across all platforms -- UNIX, NT, web auth, etc. To change your password, you just type 'passwd' in any web browser's Location: box, and DNS is set up to take you to a cgi which lets you change your password in LDAP... As far as NIS+ goes, I think it's horrible, and the web site recommends against it -- I just made that wording stonger. As of 2000, Sun still didn't use NIS+ for their own corporate maps, due to reliability problems. They were still using NIS then, and I'd expect them to slowly migrate to LDAP next, due to iPlanet. Anyone know current status? > If anyone could give me any warm fuzzies with regards to NIS not being the > security concern that some make it out to be, let me know. Otherwise, it > looks like I'll have to make this work with NIS+. NIS+ is a greater security concern in terms of denial of service -- it's too easy to corrupt. If you enforce good passwords in the first place, then neither NIS nor LDAP are security concerns in any practical sense. > Having said that, it was brought to my attention today that some users may > eventually be running Linux on a laptop. Naturally this introduces the > problem of being able to authenticate while disconnected from the network. > I recall that the Infrastructures site suggests making every client a slave > (with the added overhead associated with pushing the maps, and BTW, what > happens if you try to push those maps when half the machines (laptops) are > disconnected for the night?), but I'm not particularly smitten with that > approach. Pull, never push. > First problem is, how to deal with the need for disconnected computing? > Assuming the problem of user authentication has been dealt with, it would be > great if the user automounted their home directory. Great for enabling a > user to login to any workstation, having their desktop available, anywhere. > But disconnect that laptop, and... limbo. > I have a cursory awareness of CodaFS. Great for Linux, but I'm not sure how > stable it is on the Solaris side. Has anyone used InterMezzo yet? > I can't help but wonder if I'm simply looking at the wrong approach/tools to > solve the problem. Perhaps something like rsync for keeping home > directories up to date...? Ouch! > Maybe NIS slaves on every workstation *is* the answer...? Yes, if you're using NIS. Worked for thousands of mission-critical trader workstations for years. Requires some intelligent watchdog code -- see http://www.terraluna.org/dgp/cvsweb/isconf2i/bin/nis_watchdog?rlog. > Maybe CodaFS is the answer for home directories, providing we > set those users up with a Linux-based CodaFS server? Does anyone have any > feel on how "ready for prime time" CodaFS is? Likely never -- InterMezzo is a retry, by one of the Coda developers. Again, words go stale. Fixed on the site. > Side question: The development team already has a CVS server in place. > Would it be best to just take advantage of this server for infrastructure > management purposes, or is there a reasonable reason why we should set up > our own? Every time I've hit this question we've ended up setting up our own, if for no other reason than security -- it's easier to keep application developers from mucking about in the infrastructure code, even accidentally. The drawback of a separate server or repository is when you want to share infrastructure documentation with those same developers or management. Steve -- Stephen G. Traugott UNIX/Linux Infrastructure Architect, TerraLuna LLC stevegt@TerraLuna.Org http://www.stevegt.com From cavin@mit.edu Thu Apr 25 16:02:09 2002 Received: from lap1-wccf.mit.edu (LAP1-WCCF.MIT.EDU [18.42.2.105]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id QAA27211 for ; Thu, 25 Apr 2002 16:01:57 -0700 Received: (from cavin@localhost) by lap1-wccf.mit.edu (8.9.3) id TAA02755; Thu, 25 Apr 2002 19:02:04 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15560.35564.334293.262701@lap1-wccf.mit.edu> Date: Thu, 25 Apr 2002 19:02:04 -0400 From: Tom Cavin To: Will Partain Cc: infrastructures@TerraLuna.Org Subject: [Infrastructures] academic vs industrial (was: Re: The Turing Synthesis ...) In-Reply-To: References: <200204211901.MAA01482@roton.terraluna.org> <15556.37718.823408.23363@lap1-wccf.mit.edu> X-Mailer: VM 6.99 under Emacs 20.7.1 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi Will, While I did comment on two different environments, I'd like to say that I agree with you that these two environments are extremes at either end of a spectrum. (Actually, I'm not quite sure if the range is limited to a 1 dimensional variance...it's probably at least two or three dimensions including allowable downtime, allowable customization, size/importance of a service's client base, etc.) Your average site is going to be some combination of things that have to work all the time, most of the time, and some of the time. Since I'm at MIT, I have access to the Athena environment and use it constantly. I'm not involved with the development or maintenance of the core systems, but I do work on adapting the basic platform for the needs of the specific labs and centers that I support. What this gives me is a stable and reliable platform that provides such nice things as automatic updates for security patches, Kerberos, a huge amount of tools, utilities and programs in AFS space, and hooks to do local customizations for those labs and users that need specialty items such as programs for functional magnetic resonance imaging (fMRI). So I have a solid foundation to build on, and I only need to pay attention to the areas where my limited, custom lab tools intersect the basic Athena foundation. (That's the ideal, anyway. In reality I have to cope with visitors who bring their own systems into the network and want access to everything without having me rebuild their custom systems, professors who have done things just-this-way for the last 20 years and don't see why they can't continue to do so for the next 20 years, and all the fun stuff like an overhead computer room cooling unit that just decided to start dripping over a key file server...) Athena had all this wonderful stuff ten years ago when the Athena Project finished basic development and was turned over to Athena Operations. I personally believe that the world (or at least the internet) would have been vastly different if the Athena folks had done a little more in the way of either making the system easy to run elsewhere or setting up some sort of "Athena franchise" in order to leverage all the fine infrastructure of their system in other locations. It really doesn't make sense to reinvent the wheel everywhere, and that's what's been happening. If a company could have easily and inexpensively "subscribed" to Athena infrastructure services back then, and if the program had been properly advertised and supported by even a minimal bit of marketing, I truly doubt that Microsoft would have become what it is today. Then again, there were a lot of other things going on at the time and this is all speculative. What is more to the point, however, is that something like this could come out of the Arusha Project. If we can pull together a basic infrastructure like Steve's, and couple it with add-on packages developed by individual domain specialists via Arusha, we could have a global system that would allow an infrastructure novice to get basic core functions up easily, and then pick and choose from a menu of authenticated specialty systems that work within the basic infrastructure to supply just what the local users need. If the Arusha packages are designed with integration standards that include dependencies in terms of required and provided services and documented APIs (instead of specific package dependencies as in RPM and other similar tools) they become easier to isolate for verification testing and maintenance. If there are no hidden dependencies, you could conceivably have individual systems query a central database to find out how they should configure themselves at boot time (which is already possible) and then have a normal user login and say "please add specialty service X to this workstation" and have the system add those services to the current mix. (Access to programs is already possibly in this way via remote file systems, but getting reliable services started for special packages is another degree of difficulty.) I'm starting to wander here... the vision is nice, but the devil (devel?) is in the details. In any case I'm waiting to see how the Arusha Project develops. From what I read in the LISA XV proceedings, there are lots of possibilities... --Tom Will Partain writes: > Tom Cavin and others have been commenting > (very sensibly) on the "two different environments" > (academic and corporate) that "lead to two different > approaches to infrastructures". > > I would suggest that it's more "spectrum" than "dichotomy". > For example, there are "corporate" environments, often > innovation-driven/led, where there's lots of "churn" of new > software packages and tools, and perhaps hardware, too. > *Way* more free-wheeling than what Steve has traditionally > worried about, but still managed in some sensible way. > > Will Partain > (Arusha Project) -- Tom Cavin Phone: (617) 258 - 7806 Computer Operations Manager Email: cavin@mit.edu MIT - Whitaker College Computer Facility or tec@ai.mit.edu From jrstear@sandia.gov Thu Apr 25 16:15:19 2002 Received: from mm02snlnto.sandia.gov (mm02snlnto.sandia.gov [132.175.109.21]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id QAA02519; Thu, 25 Apr 2002 16:15:19 -0700 Received: from 132.175.109.1 by mm02snlnto.sandia.gov with ESMTP ( Tumbleweed MMS SMTP Relay (MMS v4.7)); Thu, 25 Apr 2002 17:14:55 -0600 X-Server-Uuid: 95b8ca9b-fe4b-44f7-8977-a6cb2d3025ff Received: from mercy.sandia.gov (mercy.sandia.gov [134.253.242.218]) by sass165.sandia.gov (8.12.3/8.12.3) with ESMTP id g3PNFXP7015748; Thu, 25 Apr 2002 17:15:33 -0600 (MDT) Received: from jrstear by mercy.sandia.gov with local (Exim 3.33 #1 ( Debian)) id 170sSZ-0007Xh-00; Thu, 25 Apr 2002 17:15:31 -0600 Date: Thu, 25 Apr 2002 17:15:30 -0600 To: "Steve Traugott" cc: infrastructures@terraluna.org, luke@madstop.com, labrown@nc.rr.com Subject: Re: [Infrastructures] The Turing Synthesis and Automated Administration Message-ID: <20020425231530.GI2481@sandia.gov> References: <20020419103809.A4927@scramjet.TerraLuna.Org> MIME-Version: 1.0 In-Reply-To: <20020419103809.A4927@scramjet.TerraLuna.Org> User-Agent: Mutt/1.3.24i From: "Jon Stearley" X-Filter-Version: 1.8 (sass165) X-WSS-ID: 10D652651816631-01-01 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: 7bit Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > Self-administering hosts are in effect Turing machines; they do brain > surgery on themselves. They don't behave according to strict > Von-Neumann rules, but have complex feedback loops, recursion, and > self-modifying code. > > *No* language or syntax can completely specify administrative actions > to be applied to a machine without also specifying the complete > history and order of previous changes. haven't been able to keep on the whole thread but wanted to say that after more though, i realize that my infrastructure does indeed have a very extensive state graph, full of dependancies and ordering. in my case, i chose debian which has a very extensive pkg dependancy system which runs nightly as directed by cfengine. cfengine also does the relatively cosmetic stuff (selecting which packages should be installed, their site-specific configuration, misc symlinks and other config, cleaning and etc - i LOVE cfengine), but without apt-get doing the brain surgery i would be _much_afraid_. so "you go" steve, this is an excellent topic! -jon From jcarro10@sprintspectrum.com Fri Apr 26 08:56:00 2002 Received: from smtpgw5.sprintspectrum.com (smtpgw5.sprintspectrum.com [207.40.188.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA07550; Fri, 26 Apr 2002 08:55:58 -0700 Received: from pkcex003.sprintspectrum.com (pkcex003.sprintspectrum.com [208.10.75.138]) by smtpgw5.sprintspectrum.com (8.11.2/8.11.3) with ESMTP id g3QFtiT10157; Fri, 26 Apr 2002 10:55:44 -0500 (CDT) Received: by pkcex003.sprintspectrum.com with Internet Mail Service (5.5.2654.89) id ; Fri, 26 Apr 2002 10:55:44 -0500 Message-ID: From: "Carroll, Jim P" To: "'Steve Traugott'" Cc: "Infrastructures (E-mail)" Subject: RE: [Infrastructures] authentication & client file access Date: Fri, 26 Apr 2002 10:55:41 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2654.89) Content-Type: text/plain; charset="iso-8859-1" Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > I've just modified all mentions of NIS on Infrastructures.Org, to now > also mention LDAP. Nutshell: LDAP has been proven to be a better > alternative than NIS, much more flexible, and is usable even on hosts > which don't explicitly support it, by using Perl scripts as glue. > When I was at Netscape Corp. in 2000 I only had one password at a > time, across all platforms -- UNIX, NT, web auth, etc. To change your > password, you just type 'passwd' in any web browser's Location: box, > and DNS is set up to take you to a cgi which lets you change your > password in LDAP... Alright, let's say for the sake of argument that I'm considering LDAP for the first time. I understand that it should work well for user authentication, but can it facilitate the likes of netgroups and automounter maps without going to a hybrid NIS/NIS+/LDAP solution? > As far as NIS+ goes, I think it's horrible, and the web site > recommends against it -- I just made that wording stonger. As of > 2000, Sun still didn't use NIS+ for their own corporate maps, due to > reliability problems. They were still using NIS then, and I'd expect > them to slowly migrate to LDAP next, due to iPlanet. Anyone know > current status? Yeah, the Sun/NIS+ story seems to be everybody's favourite even-the-cobbler-won't-wear-his-own-shoes story. A classic. No idea on the current status. Oh wait, stop the presses. I just got word from an inside contact that Sun is on a massive internal campaign to move to SunONE directory (formerly iPlanet directory) and LDAP. > NIS+ is a greater security concern in terms of denial of service -- > it's too easy to corrupt. If you enforce good passwords in the first > place, then neither NIS nor LDAP are security concerns in any > practical sense. Makes sense. I've certainly been experiencing credential corruption, thanks to Linux. :/ > Pull, never push. Did I say push? My bad. ;) > > Maybe NIS slaves on every workstation *is* the answer...? > > Yes, if you're using NIS. Worked for thousands of mission-critical > trader workstations for years. Requires some intelligent watchdog > code -- see > http://www.terraluna.org/dgp/cvsweb/isconf2i/bin/nis_watchdog?rlog. Hhhmmm. I'd be interested in some loose thoughts regarding what sort of crafting would be required to do this (if any), but in an LDAP context. > > Maybe CodaFS is the answer for home directories, providing we > > set those users up with a Linux-based CodaFS server? Does > anyone have any > > feel on how "ready for prime time" CodaFS is? > > Likely never -- InterMezzo is a retry, by one of the Coda developers. > Again, words go stale. Fixed on the site. InterMezzo... I'll have to take a look. Thanks! > Every time I've hit this question we've ended up setting up our own, > if for no other reason than security -- it's easier to keep > application developers from mucking about in the infrastructure > code, even accidentally. The drawback of a separate server or > repository is when you want to share infrastructure documentation > with those same developers or management. There's no way to set up security in CVS? Grr.... jc From ShprentzJ@nima.mil Fri Apr 26 10:20:41 2002 Received: from relay2.nima.mil (relay2.nima.mil [164.214.4.52]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA19967 for ; Fri, 26 Apr 2002 10:20:39 -0700 Received: by relay2.nima.mil; id NAA17435; Fri, 26 Apr 2002 13:20:33 -0400 (EDT) Received: from crusher.nima.mil(164.214.16.12) by relay2.nima.mil via smap (V5.5) id xma017315; Fri, 26 Apr 02 13:20:16 -0400 Received: by smtpgate.nima.mil with Internet Mail Service (5.5.2653.19) id <252WN3QH>; Fri, 26 Apr 2002 13:20:15 -0400 Message-ID: <4D8B7F4EF380D51181C70008C786EBAC840197@wshx03.nima.mil> From: "Shprentz, Joel [C] " To: "'infrastructures@mailman.terraluna.org'" Date: Fri, 26 Apr 2002 13:20:14 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" Subject: [Infrastructures] Solaris version of aix.mk Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Isconf includes a sample aix.mk, the makefile that specifies the configuration and installation steps to build complete workstations and servers. Does anyone have a similar solaris.mk they could share? -- Joel Shprentz National Imagery and Mapping Agency Mailstop N-17 Washington Navy Yard, Building 213 1200 First Street, SE Washington, DC 20303-0001 202-685-3534 From hakanson@cse.ogi.edu Fri Apr 26 10:47:41 2002 Received: from church.cse.ogi.edu (root@cse.ogi.edu [129.95.20.2]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA00726 for ; Fri, 26 Apr 2002 10:47:40 -0700 Received: from cse.ogi.edu (hepburn.cse.ogi.edu [129.95.40.57]) by church.cse.ogi.edu (8.9.3/8.9.3) with ESMTP id KAA09232 for ; Fri, 26 Apr 2002 10:47:53 -0700 (PDT) Message-Id: <200204261747.KAA09232@church.cse.ogi.edu> X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: "Infrastructures (E-mail)" Subject: Re: [Infrastructures] authentication & client file access In-reply-to: message of "Fri, 26 Apr 2002 10:55:41 CDT." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 26 Apr 2002 10:47:52 -0700 From: Marion Hakanson Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Jim P Carroll wrote: > . . . > Alright, let's say for the sake of argument that I'm considering LDAP for > the first time. I understand that it should work well for user > authentication, but can it facilitate the likes of netgroups and automounter > maps without going to a hybrid NIS/NIS+/LDAP solution? Sun provides schema to map the NIS maps into LDAP, including netgroups and automounter maps. See the following articles (and others there): http://www.sun.com/blueprints/1000/ldap-sol8.pdf http://www.sun.com/blueprints/0201/dsimport.pdf > > > Maybe NIS slaves on every workstation *is* the answer...? > > > > Yes, if you're using NIS. Worked for thousands of mission-critical > > trader workstations for years. Requires some intelligent watchdog > > code -- see > > http://www.terraluna.org/dgp/cvsweb/isconf2i/bin/nis_watchdog?rlog. > > Hhhmmm. I'd be interested in some loose thoughts regarding what sort of > crafting would be required to do this (if any), but in an LDAP context. Some of the Sun Blueprints LDAP articles discuss replication issues, in particular this one, which compares NIS, NIS+, and LDAP: http://www.sun.com/blueprints/1099/solaris.pdf Note that I'm not saying Sun is the only game in town, but it does look like they've been this route. Regards, -- Marion Hakanson CSE Computing Facilities From luke@madstop.com Sat Apr 27 16:38:29 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id QAA25708 for ; Sat, 27 Apr 2002 16:38:28 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g3RNciI01709 for ; Sat, 27 Apr 2002 18:38:45 -0500 (CDT) Date: Sat, 27 Apr 2002 18:38:44 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] madstop.com, take 2? Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi all, Okay, after talking to Steve for a while the other night, and thinking a bit about what I do and don't like about slashcode, and also what I can reasonably do now, I have some greater clarity on what I am trying to do. Premise: I don't like what slashcode is generally used for, but I do like it as a framework. That is, it does a lot of what I want to do, which means I don't have to rewrite that part. So, I'm going to start with it as a framework, but probably replace a lot of what it actually does (e.g., replace index.pl with something of my own, add other sections, and probably disable some of the existing sections. So, given that I want to use slashcode as my framework (unless someone can find a better web development/community framework that is freely available), I plan on working in a few basic phases: Phase one: Provide two new sections to slash, a content section and a code section. I'm going to put code as a higher priority, but not much. Code: The code section will try to be similar to CPAN, in that you have to register as an author and there will be somewhat strict rules on how you submit things. I believe at this point that it will be easiest to only accept tarballs, even of one-file scripts, although I could easily be convinced otherwise. I would very much like to provide the ability to view code on the website, but on the other hand, is that really a big deal? In looking at other sites which revolve around community access to tools, I haven't really found one which displays actual code, and none of them seem to be hampered by that. So, it looks like the ability to display code isn't a killer feature after all. I guess in my mind I'm kind of modeling this page after BeBits (www.bebits.com), because I used it quite a bit back when BeOS was still kicking and found that it seldom got in my way. However, I plan to provide more features eventually and in general provide a more comprehensive site, especially related to how an item is rated--more like how sourceforge rates things, probably. In other words, you're on your own for revision control, but you can upload new revisions to this site. Optionally, I may provide users a choice between hosting the code themselves and having the site host it. I think it's better if the site hosts it, though, so tools can be provided to retrieve the code. Content: This would basically be things people wrote up, such as tutorials or white papers of come kind. My vision of this is pretty vague at this point, partially because I really want something like looks like RFCs, and I don't know where to draw the line. Can a whitepaper graduate to an RFC? That kind of stuff. So at this point, this would just provide people the ability to put text on the site, and categorize it in some way. This would probably be a file upload/paste-in or something like that. It would probably need the ability to store multiple pages, though, so I would also probably need the ability to upload tar files. Any more feedback on this page would be nice. This is actually seeming more like phase 1.5. Phase 2: There are two to four sections I would like to create here: Tips, RFCs, and problem reports & solutions. I thought there was another one, but I can't seem to remember what it is... I don't know how realistic it is to create all of these in what could be considered a phase, but I'd like to have all of them eventually. I may actually create the Tips section before I do anything, because it should be quite simple, and should be a good learning experience for me. On the other hand, I don't know if I've ever started off small... The RFC page should essentially provide a way to submit, vote on, and display potential RFCs, and I suppose a way to display and search existing, accepted RFCs. What is a sysadmin RFC? I suppose it's essentially a best practice or something. The Problem Reports page may never exist, but it's something I think could be useful, and it could significantly reduce the traffic on Google Groups :). It's basically a place where someone can register a problem and/or a solution, and then probably comment and vote on solutions or problems. I think this is valuable because a helluvalot of sysadmin work is related to tracking down an existing solution to a known problem; providing a place for a sysadmin to say "hey, I found it" or "hey, I need help" would be quite nice. Something like what SunManagers used to be, before the Signal to Noise went just about to nil. Conclusion: Well, I guess check the front page periodically, because I now know enough about slashcode to be dangerous, so I'm going to start messing around. We'll see how it goes... -- "When I die, I want go out just like my grandfather, in his sleep, peaceful and quiet...not kicking and screaming like the other guys in his car" From luke@madstop.com Sat Apr 27 17:46:33 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA22790 for ; Sat, 27 Apr 2002 17:46:32 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g3S0kaO01906; Sat, 27 Apr 2002 19:46:38 -0500 (CDT) Date: Sat, 27 Apr 2002 19:46:36 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: "Shprentz, Joel [C] " cc: "'infrastructures@mailman.terraluna.org'" Subject: Re: [Infrastructures] Solaris version of aix.mk In-Reply-To: <4D8B7F4EF380D51181C70008C786EBAC840197@wshx03.nima.mil> Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="-559023410-851401618-1019954796=:1878" Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. ---559023410-851401618-1019954796=:1878 Content-Type: TEXT/PLAIN; charset=US-ASCII On Fri, 26 Apr 2002, Shprentz, Joel [C] wrote: > Isconf includes a sample aix.mk, the makefile that specifies the > configuration and installation steps to build complete workstations and > servers. Does anyone have a similar solaris.mk they could share? Hmm; I don't know if you'll find this useful on it's own, but I've included what I have. You won't find it very useful, though; all the real work is done in external scripts. I can make those scripts available, if you want, along with a couple days' newer make file. I've actually been working on isconf quite a bit (including a lot of rewriting and reorganization), so if you're just starting to use isconf, you may want to wait a few days, until I release what I've done. I finally got semi-written (email) permission to release, from my company, so I need to just go ahead and do it. Ugh, there's a 900ms ping time between my keyboard, so I'm going to cut this short. Basically, though, I've got quite a bit of solaris code, and am willing to send it to you (have you seen my jumpstart page, http://pixie.madstop.com/jumpstart?), but I need a day or two to sync up with my work repository and clean it up for release (i.e., export it). If others are interested (I rewrote the isconf script in perl, rewrote most of the rc.isconf and renamed to isboothook, am now using cvs instead of rsync for updating the local copy of isconf code, split out all of the variables into a separate file, created a types file to define machine types along with a script similar to visudo to abstract the ordering of make targets, and a few more things), that might motivate me to get it done sooner. I'm supporting hp-ux and solaris 8. Luke -- "The Microsoft Exchange Information Store service depends on the Microsoft Exchange Directory service which failed to start because of the following error: The operation completed successfully." ---559023410-851401618-1019954796=:1878 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="sunos.mk" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="sunos.mk" IyAkSWQ6IHN1bm9zLm1rLHYgMS4yLjIuNy4yLjE5IDIwMDIvMDQvMjQgMjE6 MTI6Mjgga2FuaWVsYSBFeHAgJA0KIyAvaXMvY29uZi9iaW4vTWFrZWZpbGUg LSBjb25maWd1cmF0aW9uIGNvbnRyb2wgbWFrZWZpbGUgDQojDQojCVRoaXMg bWFrZWZpbGUgaXMgaW50ZW5kZWQgdG8gYmUgY2FsbGVkIGJ5IC9pcy9jb25m L2Jpbi9jb25maWd1cmUuDQojDQojCVRISVMgTUFLRUZJTEUgTVVTVCBCRSBF WEVDVVRFRCBPTiBUSEUgVEFSR0VUIE1BQ0hJTkUgKCB0aGUgbWFjaGluZQ0K Iwl3aGljaCBpcyB0byBiZSBjb25maWd1cmVkL3VwZGF0ZWQpLg0KIw0KDQoj IG5vb3AgZm9yIGJhY2t3YXJkIGNvbXBhdGliaWxpdHkNCkJsb2NrOiAgDQoN ClN1bk9TXzAwOiByc3luY19hbGwgY3ZzXzEuMTEgb3BlbnNzaF8yLjkuMiBj aGFuZ2Utcm9vdC1ob21lIHN1ZG9fMS42LjNwNSBcDQpwYXRjaC0yMDAyMDQx MA0KDQpTdW5SZWNvbW1lbmRlZFBhdGNoZXM6IA0KDQpjcm9uOiByc3luY19h bGwgI2Nyb250YWJiZXINCg0KIyMjDQojIHJlYm9vdA0KcmVib290Og0KCS0g cm0gLWYgLnJlYm9vdF9uZWVkZWQNCglpbml0IDYNCglzbGVlcCA5OTkNCg0K IyMjDQojIHBhY2thZ2UgaW5zdGFsbHMNCg0KY3ZzXzEuMTEgb3BlbnNzaF8y LjkuMiBzdWRvXzEuNi4zcDU6DQoJcGtnaW5zdGFsbCAkQA0KCXN0YW1wICRA DQoNCmRpc2tzdWl0ZV80LjIuMToNCglwa2dpbnN0YWxsICRADQoJdG91Y2gg LnJlYm9vdF9uZWVkZWQNCglzdGFtcCAkQA0KDQpwYXRjaC0yMDAyMDQxMDoN CgkkKElTX1JPT1RESVIpL2xpYi90YXJnZXRzL3N1bm9zL3BhdGNoLWluc3Rh bGwgJEANCglzdGFtcCAkQA0KCWluaXQgNg0KCXNsZWVwIDk5OQ0KDQpqYXNz LXNlcnZlci1iYXNlLWhhcmRlbjEuMDoNCglqYXNzd3JhcCAkQA0KCXRvdWNo IC5yZWJvb3RfbmVlZGVkDQoJc3RhbXAgJEANCg0Kc2V0LWZzbG9nZ2luZzoN CgkkKElTX1JPT1RESVIpL2xpYi90YXJnZXRzL3N1bm9zLyRADQoJdG91Y2gg LnJlYm9vdF9uZWVkZWQNCglzdGFtcCAkQA0KDQppbnN0YWxsLWRpc2tzdWl0 ZTogZGlza3N1aXRlXzQuMi4xDQoJJChJU19ST09URElSKS9saWIvdGFyZ2V0 cy9zdW5vcy8kQA0KCXRvdWNoIC5yZWJvb3RfbmVlZGVkDQoJc3RhbXAgJEAN CgkNCg== ---559023410-851401618-1019954796=:1878-- From luke@madstop.com Sat Apr 27 17:49:32 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA23983 for ; Sat, 27 Apr 2002 17:49:31 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g3S0ngN01914; Sat, 27 Apr 2002 19:49:43 -0500 (CDT) Date: Sat, 27 Apr 2002 19:49:42 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: Marion Hakanson cc: "Infrastructures (E-mail)" Subject: Re: [Infrastructures] authentication & client file access In-Reply-To: <200204261747.KAA09232@church.cse.ogi.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, 26 Apr 2002, Marion Hakanson wrote: > Jim P Carroll wrote: > > . . . > > Alright, let's say for the sake of argument that I'm considering LDAP for > > the first time. I understand that it should work well for user > > authentication, but can it facilitate the likes of netgroups and automounter > > maps without going to a hybrid NIS/NIS+/LDAP solution? I've been using LDAP on solaris to do automounting for about 1.5 years; it works at least as well as anything else, and is just as easy. I believe at least HP-UX supports it, although I don't think PADL's tools do (although they're open source). I can give you help, if necessary; I love it, although LDAP on Solaris isn't all that secure right now, and I wouldn't use it for production stuff because I'm not confident in the iPlanet ldap server. -- My favorite was a professor at a University I Used To Be Associated With who claimed that our requirement of a non-alphabetic character in our passwords was an abridgement of his freedom of speech. -- Jacob Haller From atp@piskorski.com Sat Apr 27 19:06:39 2002 Received: from tehun.pair.com (tehun.pair.com [209.68.2.71]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id TAA24902 for ; Sat, 27 Apr 2002 19:06:38 -0700 Received: (qmail 24878 invoked by uid 3106); 28 Apr 2002 02:06:54 -0000 Date: Sat, 27 Apr 2002 22:06:53 -0400 From: Andrew Piskorski To: infrastructures@terraluna.org Subject: Re: [Infrastructures] madstop.com, take 2? Message-ID: <20020428020653.GA14166@piskorski.com> Mail-Followup-To: infrastructures@terraluna.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.25i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Sat, Apr 27, 2002 at 06:38:44PM -0500, Luke A. Kanies wrote: > So, given that I want to use slashcode as my framework (unless > someone can find a better web development/community framework that > is freely available), I plan on working in a few basic phases: Luke, have you looked at http://openacs.org/ ? OpenACS is certainly my web development/community toolkit of choice, by far. (And it's all released under the GNU licence.) But, if Slashcode seems close to what you want, then OpenACS may not be for you, because while I'm not familiar with Slashcode at all, back on Jan. 13 2002, Don Baccus (head of the OpenACS project) mentioned being surprised that someone's publishing a whole book on Slashcode. he said: http://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0000Dl "Slash? Really? Have you ever looked at the code? It more or less fits on the back of an envelope ... it doesn't need a book. Maybe a pamphlet." In other words, sounds as if Slashcode is a dramatically less ambitious toolkit than OpenACS. Slash uses MySQL, right? No matter what, you probably REALLY do not want to be using MySQL for anything critical. For why, see Ben Adida's classic and scathing article: http://openacs.org/philosophy/why-not-mysql.html Basically, my understanding is that there are or were some planned replacemetns for MySQL's data storage backend which might eventually make MySQL meet the ACID criteria, and thus be suitable for use with valuable (never mind mission critical!) data. But even then, you'd still have a lousy relational database, inferior in terms of features, reliability, scaleability, etc., to PostgreSQL. Now, *I* have never personally used either MySQL or Postgres yet (still too busy fooling with Oracle), so feel free to take this with a grain of salt, but I'm convinced by what I've read. Basically, there doesn't seem to be ANY good reason to choose MySQL over Postgres. Sourceforge also had some interesting articles on why they had to switch from MySQL to Postgres in order to make the SourceForge site work right, but I don't have a link to those handy. FYI. (Really - I don't feel any need to try to start a toolkit flameware...) -- Andrew Piskorski http://www.piskorski.com From luke@madstop.com Sat Apr 27 21:56:59 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA32633 for ; Sat, 27 Apr 2002 21:56:58 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g3S4v2W03007; Sat, 27 Apr 2002 23:57:04 -0500 (CDT) Date: Sat, 27 Apr 2002 23:57:02 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: Andrew Piskorski cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] madstop.com, take 2? In-Reply-To: <20020428020653.GA14166@piskorski.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Sat, 27 Apr 2002, Andrew Piskorski wrote: > Luke, have you looked at http://openacs.org/ ? OpenACS is certainly > my web development/community toolkit of choice, by far. (And it's all > released under the GNU licence.) > > But, if Slashcode seems close to what you want, then OpenACS may not > be for you, because while I'm not familiar with Slashcode at all, back > on Jan. 13 2002, Don Baccus (head of the OpenACS project) mentioned > being surprised that someone's publishing a whole book on Slashcode. > he said: > > http://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0000Dl > > "Slash? Really? Have you ever looked at the code? It more or less fits > on the back of an envelope ... it doesn't need a book. Maybe a > pamphlet." Wow, that hurt. Not the insult of slashcode, I'm not real fond of it myself. What hurts is that I feel like I've made a decent investment in it, and now you bring up two (because the above link is all about comparing zope to OpenACS) different application servers. Even worse, neither of them is written in my language of choice (perl). Crap. This is going to take a lot more research. I put out a call for options like this a while ago, but I guess they're going to keep trickling in. > In other words, sounds as if Slashcode is a dramatically less > ambitious toolkit than OpenACS. I would rather describe it as Slashcode is not at all ambitious, but instead merely solves the problem of how to run Slashdot. That's its own goal, and as far as I can tell, that's the only thing it does well. It's not too bad of a framework, but it could use a lot more OO (even though it claims it does use it where appropriate, I would heartily disagree), and a helluva lot more documenation. > Slash uses MySQL, right? No matter what, you probably REALLY do not > want to be using MySQL for anything critical. For why, see Ben > Adida's classic and scathing article: > > http://openacs.org/philosophy/why-not-mysql.html [SNIP] Well, for one, I didn't choose MySQL, I chose slashcode which brought MySQL along as a kind of gift, I guess. On the other hand, I know a number of people who have satisfactorily used MySQL for a number of different things, so... > Sourceforge also had some interesting articles on why they had to > switch from MySQL to Postgres in order to make the SourceForge site > work right, but I don't have a link to those handy. To say that databases are kind of a weak point with me would certainly be glossing over the fact that I have studiously avoided learning a single damn thing about them, until tonight when I decided I had to. Dern. I really, really like perl, but I guess I'll have to look into OpenACS (tcl) and Zope (python). OpenACS (or the original by ArsDigita; why not use it? It's open source, under a variant of the Mozilla license) seems like a great one for features, and their examples sites certainly don't display the kind of conformance that slash sites are pretty much all guilty of, which is a good thing. I'll check it out; thanks. Luke -- A classic is something that everybody wants to have read and nobody wants to read. -- Mark Twain From atp@piskorski.com Sun Apr 28 11:40:41 2002 Received: from tehun.pair.com (tehun.pair.com [209.68.2.71]) by roton.terraluna.org (8.9.3/8.9.3) with SMTP id LAA12262 for ; Sun, 28 Apr 2002 11:40:39 -0700 Received: (qmail 86269 invoked by uid 3106); 28 Apr 2002 18:40:56 -0000 Date: Sun, 28 Apr 2002 14:40:56 -0400 From: Andrew Piskorski To: infrastructures@terraluna.org Cc: "Luke A. Kanies" Subject: Re: [Infrastructures] madstop.com, take 2? Message-ID: <20020428184055.GA75936@piskorski.com> Mail-Followup-To: infrastructures@terraluna.org, "Luke A. Kanies" References: <20020428020653.GA14166@piskorski.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.25i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Sat, Apr 27, 2002 at 11:57:02PM -0500, Luke A. Kanies wrote: > To say that databases are kind of a weak point with me would certainly be > glossing over the fact that I have studiously avoided learning a single > damn thing about them, until tonight when I decided I had to. Dern. "Philip and Alex's Guide to Web Publishing" has some excellent background material on RDBMS: http://www.arsdigita.com/books/panda/ But if you don't know much about RDBMS, then my advice is make your life a lot easier and just use PostgreSQL. Once you do learn more, you'll never regret the decision. Oh, and "SQL for Web Nerds" should also be useful to you: http://www.arsdigita.com/books/sql/ > I really, really like perl, but I guess I'll have to look into OpenACS > (tcl) and Zope (python). I used to use Perl. I liked it a lot, but I like Tcl a lot more. (I could go on at length about why, but I don't think you'd care.) More importantly though, it's a personal preference thing, and really doesn't matter. If you're good with Perl you'll be just as good with Tcl, and vice versa. Once nice thing about Tcl though, is that its syntax so simple and straightforward, that you'll probably be able to come up to speed switching from Perl to Tcl faster than Tcl to Perl. "Tcl for Web Nerds" is a decent introduction: http://www.arsdigita.com/books/tcl/ My only advice on Tcl is to make eventually sure you REALLY understand "eval". (You can be quite productive in Tcl without truly groking eval, but IMNSHO, you'll be missing much of the true usefullnes of Tcl until you do.) Most of the Tcl books and tutorials tell you what eval is for - to "evaluate" code, duh - but DON'T get into all the ways that eval is truly super useful in Tcl. I'm not aware of any good explanations of the manifold uses of eval. (I should really collect some together and write them up in a little article myself, if I ever find the time...) But basically, if you ever find yourself beating your head against the wall saying, "Agh, this was so easy for me in Perl (or Bourne shell, or whatever), why can't I think of a nice easy way do it in Tcl!?", I'd bet that eval is your answer. > OpenACS (or the original by ArsDigita; why not use it? It's open > source, under a variant of the Mozilla license) seems like a great > one for features, and their examples sites certainly don't display > the kind of conformance that slash sites are pretty much all guilty > of, which is a good thing. I would advise anyone who's interested in the ACS at all to use OpenACS 4.5, unless you have a very definite and specific reason to consider a different version. Basically, if you don't know what ACS you should use, you should use OpenACS 4.5. OpenACS 4.5 is a direct descendant of ACS 4.2 (Tcl), with many fixes and enhancements added. And OpenACS supports BOTH Oracle and Postgres, and includes lots of fixes and enhancements, so there's no reason to use ACS 4.2 on a new project. In fact, back when ArsDigita, Inc. still existed, they specifically pointed to openacs.org for ACS 4.2 support. I myself am actually still using a patched up version of ACS 4.2, but that's just because I haven't had the need yet to switch over my production systems to OpenACS 4.5 - I definitely will at some point. The "original ACS" was in fact the ACS 1.x - 3.x series, all of which grew more or less organically out of the demands of client work. ACS 4.0 was essentially a major refactoring of that data model and code, and is NOT easily compatible or upgradeable with the older 3.x stuff. OpenACS 3.2 was a port of ACS 3.2 from Oracle to Postgres, and importantly, supports ONLY Postgres, while all the newer OpenACS 4.x stuff supports BOTH Oracle and Postgres. If you really, really want to use Java, you could try using either ACS Java 4.2 (or whatever it was called) or ACS Java 5.0 - which are two COMPLETELY different toolkits, by the way. ACS Java 4.2 was very much like ACS Tcl 4.2, using JSP pages rather than AOLserver ADP pages, etc., and some of my former coworkers commented favorably on using it to build real websites. ACS Java 4.6 or 5.0 (or whatever it was called - the named changed several times) was much more "J2EE", with a "persistance layer" in between the RDBMS and the application code, lots of funky XSLT template processing, the "Bebop" UI API, etc. Basically, while I heard comments about some of ACS Java 5.0's supposedly nifty features, the only feedback I ever heard about using it to actually build REAL, working websites was that it SUCKED. I specifically remember phrases like, "Development was easily 10x slower than with with either ACS Tcl (any version) or ACS 4.2 Java." In fact, I'm not aware of ANY public site running ACS Java 5.0, although there probably are a few. ArsDigita no longer exists at all as a company - the company assets were bought by Red Hat. I heard some rumor of possible continued support and/or development of ACS 5.0 Java by Red Hat, but as far as I know, nothing has come of that. Which means, with ACS Java anything, you're pretty much totally on your own, and you're dealing with code that just hasn't seen much real world use at all. With the older 3.x ACS and OpenACS series, there isn't too much in terms of ongoing development (there is some), but there is a large user community out there that either used to or still does use the software, so you'd have good luck getting your questions answered. But OpenACS 4.5 is the only publically available version of the ACS toolkit currently under active maintenance and development, as far as I know. (I say "publically availabe", because I believe that there are various ex-ArsDigita clients who have essentially forked off some version of ACS 3.x or earlier for their own internal use only.) > I'll check it out; thanks. No problem, glad I could help! -- Andrew Piskorski http://www.piskorski.com From ShprentzJ@nima.mil Mon Apr 29 10:50:32 2002 Received: from relay2.nima.mil (relay2.nima.mil [164.214.4.52]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA23862 for ; Mon, 29 Apr 2002 10:50:27 -0700 Received: by relay2.nima.mil; id NAA09969; Mon, 29 Apr 2002 13:50:42 -0400 (EDT) Received: from crusher.nima.mil(164.214.16.12) by relay2.nima.mil via smap (V5.5) id xma009676; Mon, 29 Apr 02 13:49:53 -0400 Received: by smtpgate.nima.mil with Internet Mail Service (5.5.2653.19) id <252W3VJ2>; Mon, 29 Apr 2002 13:49:53 -0400 Message-ID: <4D8B7F4EF380D51181C70008C786EBAC840198@wshx03.nima.mil> From: "Shprentz, Joel [C] " To: "'Luke A. Kanies'" Cc: "'infrastructures@mailman.terraluna.org'" Subject: RE: [Infrastructures] Solaris version of aix.mk Date: Mon, 29 Apr 2002 13:49:48 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Yes, your makefile is helpful and I'll happily wait to see what else you have. When I first looked at your jumpstart page, it wasn't there (see madstop message #3 at http://madstop.com/article.pl?sid=00/01/25/1430236&mode=dynamic). Thanks for restoring it. -----Original Message----- From: Luke A. Kanies [mailto:luke@pixie.madstop.com] Sent: Saturday, April 27, 2002 8:47 PM To: Shprentz, Joel [C] Cc: 'infrastructures@mailman.terraluna.org' Subject: Re: [Infrastructures] Solaris version of aix.mk On Fri, 26 Apr 2002, Shprentz, Joel [C] wrote: > Isconf includes a sample aix.mk, the makefile that specifies the > configuration and installation steps to build complete workstations and > servers. Does anyone have a similar solaris.mk they could share? Hmm; I don't know if you'll find this useful on it's own, but I've included what I have. You won't find it very useful, though; all the real work is done in external scripts. I can make those scripts available, if you want, along with a couple days' newer make file. I've actually been working on isconf quite a bit (including a lot of rewriting and reorganization), so if you're just starting to use isconf, you may want to wait a few days, until I release what I've done. I finally got semi-written (email) permission to release, from my company, so I need to just go ahead and do it. Ugh, there's a 900ms ping time between my keyboard, so I'm going to cut this short. Basically, though, I've got quite a bit of solaris code, and am willing to send it to you (have you seen my jumpstart page, http://pixie.madstop.com/jumpstart?), but I need a day or two to sync up with my work repository and clean it up for release (i.e., export it). If others are interested (I rewrote the isconf script in perl, rewrote most of the rc.isconf and renamed to isboothook, am now using cvs instead of rsync for updating the local copy of isconf code, split out all of the variables into a separate file, created a types file to define machine types along with a script similar to visudo to abstract the ordering of make targets, and a few more things), that might motivate me to get it done sooner. I'm supporting hp-ux and solaris 8. Luke -- "The Microsoft Exchange Information Store service depends on the Microsoft Exchange Directory service which failed to start because of the following error: The operation completed successfully." From luke@madstop.com Tue May 7 19:36:18 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id TAA22356 for ; Tue, 7 May 2002 19:36:17 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g482a5F09014 for ; Tue, 7 May 2002 21:36:06 -0500 (CDT) Date: Tue, 7 May 2002 21:36:05 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] perlmonks? Everything? Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Any perl monks out there? I've begun experimenting with the Everything system used to run perlmonks.com; it seems to be somewhere between wiki and total chaos, and I think I like it. If anyone is interested in what it looks like or how it works, either check out perlmonks.com, everydevel.com, or my test site at test.madstop.com. I haven't had the chance to look at much of the code yet, but so far it looks pretty clean. Ooh, clean, and well documented! And it didn't even reset my hostname to '-f', like slashcode did! Anyway, thought I'd send the shout out, to see if anyone has any experience with this stuff. Luke -- The Chico, California, City Council enacted a ban on nuclear weapons, setting a $500 fine for anyone detonating one within city limits. From luke@madstop.com Thu May 9 08:30:12 2002 Received: from madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA18198 for ; Thu, 9 May 2002 08:30:11 -0700 Received: from localhost (luke@localhost) by madstop.com (8.11.6+Sun/8.11.6) with ESMTP id g49FUAc16380 for ; Thu, 9 May 2002 10:30:10 -0500 (CDT) Date: Thu, 9 May 2002 10:30:10 -0500 (CDT) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] isconf-3 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Well, after talking to Steve last night, and finally doing an export, here it is: Isconf 3, as we're calling it: http://test.madstop.com/index.pl?node_id=317 Just a couple of tarballs right now, but hopefully that will change over time. I've included it on my test Everything site, so let me know how you hate it. I also hope to have cvs links of it up soon, but more realistically that will probably take almost a month. If you have any questions or comments, just let me know. BTW, Steve and I are planning on creating CPAN modules for as much of this as possible. We're still discussing the exact format and structure, and how much of the code itself will be included as part of the module, but there are definite plans for CPAN, now that everything is so clearly in perl. Luke -- It's a small world, but I wouldn't want to paint it. -- Stephen Wright From stevegt@roton.terraluna.org Wed May 15 00:28:32 2002 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id AAA25273 for infrastructures@TerraLuna.Org; Wed, 15 May 2002 00:28:32 -0700 Date: Wed, 15 May 2002 00:28:32 -0700 From: Steve Traugott To: infrastructures@TerraLuna.Org Message-ID: <20020515002832.A26511@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Steve Traugott is Available Sender: infrastructures-admin@mail