From stevegt@roton.terraluna.org Sun Dec 2 11:08:38 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id LAA16657 for infrastructures@scramjet.TerraLuna.Org; Sun, 2 Dec 2001 11:08:38 -0800 Date: Sun, 2 Dec 2001 11:08:37 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011202110837.A16449@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Test Message Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: ...testing empty list and archives after initial Mailman setup. -- . . ` * Steve Traugott ` . * + Infrastructure Architect + ` stevegt@TerraLuna.Org ' * . ' +` * http://www.stevegt.com/ From stevegt@roton.terraluna.org Sun Dec 2 11:19:32 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id LAA22284 for infrastructures@scramjet.TerraLuna.Org; Sun, 2 Dec 2001 11:19:32 -0800 Date: Sun, 2 Dec 2001 11:19:31 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011202111931.A17109@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by roton.terraluna.org id LAA22284 Subject: [Infrastructures] Test Message 2 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: ...testing list with one subscriber after Mailman setup. -- . . ` * Steve Traugott ` . * + Infrastructure Architect + ` stevegt@TerraLuna.Org ' * . ' +` * http://www.stevegt.com/ From stevegt@roton.terraluna.org Sun Dec 2 23:28:38 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id XAA07668 for infrastructures@scramjet.TerraLuna.Org; Sun, 2 Dec 2001 23:28:38 -0800 Date: Sun, 2 Dec 2001 23:28:37 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011202232837.A2554@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="tThc/1wpZn/ma/RB" Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Archives are up! Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: --tThc/1wpZn/ma/RB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi All! Infrastructures list migration to mailman complete. So far it looks like the patient survived. Feel free to let me know if you have any problems. I've also migrated the majordomo archives (which were never available via the web) to mailman -- so now every stinkin' message anyone has ever posted to the infrastructures list is indexed and formatted and available at http://mailman.terraluna.org/pipermail/infrastructures/. This URL is also on the 'community' page at infrastructures.org, replacing the dead archive URL which was @skylab. Who's at LISA? Joyce and I just got here tonight... =20 Steve --=20 . . ` * =20 Steve Traugott ` . * + =20 Infrastructure Architect + ` =20 stevegt@TerraLuna.Org ' * . ' +` * =20 http://www.stevegt.com/ --tThc/1wpZn/ma/RB Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE8Cymk8rKIxO1Fc9MRAqhPAJ9+9TSkeXQf73g+n0/HT4N3sTpD4wCglywp rchpDu1KHuEqz2Gr0RgvzjU= =siRf -----END PGP SIGNATURE----- --tThc/1wpZn/ma/RB-- From mbarr@mbarr.net Thu Dec 13 20:44:42 2001 Received: from neuromancer.mbarr.net (66-108-143-133.nyc.rr.com [66.108.143.133]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id UAA22761 for ; Thu, 13 Dec 2001 20:44:34 -0800 Received: from [192.168.0.10] (unknown [192.168.0.10]) by neuromancer.mbarr.net (Postfix) with ESMTP id 77E42211D3D for ; Thu, 13 Dec 2001 23:44:31 -0500 (EST) Mime-Version: 1.0 X-Sender: mbarr@192.168.0.10 (Unverified) Message-Id: Date: Thu, 13 Dec 2001 23:44:28 -0500 To: infrastructures@roton.terraluna.org From: Matthew Barr Content-Type: text/plain; charset="us-ascii" ; format="flowed" Subject: [Infrastructures] Introduction... Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I met some of you at LISA, but I figured I should at least say hello :-) I'm actually a Unix admin/etc with an interest in the larger picture, but less interest in actually coding my own solutions from scratch. Steven suggested that this might be novel set of preferences for this list, so here I am.. Basically, I'm guessing that I'm a typical user of these design and architecture and large scale configuration tools, but not a scratch designer. I've worked at Lehman Brothers in the past, and am now somewhat independent. (translation= still looking for full time work ;-) Hope everyone is having a happy holiday season.. (Happy Chanukah for now!) Matthew -- _______________________________________________________________________ Matthew Barr mailto:mbarr@mbarr.net AIM: MBarr1244 ICQ: 22130424 From ken@scriba.org Tue Dec 11 06:12:14 2001 Received: from scriba.org (ken@scriba.org [208.178.122.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id GAA28773 for ; Tue, 11 Dec 2001 06:12:13 -0800 Received: from localhost (ken@localhost) by scriba.org (8.8.5/8.7.3) with ESMTP id JAA21580 for ; Tue, 11 Dec 2001 09:12:01 -0500 Date: Tue, 11 Dec 2001 09:12:01 -0500 (EST) From: Kenneth McCracken X-Sender: ken@mansfield To: infrastructures@TerraLuna.Org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] non-profit ISP setup Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hello Steve, I am managing Toronto Freenet(now Toronto's oldest dialup, ISP since 1994 supplying barrier-free access to Toronto citizens) these days. We are attempting to offer a reliable service, but it has been difficult and a lot of the problems have been around (volunteer) sysadmin issues and hardware failures. I was wondering about a pathway to a more solid footing. We are migrating from antiquated sun equipment to AMD equipment with dual IDE drives configured for software RAID1 running freebsd and a single cisco as5200 router. We need to grow the ISP operation in an orderly way to provide a (income) foundation for other more forward looking projects. I've mentioned the Gold server concept/infrastructures.org site to the technical people involved, and I pretty much get silence. We also will soon be bringing to various low income people a few sites(hopefully remotely administered) where they access the internet via a high speed connection through lans of various configurtions: 1. 10-20 low end pentiums and and a lan server netbooting to recover client systems if files are messed with. 2. Similar lans with fewer mid-level pentiums, with fileserver to preserve working hooome directories 3. Higher end pentiums as internet kiosks in tight quarters The president of our sponsoring (colocating and bandwidth wholesaling) host is willing to take an evening to do some of the hands on configuation stuff, but I am not sure he will take the time to set it up so that it is easily replicatible. Our budget is quite limited, so I was wondering if you might already have some out of the box solutions or prescriptions for these particular situations. Ken McCracken Executive Director Toronto Free-Net 416-828-9323 From stevegt@roton.terraluna.org Fri Dec 14 00:04:40 2001 Received: (from stevegt@localhost) by roton.terraluna.org (8.9.3/8.9.3) id AAA28957 for infrastructures@scramjet.TerraLuna.Org; Fri, 14 Dec 2001 00:04:40 -0800 Date: Fri, 14 Dec 2001 00:04:39 -0800 From: Steve Traugott To: infrastructures@roton.terraluna.org Message-ID: <20011214000439.A5671@scramjet.TerraLuna.Org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="mP3DRpeJDSE+ciuQ" Content-Disposition: inline User-Agent: Mutt/1.2.5i Subject: [Infrastructures] Wiki stuff Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: --mP3DRpeJDSE+ciuQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Paul Holbrook's got me looking at UseModWiki... And I know Bud liked his SDF TWiki. Anyone else have any other recommendations, pro or con? One thing I'd like to support, for instance, is file uploads -- which opens a hairball of security issues; would be nice to integrate the Wiki username space with Mailman in some way, and only allow uploads from list members using their Mailman password. I've just discovered Bud's "Annotated Infrastructure Links" page, which looks like a copy of the html from Bud's old TWiki. I've added a link to it from the Infrastructures.Org 'lists_and_community' page. It's at http://www.sistema.it/infrastructures/infrastructures.html. Several people at last week's LISA mentioned Bud's TWiki, so I'm happy we can make these links available again. (Bud also sent me the raw data a while back, but I've never had the chance to put it up on a web or Wiki anywhere. It's likely Bud also told me about the above URL around the same time... If so, I blame senility on my part.) Steve --=20 . . ` * =20 Steve Traugott ` . * + =20 Infrastructure Architect + ` =20 stevegt@TerraLuna.Org ' * . ' +` * =20 http://www.stevegt.com/ --mP3DRpeJDSE+ciuQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE8GbKW8rKIxO1Fc9MRAgkgAKCb+/11SKFvbevJyLlbiDLOnkF0kACglbQR ojSEaq7fOa4MZ7QxYcg2VRs= =Piqf -----END PGP SIGNATURE----- --mP3DRpeJDSE+ciuQ-- From partain@dcs.gla.ac.uk Fri Jan 4 07:29:36 2002 Received: from iona.dcs.gla.ac.uk (iona.dcs.gla.ac.uk [130.209.240.35]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA16678 for ; Fri, 4 Jan 2002 07:29:34 -0800 Received: from slicker.dcs.gla.ac.uk ([130.209.242.51] helo=dcs.gla.ac.uk) by iona.dcs.gla.ac.uk with esmtp (Exim 3.13 #1) id 16MW8Q-0001Tx-00 for infrastructures@terraluna.org; Fri, 04 Jan 2002 15:19:55 +0000 To: infrastructures@terraluna.org Date: Fri, 04 Jan 2002 15:19:53 +0000 From: Will Partain Message-Id: Subject: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Greetings, infrastructural folks! Great to meet some of you at LISA. I have been musing slightly about doing backups without tapes, and pass along my current notes (below) for your comment and criticism. (Once we get beyond the high-level issues that kinda make sense for the infrastructures list, we should probably chat elsewhere -- I'd suggest ark-dev [http://lists.sf.net/mailman/listinfo/ark-dev], as that's where I'll probably do any detailed implementation chit-chat (if it happens).) Will ========================= A tapeless backups system for a workgroup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ At a LISA 2001 session on "recovery-oriented computing" [1], Dave Patterson (Berkeley) commented that "tape is dead". Reason: if tape doesn't cost more per byte than disk yet, it soon will. Quick check (2002.01): 120GB EIDE disk for $208 (Pricewatch) = $1.73/GB, 40GB DLT IV tape for $53 (Pricescan) = $1.32/GB. Close enough for me! And, if you haven't noticed lately, tape is a *pain*. So, what can we do backup-wise [2] (NB: not `archiving'... [3]) using only cheap disks? Assumptions: * I have a place "far away" (a fire won't get there...) on my intranet where I can park a machine with disks hanging off of it. * My "workgroup" world isn't *that* big... Nor is it a nothing-must-ever-be-down-ever-no-matter-what environment. (Even in such an environment, where you will probably RAIDify every disk-bit, you still need backups...) Terminology: `disk chunk' [4] - a logically-coherent chunk of stuff on disk "in one piece" (i.e. under a single directory). Examples: "Fred's home directory" (/home/fred); "where we keep our source tarballs" (/system/open-src); ... [If you have a record of all of your 'disk chunks', you can generate your automount maps (for example).] `replica' - a copy of a disk chunk. The `primary' replica is the one you really use (perhaps read-write); all other (`secondary') replicas of a disk chunk are read-only (if made available at all). So, the idea is very simple: rsync [5] copies of your disk chunks to a remote diskful machine in an organized way. Those remote chunks (replicas) are then available themselves (read-only). Details: * A five-nights-a-week thing rsyncs all live 'disk chunks' to a remote diskful box. This set of (secondary) replicas are available (read-only) through a /yesterday automount map, using the following mapping: primary replica called... secondary called... /home/partain /yesterday/home--partain /system/open-src /yesterday/system--open-src (and so on) * A weekly (every Sunday?) thing rsyncs all live disk chunks to a remote diskful box. These sets of (secondary) replicas are available (read-only) through automount maps /weekend-1, /weekend-2, /weekend-3, and /weekend-4. (If there's a fifth Sunday, I think I just won't bother.) Thus, my home directory at the first weekend of the month: /weekend-1/home--partain * System-y bits (/usr, /, /opt) -- stuff that came straight from the vendor -- are not backed up. (You could do differently, but we're inclined to just rebuild the machine.) That's, er, um, *it*! Use cases: * User loses a file? They can go find it themselves, in /yesterday and/or /weekend-{1,2,3,4}. * Data disk dies? Either: (a) Copy the stuff from /yesterday to some other disk, creating a new primary replica; adjust automount maps accordingly; or (b) Switch affected people's machines' automount maps to point to the /yesterday replica (now read-write :-). Replace disk at leisure and shuffle things about. * System disk dies? (machine now dead) (a) [unlikely] Physical move of the data disk to another machine; adjust, season, enjoy; or (b) For all the primary disk-chunk replicas that were on this machine, as for the data-disk case. Fix machine at leisure. Note: for critical machines, it may be worth having a spare system disk ready to go. * Building burns down? You've got all the bits far away. Finer details: * rsync over ssh: yes * It's better that the "diskful boxes" *pull* the rsync copies, rather than having every client push to them. * Yes, we'll use the rsync 'excludes' mechanism to avoid backing up large recreatable files. * Database-y disk chunks (e.g. Oracle databases, ClearCase VOBs, ...): have enough smarts to wrap the rsyncs in a `lock'/`unlock' pair. * Can 'cut over' from a traditional tape-based backup scheme gradually (a disk or machine at a time...). * Need some kind of tidy-up mechanism: e.g. if a /weekend-1 replica didn't get made correctly on the Sunday, perhaps should re-try on the Monday? (Or just live with a not-quite-right /weekend-1 replica?) Variants: * Have the nightly rsync go to a *local* diskful box; it is then really quite painless to switch a user/system to that copy (no WAN delays). The remote one could then be /day-before-yesterday, taken from /yesterday before it gets refreshed... (Synchronization issues there...) * If you just can't get over that tape feeling, attach a tape drive to the remote diskful boxes, and take/keep copies of the /weekend-1 replicas. Comments?? What have I forgotten/overlooked? Notes: [1] http://roc.cs.berkeley.edu/ [2] `Backing up' solves problems ranging from `I deleted my cookie recipe' to `The building burned down'. [3] `Archiving' takes copies of logical entities (`the project that just finished', `the month-end accounts') to preserve "forever", perhaps for legal reasons. [4] `Disk chunk' (or 'dchunk' [pronounced "duh-*chunk*"]) is an Arusha Project (Sidai team) term. [5] http://rsync.samba.org/ == end From oetiker@ee.ethz.ch Fri Jan 4 07:59:57 2002 Received: from tardis.ee.ethz.ch (tardis-delek-fast.ee.ethz.ch [129.132.2.199]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA30579 for ; Fri, 4 Jan 2002 07:59:56 -0800 Received: from nova.ee.ethz.ch (nova.ee.ethz.ch [129.132.7.86]) by tardis.ee.ethz.ch (Postfix) with ESMTP id 5EAC33C336; Fri, 4 Jan 2002 16:59:54 +0100 (MET) Date: Fri, 4 Jan 2002 16:59:54 +0100 (MET) From: Tobias Oetiker To: Will Partain Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Today Will Partain wrote: > Greetings, infrastructural folks! Great to meet some of > you at LISA. > > I have been musing slightly about doing backups without > tapes, and pass along my current notes (below) for your > comment and criticism. > > And, if you haven't noticed lately, tape is a *pain*. Hi Will, One property I like about tapes very much is that they do not usually die as a whole but rather slowly degrade gracefully ... there is no controler in the tape ... Our take at the problem is, that we provide a HSM system for the whole department (30 TB) based on a Solaris Box with an IBM Robot and Magstar tapedrives. On the solaris box we run LSCIs SAMFS www.lsci.com (recently acquired by sun) this software makes the space on the tapes appear as one big filesystem to the users ... All labs run their backups to this filesystem either via ufsdump or some other means (ntbackup via samba works as well) ... We are very succesful with the service check http://jabba.ethz.ch for stats ... cheers tobi > So, what can we do backup-wise [2] (NB: not > `archiving'... [3]) using only cheap disks? Assumptions: > > * I have a place "far away" (a fire won't get there...) on > my intranet where I can park a machine with disks hanging > off of it. > > * My "workgroup" world isn't *that* big... Nor is it a > nothing-must-ever-be-down-ever-no-matter-what environment. > (Even in such an environment, where you will probably > RAIDify every disk-bit, you still need backups...) > > Terminology: > > `disk chunk' [4] - a logically-coherent chunk > of stuff on disk "in one piece" (i.e. under a single > directory). Examples: "Fred's home directory" (/home/fred); > "where we keep our source tarballs" (/system/open-src); ... > > [If you have a record of all of your 'disk chunks', you > can generate your automount maps (for example).] > > `replica' - a copy of a disk chunk. The `primary' replica > is the one you really use (perhaps read-write); all other > (`secondary') replicas of a disk chunk are read-only (if > made available at all). > > So, the idea is very simple: rsync [5] copies of your disk > chunks to a remote diskful machine in an organized way. > Those remote chunks (replicas) are then available themselves > (read-only). Details: > > * A five-nights-a-week thing rsyncs all live 'disk chunks' > to a remote diskful box. This set of (secondary) replicas > are available (read-only) through a /yesterday automount > map, using the following mapping: > > primary replica called... secondary called... > > /home/partain /yesterday/home--partain > /system/open-src /yesterday/system--open-src > (and so on) > > * A weekly (every Sunday?) thing rsyncs all live disk chunks > to a remote diskful box. These sets of (secondary) > replicas are available (read-only) through automount maps > /weekend-1, /weekend-2, /weekend-3, and /weekend-4. (If > there's a fifth Sunday, I think I just won't bother.) > Thus, my home directory at the first weekend of the > month: /weekend-1/home--partain > > * System-y bits (/usr, /, /opt) -- stuff that came straight > from the vendor -- are not backed up. (You could do > differently, but we're inclined to just rebuild the > machine.) > > That's, er, um, *it*! Use cases: > > * User loses a file? They can go find it themselves, in > /yesterday and/or /weekend-{1,2,3,4}. > > * Data disk dies? Either: (a) Copy the stuff from > /yesterday to some other disk, creating a new primary > replica; adjust automount maps accordingly; or (b) Switch > affected people's machines' automount maps to point to the > /yesterday replica (now read-write :-). Replace disk at > leisure and shuffle things about. > > * System disk dies? (machine now dead) (a) [unlikely] > Physical move of the data disk to another machine; adjust, > season, enjoy; or (b) For all the primary disk-chunk > replicas that were on this machine, as for the data-disk > case. Fix machine at leisure. Note: for critical > machines, it may be worth having a spare system disk ready > to go. > > * Building burns down? You've got all the bits far away. > > Finer details: > > * rsync over ssh: yes > > * It's better that the "diskful boxes" *pull* the rsync > copies, rather than having every client push to them. > > * Yes, we'll use the rsync 'excludes' mechanism to avoid > backing up large recreatable files. > > * Database-y disk chunks (e.g. Oracle databases, ClearCase > VOBs, ...): have enough smarts to wrap the rsyncs in > a `lock'/`unlock' pair. > > * Can 'cut over' from a traditional tape-based backup scheme > gradually (a disk or machine at a time...). > > * Need some kind of tidy-up mechanism: e.g. if a /weekend-1 > replica didn't get made correctly on the Sunday, perhaps > should re-try on the Monday? (Or just live with a > not-quite-right /weekend-1 replica?) > > Variants: > > * Have the nightly rsync go to a *local* diskful box; it is > then really quite painless to switch a user/system to that copy > (no WAN delays). > > The remote one could then be /day-before-yesterday, taken > from /yesterday before it gets > refreshed... (Synchronization issues there...) > > * If you just can't get over that tape feeling, attach a > tape drive to the remote diskful boxes, and take/keep > copies of the /weekend-1 replicas. > > Comments?? What have I forgotten/overlooked? > > Notes: > > [1] http://roc.cs.berkeley.edu/ > > [2] `Backing up' solves problems ranging from `I deleted my > cookie recipe' to `The building burned down'. > > [3] `Archiving' takes copies of logical entities (`the > project that just finished', `the month-end accounts') > to preserve "forever", perhaps for legal reasons. > > [4] `Disk chunk' (or 'dchunk' [pronounced "duh-*chunk*"]) is > an Arusha Project (Sidai team) term. > > [5] http://rsync.samba.org/ > > == end > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures > -- ______ __ _ /_ __/_ / / (_) Oetiker, ETZ J97, ETH, 8092 Zurich, Switzerland / // _ \/ _ \/ / phoneto:+41(0)1-632-5286 faxto:+41(0)1-632-1517 /_/ \.__/_.__/_/ mailto:oetiker@ee.ethz.ch http://people.ee.ethz.ch/~oetiker From don.faulkner@infosec.spectria.com Fri Jan 4 17:29:16 2002 Received: from strider (cdm-37-194.sprd.tcac.net [208.180.37.194]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id RAA12555 for ; Fri, 4 Jan 2002 17:29:08 -0800 Received: from def by strider with local (Exim 3.33 #1 (Debian)) id 16Mfdy-0006Z4-00 for ; Fri, 04 Jan 2002 19:29:06 -0600 Date: Fri, 4 Jan 2002 19:29:06 -0600 From: Don Faulkner To: infrastructures@terraluna.org Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Message-ID: <20020105012906.GB24873@infosec.spectria.com> Mail-Followup-To: infrastructures@terraluna.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.25i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, Jan 04, 2002 at 03:19:53PM +0000, Will Partain wrote: > * A five-nights-a-week thing rsyncs all live 'disk chunks' > to a remote diskful box. This set of (secondary) replicas > are available (read-only) through a /yesterday automount > map, using the following mapping: > > primary replica called... secondary called... > > /home/partain /yesterday/home--partain > /system/open-src /yesterday/system--open-src > (and so on) > The one thing (probably the only thing!) I liked about tapes was the idea of differential and incremental backups. Now, I'm one of these frugal types, in that I prefer not to back up the same bits twice. I prefer diffs, deltas, and other similar mechanisims, especially when there's a convienent (and "intuitive") mechanisim to use them. The trouble with diffs, of course, is that you can't directly access a diff'd file. I seem to remember an experimental file system or something from a few years ago that would do this transparently, so that you could access any revision of a file. I guess now I have a reason to go find that again. -- Don Faulkner, KB5WPM | | "All that is gold does not glitter." ( This space unintentionally | "not all those who wander are lost." left blank ) | --J.R.R. Tolkien From bryang@soccer.fc.hp.com Sat Jan 5 07:32:00 2002 Received: from atlrel7.hp.com (atlrel7.hp.com [156.153.255.213]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id HAA25927 for ; Sat, 5 Jan 2002 07:31:46 -0800 Received: from soccer.fc.hp.com (soccer.fc.hp.com [15.1.42.73]) by atlrel7.hp.com (Postfix) with ESMTP id 4917CE007AA for ; Sat, 5 Jan 2002 10:31:07 -0500 (EST) Received: (from bryang@localhost) by soccer.fc.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id IAA20156 for infrastructures@terraluna.org; Sat, 5 Jan 2002 08:30:52 -0700 (MST) From: Bryan Gartner Message-Id: <200201051530.IAA20156@soccer.fc.hp.com> Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) To: infrastructures@terraluna.org Date: Sat, 05 Jan 2002 8:30:51 MST In-Reply-To: ; from "Will Partain" at Jan 4, 102 12:10 (noon) Organization: Hewlett-Packard Company Entity: Computing Systems IT, Fort Collins Phone: (970 or telnet) 898-2470 X-Url: http://ncs-web.fc.hp.com/~bryang/ X-Mailer: Elm [revision: 212.2] Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Will, > I have been musing slightly about doing backups without > tapes, and pass along my current notes (below) for your > comment and criticism. I also have been toying with this notion for awhile now. An extension to what you provided is to have the remote disk farm using a journal-based file system that can also perform snapshots. In this way, the remote system has views of what changed during each snapshot period, and given enough disk space can be retained according to your desires/budget. Plus, with the right rsyncd.conf (ie. managed files) on both ends, the user could simply "restore" the data from any period themselves. All of this is really intended for an environment like you cited with a good OS/Data distinction, and the intent here was the OS would be rebuilt quicker than could be restored. The only exception would possibly be for files yet to be managed which would be linked from the OS partition into the Data partition. > * Building burns down? You've got all the bits far away. Of course if the network latency becomes a huge barrier and you can afford the luxury of yet another replica, layer either AFS or Coda onto the remote rsync data farm and you can replicate very nicely across the WAN. bryang From dert@eecs.tufts.edu Sat Jan 5 08:12:55 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA14422 for ; Sat, 5 Jan 2002 08:12:55 -0800 Received: from piano.eecs.tufts.edu (Piano.EECS.Tufts.EDU [130.64.23.40]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 2BD671A6FC for ; Sat, 5 Jan 2002 11:12:54 -0500 (EST) Received: by piano.eecs.tufts.edu (Postfix, from userid 13132) id C55A0AF9E; Sat, 5 Jan 2002 11:12:53 -0500 (EST) Date: Sat, 5 Jan 2002 11:12:53 -0500 From: Andy Davidoff To: infrastructures@terraluna.org Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Message-ID: <20020105111253.A11239@eecs.tufts.edu> Reply-To: dert@pobox.com References: <200201051530.IAA20156@soccer.fc.hp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.2i In-Reply-To: <200201051530.IAA20156@soccer.fc.hp.com>; from bryang@soccer.fc.hp.com on Sat, Jan 05, 2002 at 08:30:51AM -0700 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I just met with some salesweasels from NetApp a couple weeks ago. The have a new near-line storage product built from N 160gig EIDE disks. The performance is secondary to capacity in this offering, but it's a commercial implementation of what many of us have been playing with in the back rooms. I think the densities start at around 20tb. Personally, I like the gee-whiz factor of mixed-media filesystems, but they aren't as robust as a filer and don't offer anywhere near the performance and features on the back-end. When I can't justify the NetApp, I'll go with a fast RAID for on-line cache and a DVD changer for near-line. Mix 'em up with the aforementioned LSCI SAM-FS or OTG's competitive product for an archiving filesystem. It all depends on your network; do you need speed and snapshots or do you need raw capacity above all else. #if Bryan Gartner /* Jan 05, 08:30 */ > An extension to what you provided is to have the remote disk farm > using a journal-based file system that can also perform snapshots. > In this way, the remote system has views of what changed during > each snapshot period, and given enough disk space can be retained > according to your desires/budget. Plus, with the right rsyncd.conf > (ie. managed files) on both ends, the user could simply "restore" > the data from any period themselves. #endif /* bryang@soccer.fc.hp.com */ -- Andy Davidoff Sen. Unix SysAdmin Tufts University From joelh@cyberzod.com Mon Jan 7 10:36:22 2002 Received: from cyberzod.com (12-253-25-167.client.attbi.com [12.253.25.167]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id KAA28116 for ; Mon, 7 Jan 2002 10:36:20 -0800 Received: from saber (saber.nabs.com [192.168.128.7]) by cyberzod.com (8.9.3/8.9.3) with SMTP id LAA09928 for ; Mon, 7 Jan 2002 11:36:16 -0700 From: "Joel Huddleston" To: Subject: RE: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Date: Mon, 7 Jan 2002 11:33:09 -0700 Message-ID: <003801c197a9$c1b953a0$0780a8c0@nabs.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0 Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 In-Reply-To: Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Since this specter has been raised, I'd like to vent briefly (if that is within my abilities) on a property of every backup system I have reviewed to date that makes them unacceptable. The crux of my complaint is, backup is useless; restore is invaluable. Yet, every vendor spends 99% effort on backup and 1% (if that) on restore. (OK, in some cases 40% on backup, 65% on marketing and 5% on restore.) My ideal data integrity system is one in which, when I have to recover from disaster, I am not dragging out manuals and screwing around with some arcane restore system. I am not reinstalling the OS and then reinstalling the backup software including some kind of host key and THEN recovering over both of them. A data integrity system should provide: 1) Near zero effort, high speed, full restores without reinstall of ANY software 2) End-user interface for backup now and restore now 3) Low system impact backup 4) Differential, Incremental and Full backups 4a) Incremental backups that understand that: restore v1, v1.1, v1.2 is not the same as restore v1.2 (files deleted between incrementals are universally restored and kept, occasionally filling otherwise low-utilization file systems, grr.) 5) A host level comprehension of backup policies, status and configuration metadata that is available when the host itself is gone (e.g. what disks existed, what were their SCSI/Fiber addresses, how were they partitioned, were they VMed and so on) 6) Encryption on the wire 7) Available encryption on the media using public/private keys (these should be end-user maintainable, selectable by owner, group and/or directory and "secure" from system personnel) In order to satisfy condition #1, the backup vendor must provide bootable media that can perform a full restore to any file system they support for backup. That means FAT, FAT32, NTFS, ufs, 64bit ufs, vfs, jfs and so on. Is anyone aware of such a system? Of course not, it does not exist. I am not even aware of such a system that is available for a single platform from a single vendor, never mind multi-vendor heterogeneity. I have come to the point in my venting where I believe that if I continue, it is no longer brief. I'd really appreciate any comments or corrections the group can provide to my comments above. If anyone can suggest a framework that (even partially) satisfies condition #1 above, please tell us what it is. I'll happily proselytize to the net any solution that comes close. -- Joel Huddleston eka joelh@cyberzod.com > -----Original Message----- > From: infrastructures-admin@terraluna.org > [mailto:infrastructures-admin@terraluna.org]On Behalf Of Will Partain > Sent: Friday, January 04, 2002 8:20 AM > To: infrastructures@terraluna.org > Subject: [Infrastructures] a tapeless backups system for a workgroup > (drafty proposal v1) > > > Greetings, infrastructural folks! Great to meet some of > you at LISA. > > I have been musing slightly about doing backups without > tapes, and pass along my current notes (below) for your > comment and criticism. > > (Once we get beyond the high-level issues that kinda make > sense for the infrastructures list, we should probably chat > elsewhere -- I'd suggest ark-dev > [http://lists.sf.net/mailman/listinfo/ark-dev], as that's > where I'll probably do any detailed implementation chit-chat > (if it happens).) > > Will > > ========================= > > A tapeless backups system for a workgroup > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > > At a LISA 2001 session on "recovery-oriented computing" [1], > Dave Patterson (Berkeley) commented that "tape is dead". > Reason: if tape doesn't cost more per byte than disk yet, it > soon will. Quick check (2002.01): 120GB EIDE disk for $208 > (Pricewatch) = $1.73/GB, 40GB DLT IV tape for $53 > (Pricescan) = $1.32/GB. Close enough for me! > > And, if you haven't noticed lately, tape is a *pain*. > > So, what can we do backup-wise [2] (NB: not > `archiving'... [3]) using only cheap disks? Assumptions: > > * I have a place "far away" (a fire won't get there...) on > my intranet where I can park a machine with disks hanging > off of it. > > * My "workgroup" world isn't *that* big... Nor is it a > nothing-must-ever-be-down-ever-no-matter-what environment. > (Even in such an environment, where you will probably > RAIDify every disk-bit, you still need backups...) > > Terminology: > > `disk chunk' [4] - a logically-coherent chunk > of stuff on disk "in one piece" (i.e. under a single > directory). Examples: "Fred's home directory" (/home/fred); > "where we keep our source tarballs" (/system/open-src); ... > > [If you have a record of all of your 'disk chunks', you > can generate your automount maps (for example).] > > `replica' - a copy of a disk chunk. The `primary' replica > is the one you really use (perhaps read-write); all other > (`secondary') replicas of a disk chunk are read-only (if > made available at all). > > So, the idea is very simple: rsync [5] copies of your disk > chunks to a remote diskful machine in an organized way. > Those remote chunks (replicas) are then available themselves > (read-only). Details: > > * A five-nights-a-week thing rsyncs all live 'disk chunks' > to a remote diskful box. This set of (secondary) replicas > are available (read-only) through a /yesterday automount > map, using the following mapping: > > primary replica called... secondary called... > > /home/partain /yesterday/home--partain > /system/open-src /yesterday/system--open-src > (and so on) > > * A weekly (every Sunday?) thing rsyncs all live disk chunks > to a remote diskful box. These sets of (secondary) > replicas are available (read-only) through automount maps > /weekend-1, /weekend-2, /weekend-3, and /weekend-4. (If > there's a fifth Sunday, I think I just won't bother.) > Thus, my home directory at the first weekend of the > month: /weekend-1/home--partain > > * System-y bits (/usr, /, /opt) -- stuff that came straight > from the vendor -- are not backed up. (You could do > differently, but we're inclined to just rebuild the > machine.) > > That's, er, um, *it*! Use cases: > > * User loses a file? They can go find it themselves, in > /yesterday and/or /weekend-{1,2,3,4}. > > * Data disk dies? Either: (a) Copy the stuff from > /yesterday to some other disk, creating a new primary > replica; adjust automount maps accordingly; or (b) Switch > affected people's machines' automount maps to point to the > /yesterday replica (now read-write :-). Replace disk at > leisure and shuffle things about. > > * System disk dies? (machine now dead) (a) [unlikely] > Physical move of the data disk to another machine; adjust, > season, enjoy; or (b) For all the primary disk-chunk > replicas that were on this machine, as for the data-disk > case. Fix machine at leisure. Note: for critical > machines, it may be worth having a spare system disk ready > to go. > > * Building burns down? You've got all the bits far away. > > Finer details: > > * rsync over ssh: yes > > * It's better that the "diskful boxes" *pull* the rsync > copies, rather than having every client push to them. > > * Yes, we'll use the rsync 'excludes' mechanism to avoid > backing up large recreatable files. > > * Database-y disk chunks (e.g. Oracle databases, ClearCase > VOBs, ...): have enough smarts to wrap the rsyncs in > a `lock'/`unlock' pair. > > * Can 'cut over' from a traditional tape-based backup scheme > gradually (a disk or machine at a time...). > > * Need some kind of tidy-up mechanism: e.g. if a /weekend-1 > replica didn't get made correctly on the Sunday, perhaps > should re-try on the Monday? (Or just live with a > not-quite-right /weekend-1 replica?) > > Variants: > > * Have the nightly rsync go to a *local* diskful box; it is > then really quite painless to switch a user/system to that copy > (no WAN delays). > > The remote one could then be /day-before-yesterday, taken > from /yesterday before it gets > refreshed... (Synchronization issues there...) > > * If you just can't get over that tape feeling, attach a > tape drive to the remote diskful boxes, and take/keep > copies of the /weekend-1 replicas. > > Comments?? What have I forgotten/overlooked? > > Notes: > > [1] http://roc.cs.berkeley.edu/ > > [2] `Backing up' solves problems ranging from `I deleted my > cookie recipe' to `The building burned down'. > > [3] `Archiving' takes copies of logical entities (`the > project that just finished', `the month-end accounts') > to preserve "forever", perhaps for legal reasons. > > [4] `Disk chunk' (or 'dchunk' [pronounced "duh-*chunk*"]) is > an Arusha Project (Sidai team) term. > > [5] http://rsync.samba.org/ > > == end > _______________________________________________ > Infrastructures mailing list > Infrastructures@mailman.terraluna.org > http://mailman.terraluna.org/mailman/listinfo/infrastructures > From cwcjr@bestweb.net Tue Jan 8 12:16:23 2002 Received: from pimout3-int.prodigy.net (pimout3-ext.prodigy.net [207.115.63.102]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA12128 for ; Tue, 8 Jan 2002 12:16:22 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout3-int.prodigy.net (8.11.0/8.11.0) with SMTP id g08KGKH248282; Tue, 8 Jan 2002 15:16:20 -0500 Message-ID: <000701c19881$56908700$c5c0a8c0@prodigy.com> From: "Will Cox" To: "Joel Huddleston" , References: <003801c197a9$c1b953a0$0780a8c0@nabs.com> Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) Date: Tue, 8 Jan 2002 15:16:20 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Just ran across this item addressing bare metal restore problems: http://www.tkg.com/products/baremetal/baremetal.php /cwc From: "Joel Huddleston" > The crux of my complaint is, backup is > useless; restore is invaluable. From duprec@scorec.rpi.edu Tue Jan 8 12:55:08 2002 Received: from mail.rpi.edu (root@mail.rpi.edu [128.113.22.40]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA32423 for ; Tue, 8 Jan 2002 12:55:07 -0800 Received: from smtp.scorec.rpi.edu (smtp.scorec.rpi.edu [128.113.123.156]) by mail.rpi.edu (8.11.3/8.11.3) with ESMTP id g08Kt4543554 for ; Tue, 8 Jan 2002 15:55:05 -0500 Received: from mastermind.scorec.rpi.edu (mastermind.scorec.rpi.edu [10.0.5.17]) by smtp.scorec.rpi.edu (8.10.2/8.10.2) with ESMTP id g08KssK13641 for ; Tue, 8 Jan 2002 15:54:54 -0500 (EST) Date: Tue, 8 Jan 2002 15:54:54 -0500 (EST) From: Christophe Dupre To: In-Reply-To: <200201082001.MAA04494@roton.terraluna.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Scanned-By: MIMEDefang 2.1 Subject: [Infrastructures] Re: Infrastructures digest, Vol 1 #6 - 2 msgs Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: While having a backup system that does everything you list in your mail below would be nice, I don't know how much of a real gain you'd get. > From: "Joel Huddleston" > To: > Subject: RE: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) > Date: Mon, 7 Jan 2002 11:33:09 -0700 > > Since this specter has been raised, I'd like to vent briefly (if that is > within my abilities) on a property of every backup system I have reviewed to > date that makes them unacceptable. The crux of my complaint is, backup is > useless; restore is invaluable. Yet, every vendor spends 99% effort on > backup and 1% (if that) on restore. (OK, in some cases 40% on backup, 65% on > marketing and 5% on restore.) > > My ideal data integrity system is one in which, when I have to recover from > disaster, I am not dragging out manuals and screwing around with some arcane > restore system. I am not reinstalling the OS and then reinstalling the > backup software including some kind of host key and THEN recovering over > both of them. A data integrity system should provide: > > 1) Near zero effort, high speed, full restores without reinstall of ANY > software Use ufsdump. Boot from the network or cdrom, and ufsrestore. It may not be pretty, or user-friendly, but it will get the job done. > 2) End-user interface for backup now and restore now Don't most backup software do that already ? > 3) Low system impact backup What do you mean by low-impact ? > 4) Differential, Incremental and Full backups That's also very common. > 4a) Incremental backups that understand that: restore v1, v1.1, v1.2 is not > the same as restore v1.2 (files deleted between incrementals are universally > restored and kept, occasionally filling otherwise low-utilization file > systems, grr.) Netbackup's True Image Recovery feature does that, at the expense of disk space for a database. > 5) A host level comprehension of backup policies, status and configuration > metadata that is available when the host itself is gone (e.g. what disks > existed, what were their SCSI/Fiber addresses, how were they partitioned, > were they VMed and so on) That's not for the backup system, that's for your rebuilding infrastructure. The backup should be media-independant, as you want to be able to restore to a different machine the data was backed up from. That's the major gripe I have with NAS and the NDP protocol - not device-independant. In a controlled environments, it's quicker to just rebuild a box than to try to recover the OS from tape. In our case here, that rebuilding includes the backup client. Once the box is rebuilt, all the data can be recovered from tape from the backup server. If it's the backup server that died, well we can take the few more hours required to recover the backup system from backup with a clean copy, as that machine doesn't do anything else. As for the efforts from vendor - we spend most of our time doing backup, and comparatively little time doing restores, so it follows that that's where they spend their time. For the encryption, I agree with you - I haven't found a system that does decent encryption with any reasonable amount of ease of administration. Work is definitely required in this area. -- Christophe Dupre System Administrator, Scientific Computation Research Center Rensselaer Polytechnic Institute Troy, NY USA Phone: (518) 276-2578 - Fax: (518) 276-4886 From jmr@computing.com Tue Jan 8 13:13:55 2002 Received: from zuum.computing.com (cs2869-253.austin.rr.com [24.28.69.253]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA09982 for ; Tue, 8 Jan 2002 13:13:54 -0800 From: jmr@computing.com Received: from zuum.computing.com.computing.com (localhost [127.0.0.1]) by zuum.computing.com (8.11.6/8.11.6) with ESMTP id g08LDgO26303; Tue, 8 Jan 2002 15:13:43 -0600 (CST) (envelope-from jmr@computing.com) Date: Tue, 08 Jan 2002 15:13:42 -0600 Message-ID: <87hepwk0l5.wl@zuum.computing.com> To: "Joel Huddleston" Cc: Subject: Re: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) In-Reply-To: <003801c197a9$c1b953a0$0780a8c0@nabs.com> References: <003801c197a9$c1b953a0$0780a8c0@nabs.com> User-Agent: Wanderlust/2.4.1 (Stand By Me) SEMI/1.13.7 (Awazu) FLIM/1.13.2 (Kasanui) MULE XEmacs/21.1 (patch 14) (Cuyahoga Valley) (i386--freebsd) MIME-Version: 1.0 (generated by SEMI 1.13.7 - "Awazu") Content-Type: text/plain; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Joel: I think your rant is right on target! I don't personally know anything about this one... but: In today's paper there was an article about TKG (The Kernel Group) having been acquired by someone. TKG has apparently developed something they call "Bare Metal Restore". Sounds like they're aiming for the same target... You might check it out. TKG is in Austin. -- Jim Rowan jmr@computing.com DCSI (512) 374-1143 From partain@dcs.gla.ac.uk Wed Jan 9 09:00:50 2002 Received: from iona.dcs.gla.ac.uk (iona.dcs.gla.ac.uk [130.209.240.35]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id JAA08474 for ; Wed, 9 Jan 2002 09:00:49 -0800 Received: from slicker.dcs.gla.ac.uk ([130.209.242.51] helo=dcs.gla.ac.uk) by iona.dcs.gla.ac.uk with esmtp (Exim 3.13 #1) id 16OM5o-0003LU-00 for infrastructures@terraluna.org; Wed, 09 Jan 2002 17:00:48 +0000 To: infrastructures@terraluna.org X-Mailer: mh-e 6.0; nmh 1.0.4; Emacs 21.4 Date: Wed, 09 Jan 2002 17:00:47 +0000 From: Will Partain Message-Id: Subject: [Infrastructures] Re: a tapeless backups system ... Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Folks, thanks for the many useful comments that have come in re my "tapeless backups proposal" (prev msg). It is my faint hope to consolidate the info and have it appear on a Web page (probably in Arusha land); I will contact authors before stealing their words :-) But to back up a bit (as it were)... At least one shy reader took me to task for my numbers, asserting that $/GB for tape is still ~3x cheaper than disk. (Yes, I admit it, I don't know what I'm talking about :-) Rather than argue about that (though good references welcome!), perhaps it would be useful to collect a few back-of-envelope *sketches* of what people would do for backups, assuming they were starting fresh today... Something like... So, for the scenario I've been thinking about: 300GB of non-system data to be backed up; Shopping list: Price Items $3500 15 x 120GB cheap disks = 300GB (approx) x 5 backup copies $2000 a few PCs to hang them on Park the PCs+disks at my pal's place on the intranet and "backup" using rsync+ssh. The nightly rsync available as /yesterday, and four weekend ones are available as /weekend-[1234]. If some of these are forthcoming, I will add them to the putative document. Thanks as ever, Will From dert@eecs.tufts.edu Wed Jan 9 11:25:03 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id LAA18643 for ; Wed, 9 Jan 2002 11:25:03 -0800 Received: from sensei.EECS.Tufts.EDU (Sensei.EECS.Tufts.EDU [130.64.23.173]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 658191A50B; Wed, 9 Jan 2002 14:25:02 -0500 (EST) Received: by sensei.EECS.Tufts.EDU (Postfix, from userid 13132) id 07028F6123; Wed, 9 Jan 2002 14:25:01 -0500 (EST) Date: Wed, 9 Jan 2002 14:25:01 -0500 From: Andy Davidoff To: infrastructures@terraluna.org Subject: Re: [Infrastructures] Re: a tapeless backups system ... Message-ID: <20020109142501.A26020@eecs.tufts.edu> Reply-To: andy+infrastructures@cs.tufts.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from partain@dcs.gla.ac.uk on Wed, Jan 09, 2002 at 05:00:47PM +0000 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I'm still seeing some rough edges in this system... With tape, I can align full-backups on tape boundaries so archiving a full-backup for archival storage is easy; just swap n tapes from the library. With a disk-based solution, I'd need to build a separate removable array specifically for each autonomous replica. If you use a cheap disk array for backups, be prepared to lose a disk in the array. You should also be prepared for someone to ask why they are spending the extra bucks to put random-access media in storage. If you never archive backups, this isn't an issue. If you do, I can't see why you wouldn't want to stick with tape. At that point, the disk array becomes most relevant as a snapshot server. Ideally, /my/ only backups are the periodic tape archives I create for off-site storage. The rest of my backup needs are handled by snapshots. If you turn this proposal into the engineering of a "free" snapshot server, I think you'll find much more support and a LISA paper or two to boot. #if Will Partain /* Jan 09, 17:00 */ > So, for the scenario I've been thinking about: > > 300GB of non-system data to be backed up; Shopping list: > Price Items > $3500 15 x 120GB cheap disks = 300GB (approx) x 5 backup copies > $2000 a few PCs to hang them on > > Park the PCs+disks at my pal's place on the intranet and "backup" > using rsync+ssh. The nightly rsync available as /yesterday, > and four weekend ones are available as /weekend-[1234]. #endif /* partain@dcs.gla.ac.uk */ -- Andy Davidoff Sen. Unix SysAdmin Tufts University From luke@madstop.com Wed Jan 30 21:00:41 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA05365 for ; Wed, 30 Jan 2002 21:00:40 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g0V4uEH8025456; Wed, 30 Jan 2002 22:56:14 -0600 (CST) Date: Wed, 30 Jan 2002 22:56:14 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Joel Huddleston cc: infrastructures@terraluna.org Subject: RE: [Infrastructures] a tapeless backups system for a workgroup (drafty proposal v1) In-Reply-To: <003801c197a9$c1b953a0$0780a8c0@nabs.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 7 Jan 2002, Joel Huddleston wrote: > 1) Near zero effort, high speed, full restores without reinstall of ANY > software [SNIP] > In order to satisfy condition #1, the backup vendor must provide bootable > media that can perform a full restore to any file system they support for > backup. That means FAT, FAT32, NTFS, ufs, 64bit ufs, vfs, jfs and so on. > > Is anyone aware of such a system? Of course not, it does not exist. I am > not even aware of such a system that is available for a single platform from > a single vendor, never mind multi-vendor heterogeneity. I know I'm jumping onto the end of a long-gone train here, but I just noticed this thread... I don't have much comment about most of your rant, largely because I agree and have been mostly able to avoid the backup monkey, but HP-UX actually can make a bootable tape ofa system. I can't personally vouch for its efficacy, because I haven't used it, but considering how pervasive it appears to be within the HP-UX "community" (I'm not convinced there actually is one), it's apparently pretty decent. You do what they call a "make_recovery", which makes a bootable tape which will restore "the entire system". I'm assuming that there are some limits to this definition, because most tapes won't store what we would consider the entire system, but I don't know how hit draws the lines. I'm assuming you can decide that at creation time. I know my shop is in the habit of doing this on maintenance weekends, and everytime I mention Ignite-UX (HP's install tool, kind of a bloated, insecure, GUI version of jumpstart with a bunch of stuff that makes it hard to use), people think I am only talking about make_recovery. In fact, I recently had some trouble convincing some relatively high-level HP techs that Ignite-UX could even do bare-metal HP-UX installs. The point is, though, HP-UX has such a tool, or something similar, and many people who use HP-UX appear to use the tool. Obviously, though, its cross-platform ability is probably nil. And, of course, one of the biggest failings of the product is that it depends on being able to dump to a single tape. It would be best if make_recovery worked from within a backup program to create a bootable tape of a system _from_backups_, so that you didn't have to make a recovery tape unless you actually needed to recover. -- Tobacco stocks have taken a big tumble," says Jay Leno. "Phillip Morris fell 6 points. They lost so much money they may have to lay off two senators." From luke@madstop.com Wed Jan 30 21:42:53 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id VAA27976 for ; Wed, 30 Jan 2002 21:42:52 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g0V5cXYm025553 for ; Wed, 30 Jan 2002 23:38:33 -0600 (CST) Date: Wed, 30 Jan 2002 23:38:33 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] Sysadmin collaboration tool Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi all, Since I've broken the ice with my previous comment on tape recovery, I figured I'd go ahead and toss this out. I spoke with some of you about this at LISA, but for the rest of you... Steve and I ran a BoF together at LISA, discussing two basic themes. The reason we ran it together was because he and I realized that we had scheduled two different BoFs for the same time, we each wanted to go to the other's, and there seemed to be a significant amount of cross-over. One of the topics, unsurprisingly, was directly related to this list--the next version of isconf, and basic infrastructure concepts. The other topic, my topic, was building an online sysadmin collaboration tool. I use the word tool here because I don't really have a better word. To make a long story short, here is a conceptual summary of what I am looking for: ---------------------- We propose to build an online tool with the specific purpose of providing a means for sysadmins to collaborate on all aspects of system administration. This document is not inclusive and will be updated as the tool is developed. The assumptions which provide the premise for this online tool are the following: -Most problems we as sysadmins encounter are problems that most sysadmins encounter -The current state of system administration is limited by the lack of pervasive collaboration between sysadmins -The current lack of collaboration largely means that collaboration is done only by those with significant technial ability, the time and ability to document, and confidence enough to publish -The work of a sysadmin can be sufficiently separated from the IP of the company s/he works for that it is worth publishing -The derivative content of system administration is comprises multiple disparate types, such as code, best practices, documentation, and theory; a tool which only addresses a subset of this non-comprehensive list is inadequate. Because specific tools or code we use are only the final results of all of the planning and methodologies which make system administration difficult, this collaboration must be specifically developed for general information, methodologies, best practices, and implementation specifics, in addition to the actual code. In addition to providing a forum for the sharing of system administration related tools and information, this tool must also provide a method for the best, as determined by an as-yet undecided metric, content and authors of content to rise to the top. In short, this tool should provide a place for sysadmins to call their home, totally indispensable to their jobs but also engendering a greater sense of community among sysadmins. ------------------------- One of the main topics of discussion at our BoF was whether it made sense to try to overlay or underlay this tool into or onto the infrastructures web site and/or mailing list. The only conclusions we could come to were that most people on this list would at least be interested, and thus it should be brought up here, and also that it is the list who should answer that question, not Steve or I. My personal perspective is that the goal of this tool (and if anyone can come up with a better word or phrase for it, that would be great) is orthogonal to the infrastructure process as outlined in the bootstrapping paper. That is, the goal of this tool is to engender collaboration among sysadmins in all ways and make it easier to become a good sysadmin, whereas the goal of the infrastructure process is to take advantage of the best sysadmin tools and practices out there to actually implement a specific type of system. They definitely would create a nice feedback loop, and certainly there would be many people who were heavily involved in both, but I'm not really convinced there is a sufficient overlap. I have been contemplating such a tool for about two years now, and have not begun it for a number of reasons, largely related to laziness and not quite enough hubris to get in that far over my head. I've reached the point now, though, where I'm ready to commit to getting this out there, because I want it, I know a bunch of other people who would be interested in it, and I basically smack myself for every day it doesn't exist. Content I know that the above summary lacked any real details, and that was purposeful. I don't have a clear idea of what I want in terms of implementation, I just know a number of the high-level features that I want. One of the main differences I want between this and most of the other sysadmin sites I see out there is that I want this to be a place for sysadmins to stage their original content. I haven't been following advogato.com much, but from what I can tell, anyone can post an article there. I want something like that--any sysadmin can wander up and say "Hey, I've got some great docs for this site", post them, and let the chips fall. What I don't want is just a link fest with maybe some comments. Links are easy, but if all you do is link, then it's the sysadmin's responsibility to provide the page in the first place, and that's too high of a barrier of entry, too high of a maintenance cost, and collaboration is far too difficult. Modification Another thing that I definitely want is to provide other sysadmins the ability to basically patch someone's content. If someone writes something and you have something simple to add, either send your comments in as a patch, or have a content fork and maintain your own content if the original author won't accept your patch. By content here I mean code, documentation, a white paper on best practices, basically anything that provides a window into your sysadmin head. Scoring As mentioned in the summary, another thing I think will be extremely important, once this site has gained momentum, is the ability to score both authors and content. I believe this could easily be the hardest part of the project. Tentative plans Because I am as lazy as they come, and I recently noticed that SlashCode now has a Wiki plugin, I am planning on using it as the initial basis of this tool. I'm in the process of getting it all installed and working now, so I haven't actually looked at the slashcode yet, and thus can't really comment on its suitability. Anyone's comments would be appreciated. So, my questions to this list are: 1) Should I take this discussion somewhere else? 2) If not, should we discuss this long enough to get something more concrete and then take it somewhere else? 3) Does anyone have a better word than "tool"? 4) Is the lack of a good domain a restriction here? I own madstop.com and am planning on using that until something better comes up, and I have a couple other ideas, but none of them really shout out at me. I basically believe that the entry point is important, and so I'd like something that everyone agreed was decent. 5) Does anyone have any great ideas as to how to implement this, either conceptually, technically, or somewhere in between? 6) Finally, does anyone want to work with me on this? I've got a box on the end of a largely empty T1, ready to go. It's not a great machine, and if this tool suddenly makes the T1 very non-empty, then I might be in trouble, but for now I'm not worried. If people think it would be a good idea, I could write up a longer, more formal mission statement/white paper on what I am trying to do and why and all that jazz. At the very least, I would like any discussion process to come up with a semi-formal statement of goals, because unlike a plain chunk of software, this isn't just a question of straight feature sets and user interfaces. It's all that and a good bit more. And frankly, any piece of software should also have such a formal statement of goals... If you've read this far, you're probably interested, so I'd love to read your feedback. Luke Kanies -- "A computer lets you make more mistakes faster than any invention in human history--with the possible exceptions of handguns and tequila." -- Mitch Ratcliffe From mbarr@mbarr.net Wed Jan 30 23:32:22 2002 Received: from neuromancer.mbarr.net (postfix@66-108-143-133.nyc.rr.com [66.108.143.133]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id XAA19303 for ; Wed, 30 Jan 2002 23:32:21 -0800 Received: from [192.168.0.10] (unknown [192.168.0.10]) by neuromancer.mbarr.net (Postfix) with ESMTP id 3B7142AF428 for ; Thu, 31 Jan 2002 02:32:13 -0500 (EST) User-Agent: Microsoft-Entourage/10.0.0.1331 Date: Thu, 31 Jan 2002 02:32:10 -0500 From: Matthew Barr To: Message-ID: Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: [Infrastructures] Webmin... Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Short note: ran across an interesting tool that seems to work on half a zillion OS's, with some degree of accuracy, called webmin. It's an interesting piece of work... www.webmin.com Scaling would be hard, but a lot of the code could be very useful for controlling systems elsewhere. Matthew _______________________________________________________________________ Matthew Barr mailto:mhb8@cornell.edu AIM: MBarr1244 ICQ: 22130424 PGP Key Fingerprint = 35DC DC87 4F38 2E80 F327 2B50 FD82 A2CB CB80 80F3 From oddbjorn@tricknology.org Thu Jan 31 04:36:25 2002 Received: from home.tricknology.org (213-187-162-20.dd.nextgentel.com [213.187.162.20]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id EAA12012 for ; Thu, 31 Jan 2002 04:36:23 -0800 Received: by home.tricknology.org (Postfix, from userid 1000) id 1DFAF3EAC; Thu, 31 Jan 2002 13:35:36 +0100 (CET) Date: Thu, 31 Jan 2002 13:35:35 +0100 From: Oddbjorn Steffensen To: "Luke A. Kanies" Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020131123535.GA12474@tricknology.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.3.24i Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Wed, Jan 30, 2002 at 11:38:33PM -0600, Luke A. Kanies wrote: > We propose to build an online tool with the specific purpose of > providing a means for sysadmins to collaborate on all aspects of system > administration. This document is not inclusive and will be updated > as the tool is developed. Have you looked at the following URLs: - Sysadmin Book of Knowledge: http://ace.delos.com/taxongate - Geoff Halprins SA Body of Knowledge: http://www.sysadmin.com.au/sa-bok/ While neither is nowhere near complete, they offer interesting ways to categorize the information, and could be used as a framework for a colloborative tool. -oddbjorn From dert@eecs.tufts.edu Thu Jan 31 12:32:55 2002 Received: from yogi.eecs.tufts.edu (IDENT:postfix@Yogi.EECS.Tufts.EDU [130.64.23.171]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA00452 for ; Thu, 31 Jan 2002 12:32:54 -0800 Received: from sensei.EECS.Tufts.EDU (Sensei.EECS.Tufts.EDU [130.64.23.173]) by yogi.eecs.tufts.edu (Postfix) with ESMTP id 106CE1A504; Thu, 31 Jan 2002 15:32:48 -0500 (EST) Received: by sensei.EECS.Tufts.EDU (Postfix, from userid 13132) id 560FDF611B; Thu, 31 Jan 2002 15:32:47 -0500 (EST) Date: Thu, 31 Jan 2002 15:32:47 -0500 From: Andy Davidoff To: "Luke A. Kanies" Cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020131153246.H19406@eecs.tufts.edu> Reply-To: andy+infrastructures@cs.tufts.edu References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from luke@madstop.com on Wed, Jan 30, 2002 at 11:38:33PM -0600 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: #if Luke A. Kanies /* Jan 30, 23:38 */ > As mentioned in the summary, another thing I think will be extremely > important, once this site has gained momentum, is the ability to > score both authors and content. After the BoF, I tried to outline the issues and options that immediately came to my mind. This WIP would benefit from list-member input: http://www.eecs.tufts.edu/~dert/infrastructures/metrics.html I have only listed three scoring designs (Slashdot, Clever/Google, and Firefly/Joker) and only in the broadest sense. I think we should be able to come up with a design that has most of the benefits and few of the weaknesses of these others. This document is nearly two months old, now and I know it won't receive any more attention from me without attention from the community. I'm missing details and perspective; I'm hoping to receive those from the list. > I'm in the process of getting it all installed and working now, so I > haven't actually looked at the slashcode yet, and thus can't really > comment on its suitability. I often find that momentum is more important than quality when first starting a large project. So, good work. Let's see where SlashCode takes us. If nothing else, it will help us identify what we need and what we're missing. > 1) Should I take this discussion somewhere else? > 2) If not, should we discuss this long enough to get something more > concrete and then take it somewhere else? I'm here post-LISA, so I can't tell you how off-topic you are. :-) > 3) Does anyone have a better word than "tool"? Resource, for starters. > 4) Is the lack of a good domain a restriction here? Not to members of OUR community. > 5) Does anyone have any great ideas as to how to implement this, > either conceptually, technically, or somewhere in between? A passionate yet formal charter could draw more developers. I'd say this is the most important next-step. > 6) Finally, does anyone want to work with me on this? #endif /* luke@madstop.com */ The scoring system is most interesting to me because I've never seen it done /really/ well, though I believe the technology is there and the problem isn't particularly difficult, though it is not purely technical. -- Andy Davidoff Sen. Unix SysAdmin Tufts University From cwcjr@bestweb.net Thu Jan 31 13:39:33 2002 Received: from pimout4-int.prodigy.net (pimout4-ext.prodigy.net [207.115.63.103]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA01736 for ; Thu, 31 Jan 2002 13:39:32 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout4-int.prodigy.net (8.11.0/8.11.0) with SMTP id g0VLdON89858; Thu, 31 Jan 2002 16:39:25 -0500 Message-ID: <007c01c1aa9f$c12da2c0$c5c0a8c0@prodigy.com> From: "Will Cox" To: "Luke A. Kanies" , References: Subject: Re: [Infrastructures] Sysadmin collaboration tool Date: Thu, 31 Jan 2002 16:39:24 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-Mimeole: Produced By Microsoft MimeOLE V5.50.4910.0300 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > One of the topics, unsurprisingly, was directly related to this list--the > next version of isconf, and basic infrastructure concepts. The other > topic, my topic, was building an online sysadmin collaboration tool. > You've made some excellent points. And they tie in to a lot of what I've been thinking about since I had to manage a Notes environment. System Administration is just one particular field that needs better collaborative tools, which is what gives things like Notes, Groove, and Wiki something to do. The buzzword-compliant fields that deal with this are groupware, document management, content management, and knowlege management. Each tries to corral the problem of too much information -- while at the same time contributing to the problem by attempting to make it easy to exchange information. Jon Udell http://udell.roninhouse.com/ wrote a good paper on the subject during the course of the Software Carpentry contest at CodeSourcery. At my last position, we had lots of ways to share information: 1) the FAQ-O-Matic, 2) Wiki, 3) internal document repository built on Oracle with a web front-end, 4) Bugzilla, 5) flat html, 6) mailing lists. Nobody really put information into the documentation systems unless they were "prompted" to do so. It seemed mailing lists and Wiki yielded the best results, as the barriers to entry were the lowest. At my position now, information is hoarded, and has difficulty crossing functional boundaries. I'm finding it very easy to capture data from RSS newsfeeds and then post commentary with Radio Userland http://radio.userland.com/ . One of the UserLand staff has started a discussion on knowledge management using weblogs. I'm not certain that this _manages_ knowledge yet, but it does seem to stimulate the easy exchange of pertinent information. /cwc http://users.bestweb.net/~cwcjr/ From cwcjr@bestweb.net Thu Jan 31 13:49:10 2002 Received: from pimout4-int.prodigy.net (pimout4-ext.prodigy.net [207.115.63.103]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA07083 for ; Thu, 31 Jan 2002 13:49:09 -0800 Received: from wcox (wcox.prodigy.com [192.168.192.197]) by pimout4-int.prodigy.net (8.11.0/8.11.0) with SMTP id g0VLn1N50924; Thu, 31 Jan 2002 16:49:01 -0500 Message-ID: <008701c1aaa1$18e34a50$c5c0a8c0@prodigy.com> From: "Will Cox" To: , "Luke A. Kanies" Cc: References: <20020131153246.H19406@eecs.tufts.edu> Subject: Re: [Infrastructures] Sysadmin collaboration tool Date: Thu, 31 Jan 2002 16:49:02 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-Mimeole: Produced By Microsoft MimeOLE V5.50.4910.0300 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > http://www.eecs.tufts.edu/~dert/infrastructures/metrics.html > > I have only listed three scoring designs (Slashdot, Clever/Google, and > Firefly/Joker) and only in the broadest sense. I think we should be > able to come up with a design that has most of the benefits and few of > the weaknesses of these others. There's also Advogato, which uses group trust metrics for peer certification http://www.advogato.org/mission.html , thus avoiding poisoning problems that affect /. and others. I personally find it very difficult to pay much attention to a full /. thread: too much noise. /cwc From tim@techfocus.net Thu Jan 31 13:56:59 2002 Received: from gonzo.techfocus.net (dsl092-066-027.bos1.dsl.speakeasy.net [66.92.66.27]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA11784 for ; Thu, 31 Jan 2002 13:56:58 -0800 Received: from localhost (tim@localhost) by gonzo.techfocus.net (8.9.3+Sun/8.9.3) with ESMTP id QAA22257 for ; Thu, 31 Jan 2002 16:56:43 -0500 (EST) Date: Thu, 31 Jan 2002 16:56:43 -0500 (EST) From: Tim Quinlan To: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: <008701c1aaa1$18e34a50$c5c0a8c0@prodigy.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: I saw an intersting site called alterslash.org It's an unofficial slashdot digest. They only show a few highly moderated comments for each article, plus the have a signal-to-noise meter for the comments. On Thu, 31 Jan 2002, Will Cox wrote: > I personally find it very difficult to pay much attention to a full /. > thread: too much noise. From luke@madstop.com Fri Feb 1 08:23:49 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA30089 for ; Fri, 1 Feb 2002 08:23:48 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g11GIbec000276; Fri, 1 Feb 2002 10:18:38 -0600 (CST) Date: Fri, 1 Feb 2002 10:18:37 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Andy Davidoff cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: <20020131153246.H19406@eecs.tufts.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Thu, 31 Jan 2002, Andy Davidoff wrote: > After the BoF, I tried to outline the issues and options that immediately > came to my mind. This WIP would benefit from list-member input: > > http://www.eecs.tufts.edu/~dert/infrastructures/metrics.html Great, I'll try to send my patch to it soon. > I often find that momentum is more important than quality when first > starting a large project. So, good work. Let's see where SlashCode > takes us. If nothing else, it will help us identify what we need and > what we're missing. I agree, which why I'll point out the following: I kind of picked a bad week to start this topic, since Steve T. is actually here in Nashville doing some consulting for my company, which means that I'm working an extra-long week. Even worse, I'm leaving town this weekend in hopes of finding a car in Atlanta (because there apparently aren't any here). So, I went and started this discussion but won't really have opportunity to make much meaningful input for a while. I'll definitely be collating over the weekend, though, and I plan to continue my contributions early next week. > > 4) Is the lack of a good domain a restriction here? > > Not to members of OUR community. Okay, then for fear of overloading the term infrastructure, I am going to use madstop.com as the root of this, unless someone else has a good reason not to or an even better URL. I've been meaning to spend a bit doing a brainstorming session, trying to come up with something good, but haven't sat down to do it yet. I keep thinking (sys)admin + meta + collection + some other stuff. > > 5) Does anyone have any great ideas as to how to implement this, > > either conceptually, technically, or somewhere in between? > > A passionate yet formal charter could draw more developers. I'd > say this is the most important next-step. I agree with this, and it will be my main goal next week. I fully expect it to be patched many times, but my hope is to get something out and published next week. I'll start with the blurb from my first email, and see if I can flesh it out a bit, including some of the information from your post. > The scoring system is most interesting to me because I've never seen > it done /really/ well, though I believe the technology is there and > the problem isn't particularly difficult, though it is not purely > technical. I definitely agree with this, especially since we are talking about grading so many different types of information. I mean, who's to say that just because you trust someone's opinion on code you should also trust their opinion on general methodologies? Ugh. -- "Did you know that black paint is an excellent stain remover?" - Dogbert From luke@madstop.com Fri Feb 1 08:39:39 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id IAA05719 for ; Fri, 1 Feb 2002 08:39:38 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g11GYUgP000336; Fri, 1 Feb 2002 10:34:30 -0600 (CST) Date: Fri, 1 Feb 2002 10:34:29 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Will Cox cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: <007c01c1aa9f$c12da2c0$c5c0a8c0@prodigy.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Thu, 31 Jan 2002, Will Cox wrote: > You've made some excellent points. And they tie in to a lot of what I've > been thinking about since I had to manage a Notes environment. System > Administration is just one particular field that needs better collaborative > tools, which is what gives things like Notes, Groove, and Wiki something to > do. I certainly agree that we are not the only field lacking in good online collaboration. I guess what galls me the most is that it seems like we should be leading the conceptual charge, because we are exactly the people capable of doing so. > At my last position, we had lots of ways to share information: 1) the > FAQ-O-Matic, 2) Wiki, 3) internal document repository built on Oracle with a > web front-end, 4) Bugzilla, 5) flat html, 6) mailing lists. Nobody really > put information into the documentation systems unless they were "prompted" > to do so. It seemed mailing lists and Wiki yielded the best results, as the > barriers to entry were the lowest. At my position now, information is > hoarded, and has difficulty crossing functional boundaries. Interesting. That's one of the things that I've been thinking a lot about, how to keep a low barrier of entry. I also want to do what I can to maintain the concept of a conversation; one of the things that really frustrates me about /. is that if I respond to someone, then I have to basically refresh the page all the time to figure out if someone has responded to me. > I'm finding it very easy to capture data from RSS newsfeeds and then post > commentary with Radio Userland http://radio.userland.com/ . One of the > UserLand staff has started a discussion on knowledge management using > weblogs. I'm not certain that this _manages_ knowledge yet, but it does seem > to stimulate the easy exchange of pertinent information. I'll have to look into this, as I haven't heard of it before. As I mentioned in my previous email, I'll be picking this up next week when I get back in town. -- To define recursion, we must first define recursion. From rjw@alembic.com Fri Feb 1 20:26:45 2002 Received: from mail.alembic.net (root@mail.alembic.net [192.82.19.252]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id UAA12708 for ; Fri, 1 Feb 2002 20:26:44 -0800 Received: from tycho.alembic.com (tycho.alembic.com [192.82.19.225]) by mail.alembic.net (8.11.3/8.11.3) with SMTP id g124RtS18083; Fri, 1 Feb 2002 20:27:55 -0800 (PST) Received: from localhost by tycho.alembic.com (4.1/SMI-4.1) id AA28156; Fri, 1 Feb 02 20:30:14 PST Date: Fri, 1 Feb 2002 20:30:13 -0800 (PST) From: Ron Wickersham X-Sender: rjw@tycho To: "Luke A. Kanies" Cc: Will Cox , infrastructures@terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool In-Reply-To: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Fri, 1 Feb 2002, Luke A. Kanies wrote: ---snip--- > > At my last position, we had lots of ways to share information: 1) the > > FAQ-O-Matic, 2) Wiki, 3) internal document repository built on Oracle with a > > web front-end, 4) Bugzilla, 5) flat html, 6) mailing lists. Nobody really > > put information into the documentation systems unless they were "prompted" > > to do so. It seemed mailing lists and Wiki yielded the best results, as the > > barriers to entry were the lowest. At my position now, information is > > hoarded, and has difficulty crossing functional boundaries. > > Interesting. That's one of the things that I've been thinking a lot > about, how to keep a low barrier of entry. I also want to do what I can > to maintain the concept of a conversation; one of the things that really > frustrates me about /. is that if I respond to someone, then I have to > basically refresh the page all the time to figure out if someone has > responded to me. ---snip--- i've used Discus as a web-based "conversation" builder. one nice advantage is that users can receive updates by e-mail, and on a user profile can selectively monitor topics of interest. while i haven't used it yet, the latest version also allows e-mail posting to topics...thus users can choose web-based or e-mail interaction. http://www.discusware.com -ron From tubaman@mailandnews.com Sat Feb 2 00:56:26 2002 Received: from MailAndNews.com ([199.29.68.111]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id AAA21548 for ; Sat, 2 Feb 2002 00:56:24 -0800 Received: from [66.68.36.205] (tubaman@mailandnews.com) by MailAndNews.com; Sat, 2 Feb 2002 03:56:14 -0500 X-WM-Posted-At: MailAndNews.com; Sat, 2 Feb 02 03:56:14 -0500 Received: from polishwonder (localhost [127.0.0.1]) by polishwonder.dyn.dhs.org (8.11.6/8.11.6) with ESMTP id g0VHOvu09155 for ; Thu, 31 Jan 2002 23:24:58 +0600 Date: Thu, 31 Jan 2002 23:24:57 +0600 From: Ryan Nowakowski To: infrastructures@TerraLuna.Org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020131232457.A9020@polishwonder.dyn.dhs.org> Reply-To: ryan_nowakowski@alumni.utexas.net References: Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit In-Reply-To: ; from luke@madstop.com on Thu, Jan 31, 2002 at 11:38:33 +0600 X-Mailer: Balsa 1.2.3 Lines: 41 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On 2002.01.31 11:38 Luke A. Kanies wrote: > > > 1) Should I take this discussion somewhere else? No, since the next revision of the infrastructures.org methods will probably come from this tool, we should have input into it's design. I think that once the tool is out of the design phase and into implementation, all the gory details probably shouldn't be posted here. > > 2) If not, should we discuss this long enough to get something more > concrete and then take it somewhere else? > > 3) Does anyone have a better word than "tool"? application? webapp? > > 4) Is the lack of a good domain a restriction here? I own madstop.com > and am planning on using that until something better comes up, and I have > a couple other ideas, but none of them really shout out at me. I > basically believe that the entry point is important, and so I'd like > something that everyone agreed was decent. I think the domain is not as important. If you build it, they will come (ex: freshmeat.net) > 5) Does anyone have any great ideas as to how to implement this, either > conceptually, technically, or somewhere in between? Technically, I believe Zope gives you everything you might need, including a Wiki and content management. I've created a few apps with Zope and have found it to be very flexible. > 6) Finally, does anyone want to work with me on this? Yes, sounds cool. - Ryan From tubaman@mailandnews.com Sat Feb 2 00:56:36 2002 Received: from MailAndNews.com ([199.29.68.111]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id AAA21551 for ; Sat, 2 Feb 2002 00:56:27 -0800 Received: from polishwonder.dyn.dhs.org [66.68.36.205] (tubaman@mailandnews.com) by MailAndNews.com; Sat, 2 Feb 2002 03:56:10 -0500 X-WM-Posted-At: MailAndNews.com; Sat, 2 Feb 02 03:56:10 -0500 Received: (from tubaman@localhost) by polishwonder.dyn.dhs.org (8.11.6/8.11.6) id g1256oL02672 for infrastructures@mailman.terraluna.org; Sat, 2 Feb 2002 11:06:50 +0600 Date: Sat, 2 Feb 2002 11:06:50 +0600 From: Ryan Nowakowski To: infrastructures@roton.terraluna.org Subject: Re: [Infrastructures] Sysadmin collaboration tool Message-ID: <20020202110650.A2669@polishwonder.dyn.dhs.org> Reply-To: ryan_nowakowski@alumni.utexas.net References: Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from luke@madstop.com on Thu, Jan 31, 2002 at 11:38:33 +0600 Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On 2002.01.31 11:38 Luke A. Kanies wrote: > > > 1) Should I take this discussion somewhere else? No, since the next revision of the infrastructures.org methods will probably come from this tool, we should have input into it's design. I think that once the tool is out of the design phase and into implementation, all the gory details probably shouldn't be posted here. > > 2) If not, should we discuss this long enough to get something more > concrete and then take it somewhere else? > > 3) Does anyone have a better word than "tool"? application? webapp? > > 4) Is the lack of a good domain a restriction here? I own madstop.com > and am planning on using that until something better comes up, and I have > a couple other ideas, but none of them really shout out at me. I > basically believe that the entry point is important, and so I'd like > something that everyone agreed was decent. I think the domain is not as important. If you build it, they will come (ex: freshmeat.net) > 5) Does anyone have any great ideas as to how to implement this, either > conceptually, technically, or somewhere in between? Technically, I believe Zope gives you everything you might need, including a Wiki and content management. I've created a few apps with Zope and have found it to be very flexible. > 6) Finally, does anyone want to work with me on this? Yes, sounds cool. - Ryan From luke@madstop.com Mon Feb 4 12:58:40 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id MAA02266 for ; Mon, 4 Feb 2002 12:58:39 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14Kpx3D012188 for ; Mon, 4 Feb 2002 14:52:00 -0600 (CST) Date: Mon, 4 Feb 2002 14:51:59 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] isconf integration with existing install tools Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi all, I'm currently looking at deploying some version of Isconf in my enterprise. Steve was out here last week getting me up to speed and helping to convince my management that I'm doing the right thing, so I'm pretty familiar with the product. My situation is, I have a pretty significant investment in Jumpstart right now, specifically related to Sun's JASS toolkit. Should I attempt to develop in such a way that I can maintain backwards compatibility with JASS, thus helping to spread the love, or should I basically move away from it and bring my toys with me? I'm not very happy with the overall design of JASS, but that's the part that I would basically have to replace anyway. If I replaced all or part of the portions which actually do the work, I would think it shouldn't be too difficult to use my installed base of code while allowing both Isconf and JASS users to take advantage of additional code. I know that I'll have to jump through some hoops, but I'm just a bit hesitent to toss JASS and all of it's software out the window. For those who haven't used it, JASS provides basically two things for Jumpstart: First, a basic framework for all Jumpstart actions, such as running scripts and copying files, and second, a very strong foundation of security scripts, to do things like tighten file permissions, increase the randomness of the TCP/IP stack, and all kinds of other stuff. It basically provides a framework for Jumpstart, but one which also works in a normally booted environment, and also a lot of security scripts. As I mentioned, I don't much like the framework itself, but I like a lot of the scripts and I don't much look forward to trying to replace them (it's open source because it's all in sh, but not modifying or redistributable). For perspective, I currently have 93 different scripts set up in JASS, along with all of my file copies and my package installs, so this isn't two days worth of work or anything, and most of those ship with JASS. One of the things that I think is pretty important to the future of this level of automated systems administration is having chunkable code that we can share. Isconf provides a framework for that code, although that framework itself is in the process of being rewritten, and now we need to begin developing and distributing the little chunks of code that do all the interesting stuff. Of course, the next step is that we need to have a place to do this distribution, but that's maybe another email. It might even be one I sent last week. :) -- Nature and nature's laws lay hid in night, God said, "Let Newton be," and all was light. It did not last; the devil howling "Ho! Let Einstein be!" restored the status quo. From luke@madstop.com Mon Feb 4 13:08:46 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA07524; Mon, 4 Feb 2002 13:08:45 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14L2fc2012217; Mon, 4 Feb 2002 15:02:41 -0600 (CST) Date: Mon, 4 Feb 2002 15:02:41 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: stevegt@terraluna.org cc: infrastructures@terraluna.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: [Infrastructures] state engine Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: Hi Steve (I've CC'ed the infrastructures list because this is also pertinent to them), I'm still trying to get my brain around how make is being used as a state engine in isconf. As far as I can tell, all make does is verify that all scripts are run in the correct order and the correct number of times, that is, if a script is supposed to be run once then make guarantees that, but make is also used to make a script run all the time. So my question is, if we ignore the topic of config files for just a minute, does make currently provide functionality that couldn't be replaced with a script which essentially checked the stamps directory, and only ran a script if (a) the stamp didn't exist, or (b) the script was supposed to be run multiple times? Make is normally used to run a stanza multiple times, but in this case we generally don't actually want to run a stanza more than once. Or at least, we only want to run new members of the stanza. It's pretty different from code, where you want to recompile all of the basic chunks and then combine those basic chunks into large chunks. In this case, we are only worried about running the correct scripts in the correct order. Or am I missing something? That seems to be the case, and that difference between how Isconf uses make and how I am used to using make seems to be one of the hardest things to wrap my brain around. I guess the main reason I ask is because if this difference is as clear as I think it is, Isconf's requirements for a state engine are much less expansive than make's feature set. At this point, I'm not planning on doing anything about it, but I would definitely like to clarify this before I get much deeper. Obviously, the state engine is one of the most important aspects of this whole process. Thanks, Luke -- Some people are afraid of heights. I'm afraid of widths. -- Stephen Wright From hag@linnaean.org Mon Feb 4 13:30:24 2002 Received: from perdition.linnaean.org (h00d0b71b83ad.ne.mediaone.net [65.96.132.240]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id NAA17862; Mon, 4 Feb 2002 13:30:24 -0800 Received: by perdition.linnaean.org (Postfix, from userid 31013) id BB2E31BA0B; Mon, 4 Feb 2002 16:30:02 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Luke A. Kanies" Cc: stevegt@terraluna.org, infrastructures@terraluna.org Subject: [Infrastructures] state engine In-Reply-To: References: X-Mailer: VM 6.34 under Emacs 20.7.1 Reply-To: Daniel Hagerty From: Daniel Hagerty Date: Mon, 4 Feb 2002 16:30:02 -0500 Message-Id: <20020204213002.BB2E31BA0B@perdition.linnaean.org> Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: > From: "Luke A. Kanies" > Date: Mon, 4 Feb 2002 15:02:41 -0600 (CST) > > Hi Steve (I've CC'ed the infrastructures list because this is also > pertinent to them), > > I'm still trying to get my brain around how make is being used as a state > engine in isconf. As far as I can tell, all make does is verify that all Does this mean code for isconf is available? (maybe I was supposed to ask this years ago...) I've seen it talked about, but never seen code. From luke@madstop.com Mon Feb 4 15:44:02 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id PAA22985; Mon, 4 Feb 2002 15:44:01 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14Nbp86012595; Mon, 4 Feb 2002 17:37:51 -0600 (CST) Date: Mon, 4 Feb 2002 17:37:51 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Daniel Hagerty cc: stevegt@terraluna.org, infrastructures@terraluna.org Subject: Re: [Infrastructures] state engine In-Reply-To: <20020204213002.BB2E31BA0B@perdition.linnaean.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 4 Feb 2002, Daniel Hagerty wrote: > Does this mean code for isconf is available? (maybe I was supposed > to ask this years ago...) I've seen it talked about, but never seen > code. It's GPL'ed. Steve gave me a sanitized version recently, and I'm sure he'd give you one, or let me send you one. It's still very much a hand-roll process in terms of setup, but yeah, the code is available. -- I don't want the world, I just want your half. From luke@madstop.com Mon Feb 4 15:52:21 2002 Received: from pixie.madstop.com ([207.65.26.13]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id PAA25979 for ; Mon, 4 Feb 2002 15:52:20 -0800 Received: from localhost (luke@localhost) by pixie.madstop.com (8.12.2/8.12.2) with ESMTP id g14Nki0l012619; Mon, 4 Feb 2002 17:46:44 -0600 (CST) Date: Mon, 4 Feb 2002 17:46:44 -0600 (CST) From: "Luke A. Kanies" X-Sender: luke@pixie To: Mitch Wyle cc: infrastructures@terraluna.org Subject: Re: [Infrastructures] isconf integration with existing install tools In-Reply-To: <20020204142001.A8160@wyle.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: infrastructures-admin@mailman.terraluna.org Errors-To: infrastructures-admin@mailman.terraluna.org X-BeenThere: infrastructures@mailman.terraluna.org X-Mailman-Version: 2.0.5 Precedence: bulk List-Help: List-Post: List-Subscribe: , List-Id: List-Unsubscribe: , List-Archive: On Mon, 4 Feb 2002, Mitch Wyle wrote: > 2002-02-04 > > Dear Luke, > > I suggest you abstract the policies and requirements for your automation > need away from all frameworks and then map the needs to JASS. Then > (possibly) you can re-map your work to the Isconf-next-generation[tm] > framework when it stabilizes. My suggestion is that you spend your time > writing prose and pseudo-code that describes what you want to happen in > your environment without thinking about the capabilities of one > framework or another. Then start to map your needs to the JASS > framework first. I'm already past that point; I have a significant amount of code which does stuff which I would like to apply to any machines I build within my infrastructure. I don't need to do any planning or mapping, because the code is already there and I know I want to use it. I guess I am more looking for an overall direction of the software we as a group are planning on developing. On the one hand, we want to be able to develop sophisticated code chunks which can depend on a base setup, but on the other hand, I would really like my code to be usable by people who aren't willing or able to commit to full automation isconf-style. Currently JASS is a decent compromise between the two, because it provides a decent framework for shell scripting, but the code is already chunked into separate scripts. Well, I hope to starting plopping code into place this week, and I'm as lazy as they come so I'll be reusing as much of the code as I can. I'll probably provide a basic wrapper initially, but we'll see. I've already threatened the Sun guys with rewriting JASS, because its structure is so poor, and it may come to that. -- "The Microsoft Exchange Information Store service depends on the Microsoft Exchange Directory service which failed to start because of the following error: The operation completed successfully." From hag@linnaean.org Mon Feb 4 16:32:20 2002 Received: from perdition.linnaean.org (h00d0b71b83ad.ne.mediaone.net [65.96.132.240]) by roton.terraluna.org (8.9.3/8.9.3) with ESMTP id QAA14156; Mon, 4 Feb 2002 16:32:19 -0800 Received: by perdition.linnaean.org (Postfix, from userid 31013) id E7D7B1BA14; Mon, 4 Feb 2002 19:31:57 -0500 (EST) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: "Luke A. Kanies" Cc: stevegt@terraluna.org,