[Infrastructures] ISconf development
Steve Traugott
stevegt@TerraLuna.Org
Mon, 7 Mar 2005 21:49:18 -0800
--rJwd6BRFiFCcLxzm
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi All!
I just wanted to take this opportunity to thank you for all of the kind
words of encouragement over the last few weeks. Among other things
you've convinced me that I should start maintaining public releases
again. Here's the general plan right now:
- set up new site (SVN, bugs, wiki)
- post v4 prototype code as-is
- start project for ISFS (p2p app-layer filesystem)
- start projects for Python and Perl rewrites (in parallel, on top of
ISFS)
To recap about this ISFS thing -- right now I'm using UDP broadcasts, a
little Perl HTTP daemon on each machine, and wget to keep a local cache
directory in sync. Crude but it works, and the code for it hasn't
needed touching for over a year. This needs to be expanded to do TCP
WAN connections, selective cache purging, block-level data management,
subdirectories, PGP-based ACLs, and locking.=20
Rather than write all of the comm reliability, membership, and
token-passing code from scratch, I'm thinking of taking a shortcut and
writing ISFS on top of Spread (http://spread.org) -- let me know if
anyone has any opinions about that, pro or con.
Steve
On Fri, Feb 04, 2005 at 01:22:46AM -0800, Steve Traugott wrote:
> On Thu, Feb 03, 2005 at 12:32:40AM +0000, Matt S Trout wrote:
> > On Wed, Feb 02, 2005 at 02:17:53AM -0800, Steve Traugott wrote:
> > > On Wed, Feb 02, 2005 at 02:00:23AM +0000, Matt S Trout wrote:
> > > > http://www.infrastructures.org/bootstrap/isconf.shtml only seems to=
have
> > > > version 2. Are the later versions going to be available for downloa=
d at
> > > > some stage?
> > >=20
> > > See http://www.isconf.org/version4/index.shtml. A lot depends on
> > > interest. The response I got from many of you folks on this list 15
> > > months ago was great, and fueled the prototype that I'm still using n=
ow.
> > >
> > > <snip: stupid arguments against the isconf4 model>
> >=20
> > I remember that, and I remember looking forward to the result hugely. T=
hose
> > who detract from the model have (in every example I've seen) failed to
> > understand the model. They say "isconf4 isn't good for X" when X is ent=
irely
> > orthogonal to the aims of isconf4. This is their problem, not yours.
>=20
> That's pretty accurate, I think. I'm the kind of silly person who tends
> to look inward first though -- "gee, maybe I really don't know what I'm
> talking about"... It took me this long before I could go back and
> actually read some of the arguments objectively enough to see what they
> were going on about. At this point I think we didn't make some critical
> things clear enough in the 'turing' paper, because in each case the
> objections have hinged on the objector missing some fundamental point in
> there. =20
>=20
> > Very simply - if isconf4 can't be released relatively soon, is there any
> > chance the design documents can be? If I can't have the real thing I'm =
going
> > to have to reimplement >50% of it in order to provide the quality of
> > results to my clients that I promise.
>=20
> Unfortunately the best design document right now is the 'turing' paper. =
;-} =20
> But you've got me thinking...
>=20
> > Also, I'm a bit of a perl hacker (see CPAN directory for mstrout) and i=
f you
> > don't have time to clean it up for release, I'd be happy to - I serious=
ly
> > want access to the software that badly!
>=20
> ...Matt, the prototype code is all in Perl. It's only about 1200 lines;
> a lot of that is dead junk. Here's what needs to be done before general
> release:
>=20
> - When I wrote this a year ago I was trying to implement all of the
> subcommands as plugins that are standalone executables, including
> 'snap' and 'exec' themselves, so that people could write extensions in
> any language easily. I finally realized that makes it extremely
> tricky to upgrade isconf itself, especially if there are file-format
> changes, because each plugin is an atomic lower-level prereq -- duh,
> turing again.=20
>=20
> - I've really been lusting after (at least) two implementations -- one
> in Perl, the other in Python, regression test the two against each
> other... The code's small enough that I think we can get away with
> maintaining both. It would settle the language question, and might
> even provide a nice rosetta stone to compare between the two. I tend
> to use Python myself for large projects these days (wasn't true a year
> ago), so I was planning to do the Python version as the rewrite. This
> also opens the door to a C implementation if anyone wants to tackle
> it, for instance.
> =20
> - Multiple implementations means that design docs *will* be needed --
> wire protocols and file formats, mostly. =20
>=20
> - I started realizing a few months ago that there is a horizontal slice
> that should be done, would not cause prereq problems, and would
> actually solve some. Right now v4 runs twice -- once as a
> long-running daemon to do the p2p file serving, and again from rc or
> command line to do the actual work. The p2p code really ought to be
> split off as a separate project, an application-layer "Infrastructure
> Filesystem". (I don't mean mountable -- you talk to it via an API.)
> This would then be able to serve as a generic place to store things in
> addition to what ISconf uses it for; a storage place for large or
> small blobs, with no single point of failure. Think backups or
> install images... It would also reduce the amount of code that needs
> to run as root. =20
>=20
> ...so you've got me thinking about what I should have done years ago to
> get more people involved. Would you (or anyone else) be interested in
> working on any of this if we went with something like this plan?
>=20
> Steve
> --=20
> Stephen G. Traugott (KG6HDQ)
> UNIX/Linux Infrastructure Architect, TerraLuna LLC
> stevegt@TerraLuna.Org=20
> http://www.stevegt.com -- http://Infrastructures.Org
> _______________________________________________
> Infrastructures mailing list
> Infrastructures@mailman.terraluna.org
> http://mailman.terraluna.org/mailman/listinfo/infrastructures
--=20
Stephen G. Traugott (KG6HDQ)
UNIX/Linux Infrastructure Architect, TerraLuna LLC
stevegt@TerraLuna.Org=20
http://www.stevegt.com -- http://Infrastructures.Org
--rJwd6BRFiFCcLxzm
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)
iD8DBQFCLTze8rKIxO1Fc9MRAurwAJ9czvScYCNGrC9TGaLDocqM8AZ2fgCfTrfs
EMarQPt9WVHF9fvOLD3rpe4=
=8jjr
-----END PGP SIGNATURE-----
--rJwd6BRFiFCcLxzm--