[neurosetta] positron experiences and questions

[Dan Berger]

On Tue, Jul 01, 2003 at 07:08:48AM -0500, Stan Seibert wrote:
> 
> > My first observation is that doing file reads off the neuros must be
> > dog slow - as it's taking rsync forever to decide what files it
> > doesn't need to update.
> > 
> > My second observation is that writes are pretty slow, too - as rsync
> > reports about 500kB/s writing files.
> 
> Yes, this is a limitation of the USB 1.1 interface.  The USB 2.0
> backpacks should bump this rate up by a factor of 10-20x.

Not being much of a "USB guy" I was under the impression that USB 1
was 12Mb/s (i.e. mega bits) - so dividing by 8 to get to MB is 1.5MB/s
- and I'm seeing a third of that.  This is "normal"?

> 
> > My fifth  and final (at least for now) observation is that during
> > those times that rsync is getting poor throughput, my load average
> > peaks around 4 - but once that file is written (and flushed, perhaps?)
> > the load average drops and rsync continues.
> > 
> > My conclusion is that what seems to be going on is that positron is
> > aggravating a kernel or usb-storage bug - and that rsync, for whatever
> > reason, isn't doing the same.

So another data point - the sync ran for about an hour, then the
following appeared in my system log:

Jul  1 10:04:53 rage kernel: Filesystem panic (dev 08:01).
Jul  1 10:04:53 rage kernel:   fat_free: deleting beyond EOF
Jul  1 10:04:53 rage kernel:   File system has been set read-only

rsync continued trying, but (of course) couldn't write and ultimately
exited.

running an fsck on the drive reported:

FATs differ but appear to be intact. Using first FAT.
/music/crowded house
  "." is missing. Can't fix this yet.
/music/crowded house
  ".." is missing. Can't fix this yet.
/WOID_DB/audio/audio.mdb
  Contains a free cluster (211712). Assuming EOF.
/WOID_DB/audio/audio.mdb
  File size is 76724 bytes, cluster chain length is 57344 bytes.
  Truncating file to 57344 bytes.
Reclaiming unconnected clusters.
Checking free cluster summary.
Free cluster summary wrong (2003357 vs. really 2007453)
  Auto-correcting.
Performing changes.
/dev/sda1: 1362 files, 432038/2439491 clusters

I removed the offending directory (but not the database files, which I
figured I'd just rebuild with positron when I'm done) and restarted
rsync.

> 
> This is interesting.  I'd be curious to know what the difference is. 
> (For the record, since I couldn't find a "copy file" function in the os
> module, I wrote my own that reads 1 MB from the source and writes 1 MB
> to the destination alternately.  Would smaller chunks work better?)

As for copy file function - it's in the shutil module.  As for the 1MB
buffer, possibly - as might setting the output to unbuffered.


-- 
   Dan Berger [dberger at oubliette.org]
   http://www.oubliette.org/~dberger
   Inter arma silent leges

   "Experience should teach us to be most on our guard to protect
   liberty when the government's purposes are beneficent. Men born to
   freedom are naturally alert to repel invasion of their liberty by
   evil minded rulers. The greatest dangers to liberty lurk in
   insidious encroachment by men of zeal, well-meaning but without
   understanding."  
	Justice Louis Brandeis, dissenting, Olmstead v US (1928)

    A982 E6B1 CB2F 7A49  843A 9297 DA73 4371  1F54 8D0C
--- >8 ----
List archives:  http://www.xiph.org/archives/
Neurosetta homepage: http://www.neurosetta.org/
To unsubscribe from this list, send a message to 'neurosetta-request at xiph.org'
containing only the word 'unsubscribe' in the body.  No subject is needed.
Unsubscribe messages sent to the list will be ignored/filtered.