[aprssig] distributed findu possible ?

[Scott Miller]

> The hard stuff, as I've said, is keeping two servers with 200+GB  
> databases serving millions of dynamic web pages a day operating  
> reliably. Companies hire full time IT staffs to handle lower loads.

Well, what's the hardest part there?  Database maintenance?  Log 
management?  OS maintenance?  Yes, some companies might have a full-time 
staff for that, but some get by on less.

I don't have enough experience with large MySQL databases to know how 
much of a pain that is.  Oracle certainly scales just fine at those 
sizes... there's a steep learning curve (and it's not cheap) but with a 
properly structured database and the right hardware you can do a lot.

Back when I did big Oracle databases (mostly on VMS clusters), we relied 
a lot on the features of the database rather than the hardware.  No 
RAID, except for mirroring on the system drives.  Oracle could handle 
redundancy more efficiently by itself, mirroring tablespaces to multiple 
drives (and distributing read operations).  With partitioning (something 
I'm not sure is fully supported yet in MySQL) you could even further 
distribute the I/O load.  Major indexes each had their own drives, and 
the load was further distributed between multiple SCSI (or DSSI) buses.

Anyone know how much of that has filtered down to MySQL?  Does anyone 
even bother, or do they just throw it on a big RAID 5 and buy faster 
drives and more RAM?  I know that's what happened with the systems I 
used to run - the hardware improved faster than the system grew, and 
eventually all of that careful planning and design lost out to 
convenience and simplicity.

Scott
N1VG