[aprssig] Re: More on the findU server issue...
steve at dimse.com
Sun Sep 25 17:53:19 CDT 2005
On Sep 25, 2005, at 6:01 PM, <scott at opentrac.org>
<scott at opentrac.org> wrote:
>> Each packet results in multiple writes...for example, a
>> position report writes the raw packet, the position into the
> That's why I gave the factor of 100 increase. Again, I know very
> about MySQL architecture so I'm not sure if it's feasible to split
> up table
> and index access between multiple disks. I don't even know if it's
> got a
> redo log concept like Oracle.
> I do know that I had an Oracle 9i database on an old PII-400
> machine with
> old 8 GB SCSI drives (non-RAID - had to shut the Gladiator array
> off due to
> the power bill!) that was keeping up with all of the standard APRS
> packets (no MIC-E), updating regular and spatial indexes, and
> handling a few
> queries without breaking a sweat.
When you have a parser that handles more than just positions, like
findU's weather, status, ariss, pcsat, message, and so on, it is very
likely that each write must include a seek, whereas if you are
writing one table of position, the situation is much less likely to
require a seek on each write.
Anyway, as I say, even the backup server has no problem either with a
situation involving a few queries. The problem is when you are trying
to handle half the load of findU, which is around 20 http requests a
second. From the sound of it, you are were not parsing all of the
APRS formats, which significantly increases the load, and it sounds
like this was something in the past, with a lower APRS IS stream
rate, and even just filtering out the Mic-E takes away something like
a third to a half of the positions. Putting all this apple-to-orange
stuff, your parser may have been writing a tenth or a twentieth the
number of database records the findU parser does.
> I'm surprised that MySQL has such a hard
Just to be completely clear, MySQL does not have the problem. Running
on the primary server, it works fine. Even under the Slashdot load it
was able to almost keep up, even if not snappily.
> Anyway, none of that helps FindU. An Oracle license would cost
> more than
> the server hardware even if you could convert it. I'm just curious
> for my
> own use... I wonder if any of the other free RDBMS's are any better.
Just as I wrote about the hardware vendor issue, it is quite possible
that another software solution may be somewhat more efficient, but I
have no interest in investing the considerable time and effort such
experimentation would take. The fact is I have a solution that works,
and I want to spend my time improving the user experience, not geeked
out wringing the last bit of life out of old hardware.
More information about the aprssig