[pLog-svn] resserver usage
Paul Westbrook
paul at westbrooks.org
Fri Oct 6 16:47:50 GMT 2006
Hello,
I am happy to take a pass at this plugin. I will not be able to
start on this until next week, since we are going out of town.
--Paul
On Oct 6, 2006, at 12:48 AM, Oscar Renalias wrote:
> So just to come up with a clear list of possible actions regarding
> this issue:
>
> - We could still investigate the idea of using mod_rewrite and a
> specialized plugin to provide direct access to resources in 1.1.x. If
> Paul is interested, we could give it a try right away.
>
> - I am not sure about the iframe idea, so let's not consider it yet.
>
> - We could play around with the current implementation of resources in
> the 1.2 development branch (trunk) to provide direct access to
> resources and see if things improve. If nobody's interested, I can
> take a look at it when I have the time.
>
> Anything else?
>
> On 10/5/06, Oscar Renalias <oscar at renalias.net> wrote:
>> One thing I didn't measure in my tests was the CPU load when using
>> fopen()+fread() or direct access to the file via the web server.
>> Maybe
>> that's the reason? resserver.php has to streams big files while the
>> rest of the blog is serving small-ish pages (20-30kb compared to
>> 300-400kb sometimes?)
>>
>> On 10/5/06, Jon Daley <plogworld at jon.limedaley.com> wrote:
>> > But if it is the same, why is the load so much higher
>> with the
>> > resserver?
>> >
>> > On Thu, 5 Oct 2006, Oscar Renalias wrote:
>> > > Yes we do it in exactly the same way, but at least in those
>> cases I
>> > > don't see a clear way to improve it. At least with resources
>> we're
>> > > considering direct access which would help alleviate the
>> situation.
>> > >
>> > > On 10/5/06, Jon Daley <plogworld at jon.limedaley.com> wrote:
>> > >> But, don't we do that sort of stuff on other
>> requests? ie. when I
>> > >> use custom URLs, I don't have the blogId, etc. Maybe there
>> needs to be
>> > >> some different/new indexes?
>> > >>
>> > >> On Thu, 5 Oct 2006, Oscar Renalias wrote:
>> > >>
>> > >> > I may try with fpassthru later on, but I am not sure
>> whether it will
>> > >> > make any difference. In the tests I run I noticed that in
>> the context
>> > >> > of 100 requests, usign fread() was only 1 second slower
>> than using
>> > >> > direct URL access so this probably means that the problem is
>> > >> > elsewhere.
>> > >> >
>> > >> > Right now resserver.php needs to load a bunch of classes,
>> validate
>> > >> > parameters, execute the pipeline for security stuff, load
>> the plugin
>> > >> > manager and throw two events, load the blog id based on the
>> name if
>> > >> > we're using subdomains, parse nice album names, etc. As
>> soon as I
>> > >> > removed some of that sutff, we managed to save 15 seconds
>> in 100
>> > >> > requisitions and over 1 minute when using altresserver.php,
>> which only
>> > >> > implements the best case scenario: no subdomains and the
>> blog id and
>> > >> > the resource name are in the request (no need to load any
>> extra
>> > >> > classes and parse anything)
>> > >> >
>> > >> > What I am trying to say is that the bottleneck isn't the
>> file-related
>> > >> > function calls but all what we have to do before we even
>> get there.
>> > >> >
>> > >> > On 10/5/06, Jon Daley <plogworld at jon.limedaley.com> wrote:
>> > >> >> You might want to include fpassthru in your tests
>> - that is
>> > >> >> supposed to be the "recommended" way of doing this sort of
>> thing. I
>> > >> >> couldn't tell any difference when I changed the current
>> resserver to use
>> > >> >> it though.
>> > >> >>
>> > >> >>
>> > >> >> On Thu, 5 Oct 2006, Oscar Renalias wrote:
>> > >> >> > Some more benchmarks I was working on today, tested with
>> curl loading
>> > >> the
>> > >> >> > same URL 100 times (a 450kb JPEG file)
>> > >> >> >
>> > >> >> > testfopen.php
>> > >> >> > -------------
>> > >> >> > real 0m11.127s
>> > >> >> > user 0m1.205s
>> > >> >> > sys 0m1.893s
>> > >> >> >
>> > >> >> > direct url
>> > >> >> > ----------
>> > >> >> > real 0m10.170s
>> > >> >> > user 0m1.171s
>> > >> >> > sys 0m1.846s
>> > >> >> >
>> > >> >> > resserver.php 1.1.1
>> > >> >> > -------------------
>> > >> >> > real 1m39.753s
>> > >> >> > user 0m1.209s
>> > >> >> > sys 0m2.000s
>> > >> >> >
>> > >> >> > optimized resserver.php 1.1.1
>> > >> >> > -----------------------------
>> > >> >> > real 1m24.475s
>> > >> >> > user 0m1.210s
>> > >> >> > sys 0m2.007s
>> > >> >> >
>> > >> >> > altresserver.php
>> > >> >> > ----------------
>> > >> >> > real 0m36.123s
>> > >> >> > user 0m1.230s
>> > >> >> > sys 0m2.012s
>> > >> >> >
>> > >> >> > testfopen.php is a test to see how fast a raw fopen()
>> +fread() is in
>> > >> PHP,
>> > >> >> and
>> > >> >> > it's a useless version of resserver.php. "Direct url"
>> downloads the
>> > >> URL
>> > >> >> via
>> > >> >> > the web server, "resserver.php" is the default version of
>> > >> resserver.php
>> > >> >> > included in 1.1.1, "optimized resserver.php" very
>> similar to the code
>> > >> in
>> > >> >> > 1.1.1 but removed a few things that could potentially be
>> removed, and
>> > >> >> finally
>> > >> >> > "altresserver.php" is a very small version of
>> resserver.php that still
>> > >> >> loads
>> > >> >> > a few classes (like GalleryResources and GalleryResource)
>> > >> >> >
>> > >> >> > Not much to comment on these figures, other than we're
>> currently using
>> > >> >> the
>> > >> >> > slowest method and that direct url would still be the
>> preferred way.
>> > >> >> >
>> > >> >> > At this point I am not sure if it's a good idea to
>> create complex
>> > >> rewrite
>> > >> >> > rules or use obscure modules that no user will probably
>> ever have
>> > >> >> available,
>> > >> >> > and instead we should concentrate on doing it properly
>> once and for
>> > >> all.
>> > >> >> Some
>> > >> >> > of these workarounds could be used as temporary
>> solutions while we fix
>> > >> >> what
>> > >> >> > is fundamentally slow (or broken), and that is the way
>> in which we
>> > >> handle
>> > >> >> and
>> > >> >> > serve files.
>> > >> >> >
>> > >> >> > By the way I've attached the files I used for these
>> tests, although if
>> > >> >> you
>> > >> >> > want to run them you'll need to make a few changes (file
>> paths, URLs,
>> > >> >> etc)
>> > >> >> >
>> > >> >>
>> > >> >> --
>> > >> >> Jon Daley
>> > >> >> http://jon.limedaley.com/
>> > >> >>
>> > >> >> Teachers open the door, but you must enter by yourself.
>> > >> >> -- Chinese Proverb
>> > >> >> _______________________________________________
>> > >> >> pLog-svn mailing list
>> > >> >> pLog-svn at devel.lifetype.net
>> > >> >> http://devel.lifetype.net/mailman/listinfo/plog-svn
>> > >> >>
>> > >> > _______________________________________________
>> > >> > pLog-svn mailing list
>> > >> > pLog-svn at devel.lifetype.net
>> > >> > http://devel.lifetype.net/mailman/listinfo/plog-svn
>> > >> >
>> > >>
>> > >> --
>> > >> Jon Daley
>> > >> http://jon.limedaley.com/
>> > >>
>> > >> Research is what I'm doing when I don't know what I'm doing.
>> > >> -- von Braun
>> > >> _______________________________________________
>> > >> pLog-svn mailing list
>> > >> pLog-svn at devel.lifetype.net
>> > >> http://devel.lifetype.net/mailman/listinfo/plog-svn
>> > >>
>> > > _______________________________________________
>> > > pLog-svn mailing list
>> > > pLog-svn at devel.lifetype.net
>> > > http://devel.lifetype.net/mailman/listinfo/plog-svn
>> > >
>> >
>> > --
>> > Jon Daley
>> > http://jon.limedaley.com/
>> >
>> > If you have any trouble sounding condescending,
>> > find a Unix user to show you how it's done
>> > -- Scott Adams
>> > _______________________________________________
>> > pLog-svn mailing list
>> > pLog-svn at devel.lifetype.net
>> > http://devel.lifetype.net/mailman/listinfo/plog-svn
>> >
>>
> _______________________________________________
> pLog-svn mailing list
> pLog-svn at devel.lifetype.net
> http://devel.lifetype.net/mailman/listinfo/plog-svn
>
--
Paul Westbrook
paul at westbrooks.org
<http://www.westbrooks.org>
More information about the pLog-svn
mailing list