[pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related: . class/action class/view locale templates
Paul Westbrook
paul at westbrooks.org
Fri Sep 14 14:11:44 EDT 2007
Hello,
My only concern about doing the database query in the plugin itself is
that the plugin would be dependent on the schema for that version of the
database, while the search engine api would be more likely to remain the
same.
I like the changes that you have made. The only thing that I would
change is to make sure to only return published articles when pulling the
related articles from the cache, and to limit the number of articles
returned when the results come directly from the search engine.
Feel free to check these changes with the tree, or I am happy to as well.
--Paul
On 9/14/07, Ayalon <ayalon at blog.nl> wrote:
>
> Hi Paul,
>
>
>
> 1. Mine is set to 5, not really helping
>
> 2. 720 is set…
>
> 3. Also no body checking.
>
>
>
> What i think is that the search engine that's build in is not really build
> for so many queries at a time. Look at the wordpress related plug-in:
> http://peter.mapledesign.co.uk/weblog/wp-content/uploads/related-posts.phps
>
>
>
> They do the same thing, but instead of using the build in search engine of
> wordpress they directly go into the database. If we would do these and save
> those queries?! I know that it's maybe not the nicest option, but is this
> something you can try? Personally I think it's the only solution for having
> it work without taking to much mem currently…
>
>
>
> Attached is my optimized related plug in, now it takes a bit less memory
> to do the process, but still not perfect..
>
>
>
>
>
> Regards,
>
> Ayalon
>
>
>
>
>
> *Van:* plog-svn-bounces at devel.lifetype.net [mailto:
> plog-svn-bounces at devel.lifetype.net] *Namens *Paul Westbrook
> *Verzonden:* donderdag 13 september 2007 22:53
> *Aan:* LifeType Developer List
> *Onderwerp:* Re: [pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related:
> . class/action class/view locale templates
>
>
>
> Hello,
> Some of the options in the admin interface will help reduce the CPU
> usage:
>
> 1. Minimum Keyword Length - Set this to a larger number (Will
> reduce the number of search engine queries)
> 2. Cache Lifetime - This will increase the time between the list of
> related articles is rebuilt.
> 3. Parse Body - Uncheck this, and the plugin will only get keywords
> from the subject.
>
> --Paul
>
> On 9/12/07, *Paul Westbrook* <paul at westbrooks.org> wrote:
>
> Hello,
> This plugin uses a brute force algorithm. In order to calculate the
> list of related articles for a given post, first it gets a list of all of
> the unique words in the post. Then it for each keyword, it uses LifeType's
> builtin search engine to find a list of the posts that have that
> keyword. After it does that for all of the keywords, it lists the returned
> articles based on the number of times that the article is returned by the
> search engine. Then the plugin returns the top x articles.
>
> Since this is expensive, the plugin implements a cache. The list of
> related articles for a post is cached in the file system, so the next time
> that it runs, the queries do not have to be run. The user can specify the
> lifetime of that cache. For example, I have mine set to 1 month, so the
> queries will only be run once every 30 days for each article.
>
> There are serveral things that could be done to improve performance
>
> 1. Implement or use a library that will actually summarize the
> article text. This shouldn't simply take the nth sentence, but should
> generate a representative summary that contains the important keywords.
> This would reduce the number of search engine queries that are run
> 2. Change the search engine to be able to return a list of articles
> that contains at least one of a specified list of words. This would allow
> the search engine to be run only once for each article
> 3. Use native tag support. When lifetype nativelly supports tags,
> the plugins wouldn't have to use all of the text in the article, but could
> just find posts that are tagged with the same tags.
>
>
> --Paul
>
>
>
>
>
> On 9/12/07, Ayalon < ayalon at blog.nl> wrote:
> > Hi There,
> >
> > This is really a great plugin, but there are some problems with it.
> >
> > When i switch on this plugin my apache process rise to a memory use
> that's
> > incredible. Is there something to change about it? My database is pretty
> > large with a lot of articles and I have a lot of reads on the blogs.
> Anybody
> > an idea how to optimize?
> >
> > I know this is not really something for the list, but I tried to
> reprogram
> > the plugin making a different way of searching but I got stuck..
> >
> > Regards
> >
> >
> > -----Oorspronkelijk bericht-----
> > Van: plog-svn-bounces at devel.lifetype.net
> > [mailto: plog-svn-bounces at devel.lifetype.net] Namens
> > pwestbro at devel.lifetype.net
> > Verzonden: dinsdag 11 september 2007 7:10
> > Aan: plog-svn at devel.lifetype.net
> > Onderwerp: [pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related:
> .
> > class/action class/view locale templates
> >
> > Author: pwestbro
> > Date: 2007-09-11 01:09:42 -0400 (Tue, 11 Sep 2007)
> > New Revision: 5925
> >
> > Modified:
> >
> > plugins/branches/lifetype-1.2
> /related/class/action/pluginrelatedupdateconfig
> > action.class.php
> >
> > plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
> > ss.php
> > plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > plugins/branches/lifetype-1.2/related/templates/related.template
> > Log:
> > Added a setting so the minimum number for keywords that are used to
> generate
> > the list of related articles
> >
> >
> > Modified:
> > plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
>
> > action.class.php
> > ===================================================================
> > ---
> > plugins/branches/lifetype-1.2
> /related/class/action/pluginrelatedupdateconfig
> > action.class.php 2007-09-10 19:45:42 UTC (rev 5924)
> > +++
> > plugins/branches/lifetype-1.2
> /related/class/action/pluginrelatedupdateconfig
> > action.class.php 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -18,43 +18,45 @@
> > Foundation, Inc., 59 Temple Place, Suite 330, Boston,
> MA 02111-1307
> > USA
> > */
> >
> > - lt_include(
> > PLOG_CLASS_PATH."class/action/admin/adminaction.class.php" );
> > + lt_include(
> PLOG_CLASS_PATH."class/action/admin/adminaction.class.php"
> > );
> > lt_include(
> >
> PLOG_CLASS_PATH."plugins/related/class/view/pluginrelatedconfigview.class.ph
>
> > p" );
> >
> > $apiKeyValid = true;
> >
> > class PluginRelatedUpdateConfigAction extends AdminAction
> > - {
> > -
> > - var $_pluginEnabled;
> > - var $_numRelatedArticles;
> > - var $_minWordLength;
> > - var $_refreshInterval;
> > - var $_parseBody;
> > - var $_bannedKeywords;
> > + {
> > +
> > + var $_pluginEnabled;
> > + var $_numRelatedArticles;
> > + var $_minWordLength;
> > + var $_minNumKeywords;
> > + var $_refreshInterval;
> > + var $_parseBody;
> > + var $_bannedKeywords;
> >
> >
> > - /**
> > + /**
> > * Constructor. If nothing else, it also has to call the
> > constructor of the parent
> > * class, BlogAction with the same parameters
> > */
> > function PluginRelatedUpdateConfigAction( $actionInfo, $request
> )
> > {
> > - $this->AdminAction( $actionInfo, $request );
> > + $this->AdminAction( $actionInfo, $request );
> > }
> > -
> > - function validate()
> > - {
> > +
> > + function validate()
> > + {
> >
> > $this->_pluginEnabled = $this->_request->getValue(
> > "pluginEnabled" );
> > - $this->_pluginEnabled = ($this->_pluginEnabled != "" );
> >
> > + $this->_pluginEnabled = ($this->_pluginEnabled != "" );
> >
> >
> >
> > $this->_numRelatedArticles = $this->_request->getValue(
> > "numArticles" );
> > $this->_minWordLength = $this->_request->getValue(
> > "minWordLength" );
> > + $this->_minNumKeywords = $this->_request->getValue(
> > "minNumKeywords" );
> > $this->_refreshInterval = $this->_request->getValue(
> "interval"
> > );
> > $this->_parseBody = $this->_request->getValue( "parseBody"
> );
> > - $this->_parseBody = ($this->_parseBody != "" );
> >
> > + $this->_parseBody = ($this->_parseBody != "" );
> >
> > $this->_bannedKeywords = $this->_request->getValue(
> > "bannedKeywords" );
> >
> >
> > @@ -98,28 +100,49 @@
> > return false;
> > }
> > }
> > +
> > +
> > + if( $this->_minNumKeywords == "" ) {
> > + $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > + $this->_view->setErrorMessage(
> > $this->_locale->tr("related_missing_num_keywords"));
> > + $this->setCommonData();
> > +
> > + return false;
> > + }
> > + else {
> > + $val3 = new IntegerValidator();
> > + if( !$val3->validate( $this->_minNumKeywords )) {
> > + $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > + $this->_view->setErrorMessage(
> > $this->_locale->tr("related_invalid_num_keywords"));
> > + $this->setCommonData();
> > +
> > + return false;
> > + }
> > + }
> > +
> > }
> > - return true;
> > - }
> > -
> > + return true;
> > + }
> > +
> > /**
> > * Carries out the specified action
> > */
> > function perform()
> > {
> > // update the plugin configurations to blog setting
> > - $blogSettings = $this->_blogInfo->getSettings();
> > + $blogSettings =
> $this->_blogInfo->getSettings();
> > $blogSettings->setValue( "plugin_related_enabled",
> > $this->_pluginEnabled );
> > $blogSettings->setValue( "plugin_related_num_articles",
> > $this->_numRelatedArticles );
> > $blogSettings->setValue( "plugin_related_min_word_length",
> > $this->_minWordLength );
> > + $blogSettings->setValue( "plugin_related_min_num_keywords",
>
> > $this->_minNumKeywords );
> > $blogSettings->setValue( "plugin_related_refresh_interval",
> > $this->_refreshInterval );
> > $blogSettings->setValue(
> > "plugin_related_extract_keywords_from_body", $this->_parseBody );
> > $blogSettings->setValue( "plugin_related_banned_keywords",
> > $this->_bannedKeywords );
> >
> > $this->_blogInfo->setSettings( $blogSettings );
> > -
> > - // save the blogs settings
> > - $blogs = new Blogs();
> > +
> > + // save the blogs settings
> > + $blogs = new Blogs();
> > if( !$blogs->updateBlog( $this->_blogInfo )) {
> > $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > $this->_view->setErrorMessage(
> > $this->_locale->tr("error_updating_settings"));
> > @@ -127,20 +150,20 @@
> >
> > return false;
> > }
> > -
> > - // if everything went ok...
> > +
> > + // if everything went ok...
> > $this->_blogInfo->setSettings( $blogSettings );
> > $this->_session->setValue( "blogInfo", $this->_blogInfo );
> > $this->saveSession();
> > -
> > - $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > - $this->_view->setSuccessMessage(
> > $this->_locale->tr("related_settings_saved_ok"));
> > - $this->setCommonData();
> > -
> > - // clear the cache
> > - CacheControl::resetBlogCache(
> > $this->_blogInfo->getId());
> > +
> > + $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > + $this->_view->setSuccessMessage(
> > $this->_locale->tr("related_settings_saved_ok"));
> > + $this->setCommonData();
> > +
> > + // clear the cache
> > + CacheControl::resetBlogCache(
> > $this->_blogInfo->getId());
> >
> > - return true;
> > + return true;
> > }
> > }
> >
> >
> > Modified:
> > plugins/branches/lifetype- 1.2
> /related/class/view/pluginrelatedconfigview.cla
> > ss.php
> > ===================================================================
> > ---
> > plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
>
> > ss.php 2007-09-10 19:45:42 UTC (rev 5924)
> > +++
> > plugins/branches/lifetype-1.2
> /related/class/view/pluginrelatedconfigview.cla
> > ss.php 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -38,6 +38,7 @@
> > $pluginEnabled = $blogSettings->getValue(
> > "plugin_related_enabled" );
> > $numArticles = $blogSettings->getValue(
> > "plugin_related_num_articles" );
> > $minWordLength = $blogSettings->getValue(
> > "plugin_related_min_word_length" );
> > + $minNumKeyword = $blogSettings->getValue(
> > "plugin_related_min_num_keywords" );
> > $refreshInterval = $blogSettings->getValue(
> > "plugin_related_refresh_interval" );
> > $parseBody = $blogSettings->getValue(
> > "plugin_related_extract_keywords_from_body" );
> > $bannedKeywords = $blogSettings->getValue(
> > "plugin_related_banned_keywords" );
> > @@ -49,6 +50,7 @@
> > $this->setValue( "pluginEnabled", $pluginEnabled
> );
> > $this->setValue( "numArticles", $numArticles );
> > $this->setValue( "minWordLength", $minWordLength
> );
> > + $this->setValue( "minNumKeywords",
> $minNumKeyword );
> > $this->setValue( "interval", $refreshInterval );
> > $this->setValue( "parseBody", $parseBody );
> > $this->setValue( "bannedKeywords",
> $bannedKeywords
> > );
> >
> > Modified: plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > ===================================================================
> > --- plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > 2007-09-10 19:45:42 UTC (rev 5924)
> > +++ plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -10,12 +10,15 @@
> > $messages["related_settings_saved_ok"] = "Related Posts settings saved
> > successfully!";
> > $messages["related_missing_num_articles"] = "Number of articles needs
> to be
> > specified";
> > $messages["related_invalid_num_articles"] = "Number of articles needs
> to be
> > an integer";
> > +$messages["related_missing_num_keywords"] = "Number of keywords needs
> to be
> > specified";
> > +$messages["related_invalid_num_keywords"] = "Number of keywords needs
> to be
> > an integer";
> > $messages["related_missing_min_length"] = "Minimum keyword length needs
> to
> > be specified";
> > $messages["related_invalid_min_length"] = "Minumum keyword length needs
> to
> > be an integer";
> > $messages["related_banned_keywords"] = "Keywords that should not be
> used to
> > find related posts (comma separated).";
> >
> > $messages["related_articles"] = "Number of related articles to
> return.";
> > $messages["related_word_length"] = "Minimum length of keyword used to
> > generate related article.";
> > +$messages["related_num_keywords"] = "Minimum number of keywords
> required to
> > determine list of related articles.";
> > $messages["related_cache"] = "Lifetime for the related article cache.";
> > $messages["parse_body"] = "Parse the body of articles to generate
> keywords.
> > (This may cause generating related posts to take longer.)";
> >
> > @@ -24,6 +27,7 @@
> >
> > $messages["related_max_articles"] = "Number Articles";
> > $messages["related_min_word_length"] = "Minimum Keyword Length";
> > +$messages["related_min_num_keywords"] = "Minimum Number of Keywords";
> > $messages["related_cache_lifetime"] = "Cache Lifetime";
> > $messages["related_parse_body"] = "Parse Body";
> > $messages["banned_keywords"] = "Banned Keywords";
> >
> > Modified: plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > ===================================================================
> > --- plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > 2007-09-10 19:45:42 UTC (rev 5924)
> > +++ plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -25,10 +25,11 @@
> > var $pluginEnabled;
> > var $numRelatedArticles;
> > var $minWordLength;
> > + var $minNumKeywords;
> > var $refreshInterval;
> > - var $cacheFolder;
> > - var $extractKeywordsFromBody;
> > - var $bannedWords;
> > + var $cacheFolder;
> > + var $extractKeywordsFromBody;
> > + var $bannedWords;
> >
> > function PluginRelated( $source = "" )
> > {
> > @@ -38,7 +39,7 @@
> > $this->desc = "The Related plugin will
> generate a
> > list of related posts.";
> > $this->author = "Paul Westbrook";
> > $this->locales = Array( "en_UK" );
> > - $this->version = "20070602";
> > + $this->version = "20070910";
> >
> >
> > if( $source == "admin" )
> > @@ -62,6 +63,7 @@
> > $this->pluginEnabled = $blogSettings->getValue(
> > "plugin_related_enabled" );
> > $this->numRelatedArticles =
> $blogSettings->getValue(
> > "plugin_related_num_articles" );
> > $this->minWordLength = $blogSettings->getValue(
> > "plugin_related_min_word_length" );
> > + $this->minNumKeywords = $blogSettings->getValue(
> > "plugin_related_min_num_keywords" );
> > $this->refreshInterval =
> $blogSettings->getValue(
> > "plugin_related_refresh_interval" );
> > $this->extractKeywordsFromBody =
> > $blogSettings->getValue( "plugin_related_extract_keywords_from_body" );
> > $this->bannedWords = $blogSettings->getValue(
> > "plugin_related_banned_keywords" );
> > @@ -116,8 +118,14 @@
> >
> > // Get the keywords
> > $keywords = $this->getArticleKeywords($article);
> > +
> > + // Make sure that there are enough keywords to make
> > + // generating the list of articles worth while
> > + if ($this->minNumKeywords != "" && count($keywords) <
> > $this->minNumKeywords) {
> > + return $relatedArticles;
> > + }
> > +
> >
> > -
> > foreach($keywords as $word) {
> > // Build the list of articles that have this
> keyword
> > lt_include(
> > PLOG_CLASS_PATH."class/dao/searchengine.class.php" );
> >
> > Modified: plugins/branches/lifetype-1.2
> /related/templates/related.template
> > ===================================================================
> > --- plugins/branches/lifetype-1.2/related/templates/related.template
> > 2007-09-10 19:45:42 UTC (rev 5924)
> > +++ plugins/branches/lifetype-1.2/related/templates/related.template
> > 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -35,6 +35,17 @@
> > </div>
> >
> > <div class="field">
> > + <label for="width">{$locale->tr("related_min_num_keywords")}</label>
>
> > + <span class="required">*</span>
> > + <div class="formHelp">{$locale->tr("related_num_keywords")}</div>
> > + <input class="text" type="text" name="minNumKeywords"
> > id="minNumKeywords"
> > + {user_cannot_override
> > + key=plugin_related_min_num_keywords}readonly="readonly"
> > + {/user_cannot_override}
> > + value="{$minNumKeywords}" width="10" />
> > + </div>
> > +
> > + <div class="field">
> > <label for="size">{$locale->tr("related_cache_lifetime")}</label>
> > <span class="required">*</span>
> > <div class="formHelp">{$locale->tr("related_cache")}</div>
> >
> > _______________________________________________
> > pLog-svn mailing list
> > pLog-svn at devel.lifetype.net
> > http://limedaley.com/mailman/listinfo/plog-svn
> >
> >
> > _______________________________________________
> > pLog-svn mailing list
> > pLog-svn at devel.lifetype.net
> > http://limedaley.com/mailman/listinfo/plog-svn
> >
>
>
>
> _______________________________________________
> pLog-svn mailing list
> pLog-svn at devel.lifetype.net
> http://limedaley.com/mailman/listinfo/plog-svn
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://limedaley.com/pipermail/plog-svn/attachments/20070914/050e19b0/attachment-0001.htm
More information about the pLog-svn
mailing list