[pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related: . class/action class/view locale templates
Paul Westbrook
paul at westbrooks.org
Thu Sep 13 16:52:46 EDT 2007
Hello,
Some of the options in the admin interface will help reduce the CPU
usage:
1. Minimum Keyword Length - Set this to a larger number (Will reduce
the number of search engine queries)
2. Cache Lifetime - This will increase the time between the list of
related articles is rebuilt.
3. Parse Body - Uncheck this, and the plugin will only get keywords
from the subject.
--Paul
On 9/12/07, Paul Westbrook <paul at westbrooks.org> wrote:
>
> Hello,
> This plugin uses a brute force algorithm. In order to calculate the
> list of related articles for a given post, first it gets a list of all of
> the unique words in the post. Then it for each keyword, it uses LifeType's
> builtin search engine to find a list of the posts that have that
> keyword. After it does that for all of the keywords, it lists the returned
> articles based on the number of times that the article is returned by the
> search engine. Then the plugin returns the top x articles.
>
> Since this is expensive, the plugin implements a cache. The list of
> related articles for a post is cached in the file system, so the next time
> that it runs, the queries do not have to be run. The user can specify the
> lifetime of that cache. For example, I have mine set to 1 month, so the
> queries will only be run once every 30 days for each article.
>
> There are serveral things that could be done to improve performance
>
> 1. Implement or use a library that will actually summarize the
> article text. This shouldn't simply take the nth sentence, but should
> generate a representative summary that contains the important keywords.
> This would reduce the number of search engine queries that are run
> 2. Change the search engine to be able to return a list of articles
> that contains at least one of a specified list of words. This would allow
> the search engine to be run only once for each article
> 3. Use native tag support. When lifetype nativelly supports tags,
> the plugins wouldn't have to use all of the text in the article, but could
> just find posts that are tagged with the same tags.
>
>
> --Paul
>
>
>
> On 9/12/07, Ayalon <ayalon at blog.nl> wrote:
> > Hi There,
> >
> > This is really a great plugin, but there are some problems with it.
> >
> > When i switch on this plugin my apache process rise to a memory use
> that's
> > incredible. Is there something to change about it? My database is pretty
> > large with a lot of articles and I have a lot of reads on the blogs.
> Anybody
> > an idea how to optimize?
> >
> > I know this is not really something for the list, but I tried to
> reprogram
> > the plugin making a different way of searching but I got stuck..
> >
> > Regards
> >
> >
> > -----Oorspronkelijk bericht-----
> > Van: plog-svn-bounces at devel.lifetype.net
> > [mailto: plog-svn-bounces at devel.lifetype.net] Namens
> > pwestbro at devel.lifetype.net
> > Verzonden: dinsdag 11 september 2007 7:10
> > Aan: plog-svn at devel.lifetype.net
> > Onderwerp: [pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related:
> .
> > class/action class/view locale templates
> >
> > Author: pwestbro
> > Date: 2007-09-11 01:09:42 -0400 (Tue, 11 Sep 2007)
> > New Revision: 5925
> >
> > Modified:
> >
> > plugins/branches/lifetype-1.2
> /related/class/action/pluginrelatedupdateconfig
> > action.class.php
> >
> > plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
>
> > ss.php
> > plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > plugins/branches/lifetype-1.2/related/templates/related.template
> > Log:
> > Added a setting so the minimum number for keywords that are used to
> generate
> > the list of related articles
> >
> >
> > Modified:
> > plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
>
> > action.class.php
> > ===================================================================
> > ---
> > plugins/branches/lifetype-1.2
> /related/class/action/pluginrelatedupdateconfig
> > action.class.php 2007-09-10 19:45:42 UTC (rev 5924)
> > +++
> > plugins/branches/lifetype-1.2
> /related/class/action/pluginrelatedupdateconfig
> > action.class.php 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -18,43 +18,45 @@
> > Foundation, Inc., 59 Temple Place, Suite 330, Boston,
> MA 02111-1307
> > USA
> > */
> >
> > - lt_include(
> > PLOG_CLASS_PATH."class/action/admin/adminaction.class.php" );
> > + lt_include(
> PLOG_CLASS_PATH."class/action/admin/adminaction.class.php"
> > );
> > lt_include(
> >
> PLOG_CLASS_PATH."plugins/related/class/view/pluginrelatedconfigview.class.ph
> > p" );
> >
> > $apiKeyValid = true;
> >
> > class PluginRelatedUpdateConfigAction extends AdminAction
> > - {
> > -
> > - var $_pluginEnabled;
> > - var $_numRelatedArticles;
> > - var $_minWordLength;
> > - var $_refreshInterval;
> > - var $_parseBody;
> > - var $_bannedKeywords;
> > + {
> > +
> > + var $_pluginEnabled;
> > + var $_numRelatedArticles;
> > + var $_minWordLength;
> > + var $_minNumKeywords;
> > + var $_refreshInterval;
> > + var $_parseBody;
> > + var $_bannedKeywords;
> >
> >
> > - /**
> > + /**
> > * Constructor. If nothing else, it also has to call the
> > constructor of the parent
> > * class, BlogAction with the same parameters
> > */
> > function PluginRelatedUpdateConfigAction( $actionInfo, $request
> )
> > {
> > - $this->AdminAction( $actionInfo, $request );
> > + $this->AdminAction( $actionInfo, $request );
> > }
> > -
> > - function validate()
> > - {
> > +
> > + function validate()
> > + {
> >
> > $this->_pluginEnabled = $this->_request->getValue(
> > "pluginEnabled" );
> > - $this->_pluginEnabled = ($this->_pluginEnabled != "" );
> >
> > + $this->_pluginEnabled = ($this->_pluginEnabled != "" );
> >
> >
> >
> > $this->_numRelatedArticles = $this->_request->getValue(
> > "numArticles" );
> > $this->_minWordLength = $this->_request->getValue(
> > "minWordLength" );
> > + $this->_minNumKeywords = $this->_request->getValue(
> > "minNumKeywords" );
> > $this->_refreshInterval = $this->_request->getValue(
> "interval"
> > );
> > $this->_parseBody = $this->_request->getValue( "parseBody"
> );
> > - $this->_parseBody = ($this->_parseBody != "" );
> >
> > + $this->_parseBody = ($this->_parseBody != "" );
> >
> > $this->_bannedKeywords = $this->_request->getValue(
> > "bannedKeywords" );
> >
> >
> > @@ -98,28 +100,49 @@
> > return false;
> > }
> > }
> > +
> > +
> > + if( $this->_minNumKeywords == "" ) {
> > + $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > + $this->_view->setErrorMessage(
> > $this->_locale->tr("related_missing_num_keywords"));
> > + $this->setCommonData();
> > +
> > + return false;
> > + }
> > + else {
> > + $val3 = new IntegerValidator();
> > + if( !$val3->validate( $this->_minNumKeywords )) {
> > + $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > + $this->_view->setErrorMessage(
> > $this->_locale->tr("related_invalid_num_keywords"));
> > + $this->setCommonData();
> > +
> > + return false;
> > + }
> > + }
> > +
> > }
> > - return true;
> > - }
> > -
> > + return true;
> > + }
> > +
> > /**
> > * Carries out the specified action
> > */
> > function perform()
> > {
> > // update the plugin configurations to blog setting
> > - $blogSettings = $this->_blogInfo->getSettings();
> > + $blogSettings =
> $this->_blogInfo->getSettings();
> > $blogSettings->setValue( "plugin_related_enabled",
> > $this->_pluginEnabled );
> > $blogSettings->setValue( "plugin_related_num_articles",
> > $this->_numRelatedArticles );
> > $blogSettings->setValue( "plugin_related_min_word_length",
> > $this->_minWordLength );
> > + $blogSettings->setValue( "plugin_related_min_num_keywords",
>
> > $this->_minNumKeywords );
> > $blogSettings->setValue( "plugin_related_refresh_interval",
> > $this->_refreshInterval );
> > $blogSettings->setValue(
> > "plugin_related_extract_keywords_from_body", $this->_parseBody );
> > $blogSettings->setValue( "plugin_related_banned_keywords",
> > $this->_bannedKeywords );
> >
> > $this->_blogInfo->setSettings( $blogSettings );
> > -
> > - // save the blogs settings
> > - $blogs = new Blogs();
> > +
> > + // save the blogs settings
> > + $blogs = new Blogs();
> > if( !$blogs->updateBlog( $this->_blogInfo )) {
> > $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > $this->_view->setErrorMessage(
> > $this->_locale->tr("error_updating_settings"));
> > @@ -127,20 +150,20 @@
> >
> > return false;
> > }
> > -
> > - // if everything went ok...
> > +
> > + // if everything went ok...
> > $this->_blogInfo->setSettings( $blogSettings );
> > $this->_session->setValue( "blogInfo", $this->_blogInfo );
> > $this->saveSession();
> > -
> > - $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > - $this->_view->setSuccessMessage(
> > $this->_locale->tr("related_settings_saved_ok"));
> > - $this->setCommonData();
> > -
> > - // clear the cache
> > - CacheControl::resetBlogCache(
> > $this->_blogInfo->getId());
> > +
> > + $this->_view = new PluginRelatedConfigView(
> > $this->_blogInfo );
> > + $this->_view->setSuccessMessage(
> > $this->_locale->tr("related_settings_saved_ok"));
> > + $this->setCommonData();
> > +
> > + // clear the cache
> > + CacheControl::resetBlogCache(
> > $this->_blogInfo->getId());
> >
> > - return true;
> > + return true;
> > }
> > }
> >
> >
> > Modified:
> > plugins/branches/lifetype- 1.2
> /related/class/view/pluginrelatedconfigview.cla
> > ss.php
> > ===================================================================
> > ---
> > plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
>
> > ss.php 2007-09-10 19:45:42 UTC (rev 5924)
> > +++
> > plugins/branches/lifetype-1.2
> /related/class/view/pluginrelatedconfigview.cla
> > ss.php 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -38,6 +38,7 @@
> > $pluginEnabled = $blogSettings->getValue(
> > "plugin_related_enabled" );
> > $numArticles = $blogSettings->getValue(
> > "plugin_related_num_articles" );
> > $minWordLength = $blogSettings->getValue(
> > "plugin_related_min_word_length" );
> > + $minNumKeyword = $blogSettings->getValue(
> > "plugin_related_min_num_keywords" );
> > $refreshInterval = $blogSettings->getValue(
> > "plugin_related_refresh_interval" );
> > $parseBody = $blogSettings->getValue(
> > "plugin_related_extract_keywords_from_body" );
> > $bannedKeywords = $blogSettings->getValue(
> > "plugin_related_banned_keywords" );
> > @@ -49,6 +50,7 @@
> > $this->setValue( "pluginEnabled", $pluginEnabled
> );
> > $this->setValue( "numArticles", $numArticles );
> > $this->setValue( "minWordLength", $minWordLength
> );
> > + $this->setValue( "minNumKeywords",
> $minNumKeyword );
> > $this->setValue( "interval", $refreshInterval );
> > $this->setValue( "parseBody", $parseBody );
> > $this->setValue( "bannedKeywords",
> $bannedKeywords
> > );
> >
> > Modified: plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > ===================================================================
> > --- plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > 2007-09-10 19:45:42 UTC (rev 5924)
> > +++ plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> > 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -10,12 +10,15 @@
> > $messages["related_settings_saved_ok"] = "Related Posts settings saved
> > successfully!";
> > $messages["related_missing_num_articles"] = "Number of articles needs
> to be
> > specified";
> > $messages["related_invalid_num_articles"] = "Number of articles needs
> to be
> > an integer";
> > +$messages["related_missing_num_keywords"] = "Number of keywords needs
> to be
> > specified";
> > +$messages["related_invalid_num_keywords"] = "Number of keywords needs
> to be
> > an integer";
> > $messages["related_missing_min_length"] = "Minimum keyword length needs
> to
> > be specified";
> > $messages["related_invalid_min_length"] = "Minumum keyword length needs
> to
> > be an integer";
> > $messages["related_banned_keywords"] = "Keywords that should not be
> used to
> > find related posts (comma separated).";
> >
> > $messages["related_articles"] = "Number of related articles to
> return.";
> > $messages["related_word_length"] = "Minimum length of keyword used to
> > generate related article.";
> > +$messages["related_num_keywords"] = "Minimum number of keywords
> required to
> > determine list of related articles.";
> > $messages["related_cache"] = "Lifetime for the related article cache.";
> > $messages["parse_body"] = "Parse the body of articles to generate
> keywords.
> > (This may cause generating related posts to take longer.)";
> >
> > @@ -24,6 +27,7 @@
> >
> > $messages["related_max_articles"] = "Number Articles";
> > $messages["related_min_word_length"] = "Minimum Keyword Length";
> > +$messages["related_min_num_keywords"] = "Minimum Number of Keywords";
> > $messages["related_cache_lifetime"] = "Cache Lifetime";
> > $messages["related_parse_body"] = "Parse Body";
> > $messages["banned_keywords"] = "Banned Keywords";
> >
> > Modified: plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > ===================================================================
> > --- plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > 2007-09-10 19:45:42 UTC (rev 5924)
> > +++ plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> > 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -25,10 +25,11 @@
> > var $pluginEnabled;
> > var $numRelatedArticles;
> > var $minWordLength;
> > + var $minNumKeywords;
> > var $refreshInterval;
> > - var $cacheFolder;
> > - var $extractKeywordsFromBody;
> > - var $bannedWords;
> > + var $cacheFolder;
> > + var $extractKeywordsFromBody;
> > + var $bannedWords;
> >
> > function PluginRelated( $source = "" )
> > {
> > @@ -38,7 +39,7 @@
> > $this->desc = "The Related plugin will
> generate a
> > list of related posts.";
> > $this->author = "Paul Westbrook";
> > $this->locales = Array( "en_UK" );
> > - $this->version = "20070602";
> > + $this->version = "20070910";
> >
> >
> > if( $source == "admin" )
> > @@ -62,6 +63,7 @@
> > $this->pluginEnabled = $blogSettings->getValue(
> > "plugin_related_enabled" );
> > $this->numRelatedArticles =
> $blogSettings->getValue(
> > "plugin_related_num_articles" );
> > $this->minWordLength = $blogSettings->getValue(
> > "plugin_related_min_word_length" );
> > + $this->minNumKeywords = $blogSettings->getValue(
> > "plugin_related_min_num_keywords" );
> > $this->refreshInterval =
> $blogSettings->getValue(
> > "plugin_related_refresh_interval" );
> > $this->extractKeywordsFromBody =
> > $blogSettings->getValue( "plugin_related_extract_keywords_from_body" );
> > $this->bannedWords = $blogSettings->getValue(
> > "plugin_related_banned_keywords" );
> > @@ -116,8 +118,14 @@
> >
> > // Get the keywords
> > $keywords = $this->getArticleKeywords($article);
> > +
> > + // Make sure that there are enough keywords to make
> > + // generating the list of articles worth while
> > + if ($this->minNumKeywords != "" && count($keywords) <
> > $this->minNumKeywords) {
> > + return $relatedArticles;
> > + }
> > +
> >
> > -
> > foreach($keywords as $word) {
> > // Build the list of articles that have this
> keyword
> > lt_include(
> > PLOG_CLASS_PATH."class/dao/searchengine.class.php" );
> >
> > Modified: plugins/branches/lifetype-1.2
> /related/templates/related.template
> > ===================================================================
> > --- plugins/branches/lifetype-1.2/related/templates/related.template
> > 2007-09-10 19:45:42 UTC (rev 5924)
> > +++ plugins/branches/lifetype-1.2/related/templates/related.template
> > 2007-09-11 05:09:42 UTC (rev 5925)
> > @@ -35,6 +35,17 @@
> > </div>
> >
> > <div class="field">
> > + <label for="width">{$locale->tr("related_min_num_keywords")}</label>
>
> > + <span class="required">*</span>
> > + <div class="formHelp">{$locale->tr("related_num_keywords")}</div>
> > + <input class="text" type="text" name="minNumKeywords"
> > id="minNumKeywords"
> > + {user_cannot_override
> > + key=plugin_related_min_num_keywords}readonly="readonly"
> > + {/user_cannot_override}
> > + value="{$minNumKeywords}" width="10" />
> > + </div>
> > +
> > + <div class="field">
> > <label for="size">{$locale->tr("related_cache_lifetime")}</label>
> > <span class="required">*</span>
> > <div class="formHelp">{$locale->tr("related_cache")}</div>
> >
> > _______________________________________________
> > pLog-svn mailing list
> > pLog-svn at devel.lifetype.net
> > http://limedaley.com/mailman/listinfo/plog-svn
> >
> >
> > _______________________________________________
> > pLog-svn mailing list
> > pLog-svn at devel.lifetype.net
> > http://limedaley.com/mailman/listinfo/plog-svn
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://limedaley.com/pipermail/plog-svn/attachments/20070913/a4e28627/attachment-0001.htm
More information about the pLog-svn
mailing list