Hello,<br> This plugin uses a brute force algorithm. In order to calculate the list of related articles for a given post, first it gets a list of all of the unique words in the post. Then it for each keyword, it uses LifeType's builtin search engine to find a list of the posts that have that keyword. After it does that for all of the keywords, it lists the returned articles based on the number of times that the article is returned by the search engine. Then the plugin returns the top x articles.
<br><br> Since this is expensive, the plugin implements a cache. The list of related articles for a post is cached in the file system, so the next time that it runs, the queries do not have to be run. The user can specify the lifetime of that cache. For example, I have mine set to 1 month, so the queries will only be run once every 30 days for each article.
<br><br> There are serveral things that could be done to improve performance<br><ol><li>Implement or use a library that will actually summarize the article text. This shouldn't simply take the nth sentence, but should generate a representative summary that contains the important keywords. This would reduce the number of search engine queries that are run
</li><li>Change the search engine to be able to return a list of articles that contains at least one of a specified list of words. This would allow the search engine to be run only once for each article</li><li>Use native tag support. When lifetype nativelly supports tags, the plugins wouldn't have to use all of the text in the article, but could just find posts that are tagged with the same tags.
<br></li></ol><br>--Paul<br><br><br><br>On 9/12/07, Ayalon <<a href="mailto:ayalon@blog.nl">ayalon@blog.nl</a>> wrote:<br>> Hi There,<br>> <br>> This is really a great plugin, but there are some problems with it.
<br>> <br>> When i switch on this plugin my apache process rise to a memory use that's<br>> incredible. Is there something to change about it? My database is pretty<br>> large with a lot of articles and I have a lot of reads on the blogs. Anybody
<br>> an idea how to optimize?<br>> <br>> I know this is not really something for the list, but I tried to reprogram<br>> the plugin making a different way of searching but I got stuck..<br>> <br>> Regards
<br>> <br>> <br>> -----Oorspronkelijk bericht-----<br>> Van: <a href="mailto:plog-svn-bounces@devel.lifetype.net">plog-svn-bounces@devel.lifetype.net</a><br>> [mailto:<a href="mailto:plog-svn-bounces@devel.lifetype.net">
plog-svn-bounces@devel.lifetype.net</a>] Namens<br>> <a href="mailto:pwestbro@devel.lifetype.net">pwestbro@devel.lifetype.net</a><br>> Verzonden: dinsdag 11 september 2007 7:10<br>> Aan: <a href="mailto:plog-svn@devel.lifetype.net">
plog-svn@devel.lifetype.net</a><br>> Onderwerp: [pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related: .<br>> class/action class/view locale templates<br>> <br>> Author: pwestbro<br>> Date: 2007-09-11 01:09:42 -0400 (Tue, 11 Sep 2007)
<br>> New Revision: 5925<br>> <br>> Modified:<br>> <br>> plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig<br>> action.class.php<br>> <br>> plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
<br>> ss.php<br>> plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php<br>> plugins/branches/lifetype-1.2/related/pluginrelated.class.php<br>> plugins/branches/lifetype-1.2/related/templates/related.template
<br>> Log:<br>> Added a setting so the minimum number for keywords that are used to generate<br>> the list of related articles<br>> <br>> <br>> Modified:<br>> plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
<br>> action.class.php<br>> ===================================================================<br>> ---<br>> plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig<br>> action.class.php
2007-09-10 19:45:42 UTC (rev 5924)<br>> +++<br>> plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig<br>> action.class.php 2007-09-11 05:09:42 UTC (rev 5925)<br>> @@ -18,43 +18,45 @@
<br>> Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307<br>> USA<br>> */<br>> <br>> - lt_include(<br>> PLOG_CLASS_PATH."class/action/admin/adminaction.class.php" );
<br>> + lt_include( PLOG_CLASS_PATH."class/action/admin/adminaction.class.php"<br>> );<br>> lt_include(<br>> PLOG_CLASS_PATH."plugins/related/class/view/pluginrelatedconfigview.class.ph<br>
> p" );<br>> <br>> $apiKeyValid = true;<br>> <br>> class PluginRelatedUpdateConfigAction extends AdminAction<br>> - {<br>> -<br>> - var $_pluginEnabled;<br>> - var $_numRelatedArticles;
<br>> - var $_minWordLength;<br>> - var $_refreshInterval;<br>> - var $_parseBody;<br>> - var $_bannedKeywords;<br>> + {<br>> +<br>> + var $_pluginEnabled;
<br>> + var $_numRelatedArticles;<br>> + var $_minWordLength;<br>> + var $_minNumKeywords;<br>> + var $_refreshInterval;<br>> + var $_parseBody;
<br>> + var $_bannedKeywords;<br>> <br>> <br>> - /**<br>> + /**<br>> * Constructor. If nothing else, it also has to call the<br>> constructor of the parent<br>> * class, BlogAction with the same parameters
<br>> */<br>> function PluginRelatedUpdateConfigAction( $actionInfo, $request )<br>> {<br>> - $this->AdminAction( $actionInfo, $request );<br>> + $this->AdminAction( $actionInfo, $request );
<br>> }<br>> -<br>> - function validate()<br>> - {<br>> +<br>> + function validate()<br>> + {<br>> <br>> $this->_pluginEnabled = $this->_request->getValue(
<br>> "pluginEnabled" );<br>> - $this->_pluginEnabled = ($this->_pluginEnabled != "" );<br>> <br>> + $this->_pluginEnabled = ($this->_pluginEnabled != "" );
<br>> <br>> <br>> <br>> $this->_numRelatedArticles = $this->_request->getValue(<br>> "numArticles" );<br>> $this->_minWordLength = $this->_request->getValue(
<br>> "minWordLength" );<br>> + $this->_minNumKeywords = $this->_request->getValue(<br>> "minNumKeywords" );<br>> $this->_refreshInterval = $this->_request->getValue( "interval"
<br>> );<br>> $this->_parseBody = $this->_request->getValue( "parseBody" );<br>> - $this->_parseBody = ($this->_parseBody != "" );<br>> <br>> + $this->_parseBody = ($this->_parseBody != "" );
<br>> <br>> $this->_bannedKeywords = $this->_request->getValue(<br>> "bannedKeywords" );<br>> <br>> <br>> @@ -98,28 +100,49 @@<br>> return false;
<br>> }<br>> }<br>> +<br>> +<br>> + if( $this->_minNumKeywords == "" ) {<br>> + $this->_view = new PluginRelatedConfigView(
<br>> $this->_blogInfo );<br>> + $this->_view->setErrorMessage(<br>> $this->_locale->tr("related_missing_num_keywords"));<br>> + $this->setCommonData();
<br>> +<br>> + return false;<br>> + }<br>> + else {<br>> + $val3 = new IntegerValidator();<br>> + if( !$val3->validate( $this->_minNumKeywords )) {
<br>> + $this->_view = new PluginRelatedConfigView(<br>> $this->_blogInfo );<br>> + $this->_view->setErrorMessage(<br>> $this->_locale->tr("related_invalid_num_keywords"));
<br>> + $this->setCommonData();<br>> +<br>> + return false;<br>> + }<br>> + }<br>> +<br>> }<br>> - return true;
<br>> - }<br>> -<br>> + return true;<br>> + }<br>> +<br>> /**<br>> * Carries out the specified action<br>> */<br>> function perform()
<br>> {<br>> // update the plugin configurations to blog setting<br>> - $blogSettings = $this->_blogInfo->getSettings();<br>> + $blogSettings = $this->_blogInfo->getSettings();
<br>> $blogSettings->setValue( "plugin_related_enabled",<br>> $this->_pluginEnabled );<br>> $blogSettings->setValue( "plugin_related_num_articles",<br>> $this->_numRelatedArticles );
<br>> $blogSettings->setValue( "plugin_related_min_word_length",<br>> $this->_minWordLength );<br>> + $blogSettings->setValue( "plugin_related_min_num_keywords",
<br>> $this->_minNumKeywords );<br>> $blogSettings->setValue( "plugin_related_refresh_interval",<br>> $this->_refreshInterval );<br>> $blogSettings->setValue(<br>
> "plugin_related_extract_keywords_from_body", $this->_parseBody );<br>> $blogSettings->setValue( "plugin_related_banned_keywords",<br>> $this->_bannedKeywords );<br>>
<br>> $this->_blogInfo->setSettings( $blogSettings );<br>> -<br>> - // save the blogs settings<br>> - $blogs = new Blogs();<br>> +<br>> + // save the blogs settings
<br>> + $blogs = new Blogs();<br>> if( !$blogs->updateBlog( $this->_blogInfo )) {<br>> $this->_view = new PluginRelatedConfigView(<br>> $this->_blogInfo );
<br>> $this->_view->setErrorMessage(<br>> $this->_locale->tr("error_updating_settings"));<br>> @@ -127,20 +150,20 @@<br>> <br>> return false;<br>> }
<br>> -<br>> - // if everything went ok...<br>> +<br>> + // if everything went ok...<br>> $this->_blogInfo->setSettings( $blogSettings );<br>> $this->_session->setValue( "blogInfo", $this->_blogInfo );
<br>> $this->saveSession();<br>> -<br>> - $this->_view = new PluginRelatedConfigView(<br>> $this->_blogInfo );<br>> - $this->_view->setSuccessMessage(
<br>> $this->_locale->tr("related_settings_saved_ok"));<br>> - $this->setCommonData();<br>> -<br>> - // clear the cache<br>> - CacheControl::resetBlogCache(
<br>> $this->_blogInfo->getId());<br>> +<br>> + $this->_view = new PluginRelatedConfigView(<br>> $this->_blogInfo );<br>> + $this->_view->setSuccessMessage(
<br>> $this->_locale->tr("related_settings_saved_ok"));<br>> + $this->setCommonData();<br>> +<br>> + // clear the cache<br>> + CacheControl::resetBlogCache(
<br>> $this->_blogInfo->getId());<br>> <br>> - return true;<br>> + return true;<br>> }<br>> }<br>> <br>> <br>> Modified:<br>> plugins/branches/lifetype-
1.2/related/class/view/pluginrelatedconfigview.cla<br>> ss.php<br>> ===================================================================<br>> ---<br>> plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
<br>> ss.php 2007-09-10 19:45:42 UTC (rev 5924)<br>> +++<br>> plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla<br>> ss.php 2007-09-11 05:09:42 UTC (rev 5925)<br>> @@ -38,6 +38,7 @@
<br>> $pluginEnabled = $blogSettings->getValue(<br>> "plugin_related_enabled" );<br>> $numArticles = $blogSettings->getValue(<br>> "plugin_related_num_articles" );
<br>> $minWordLength = $blogSettings->getValue(<br>> "plugin_related_min_word_length" );<br>> + $minNumKeyword = $blogSettings->getValue(<br>> "plugin_related_min_num_keywords" );
<br>> $refreshInterval = $blogSettings->getValue(<br>> "plugin_related_refresh_interval" );<br>> $parseBody = $blogSettings->getValue(<br>> "plugin_related_extract_keywords_from_body" );
<br>> $bannedKeywords = $blogSettings->getValue(<br>> "plugin_related_banned_keywords" );<br>> @@ -49,6 +50,7 @@<br>> $this->setValue( "pluginEnabled", $pluginEnabled );
<br>> $this->setValue( "numArticles", $numArticles );<br>> $this->setValue( "minWordLength", $minWordLength );<br>> + $this->setValue( "minNumKeywords", $minNumKeyword );
<br>> $this->setValue( "interval", $refreshInterval );<br>> $this->setValue( "parseBody", $parseBody );<br>> $this->setValue( "bannedKeywords", $bannedKeywords
<br>> );<br>> <br>> Modified: plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php<br>> ===================================================================<br>> --- plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
<br>> 2007-09-10 19:45:42 UTC (rev 5924)<br>> +++ plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php<br>> 2007-09-11 05:09:42 UTC (rev 5925)<br>> @@ -10,12 +10,15 @@<br>> $messages["related_settings_saved_ok"] = "Related Posts settings saved
<br>> successfully!";<br>> $messages["related_missing_num_articles"] = "Number of articles needs to be<br>> specified";<br>> $messages["related_invalid_num_articles"] = "Number of articles needs to be
<br>> an integer";<br>> +$messages["related_missing_num_keywords"] = "Number of keywords needs to be<br>> specified";<br>> +$messages["related_invalid_num_keywords"] = "Number of keywords needs to be
<br>> an integer";<br>> $messages["related_missing_min_length"] = "Minimum keyword length needs to<br>> be specified";<br>> $messages["related_invalid_min_length"] = "Minumum keyword length needs to
<br>> be an integer";<br>> $messages["related_banned_keywords"] = "Keywords that should not be used to<br>> find related posts (comma separated).";<br>> <br>> $messages["related_articles"] = "Number of related articles to return.";
<br>> $messages["related_word_length"] = "Minimum length of keyword used to<br>> generate related article.";<br>> +$messages["related_num_keywords"] = "Minimum number of keywords required to
<br>> determine list of related articles.";<br>> $messages["related_cache"] = "Lifetime for the related article cache.";<br>> $messages["parse_body"] = "Parse the body of articles to generate keywords.
<br>> (This may cause generating related posts to take longer.)";<br>> <br>> @@ -24,6 +27,7 @@<br>> <br>> $messages["related_max_articles"] = "Number Articles";<br>> $messages["related_min_word_length"] = "Minimum Keyword Length";
<br>> +$messages["related_min_num_keywords"] = "Minimum Number of Keywords";<br>> $messages["related_cache_lifetime"] = "Cache Lifetime";<br>> $messages["related_parse_body"] = "Parse Body";
<br>> $messages["banned_keywords"] = "Banned Keywords";<br>> <br>> Modified: plugins/branches/lifetype-1.2/related/pluginrelated.class.php<br>> ===================================================================
<br>> --- plugins/branches/lifetype-1.2/related/pluginrelated.class.php<br>> 2007-09-10 19:45:42 UTC (rev 5924)<br>> +++ plugins/branches/lifetype-1.2/related/pluginrelated.class.php<br>> 2007-09-11 05:09:42 UTC (rev 5925)
<br>> @@ -25,10 +25,11 @@<br>> var $pluginEnabled;<br>> var $numRelatedArticles;<br>> var $minWordLength;<br>> + var $minNumKeywords;<br>> var $refreshInterval;
<br>> - var $cacheFolder;<br>> - var $extractKeywordsFromBody;<br>> - var $bannedWords;<br>> + var $cacheFolder;<br>> + var $extractKeywordsFromBody;<br>> + var $bannedWords;
<br>> <br>> function PluginRelated( $source = "" )<br>> {<br>> @@ -38,7 +39,7 @@<br>> $this->desc = "The Related plugin will generate a
<br>> list of related posts.";<br>> $this->author = "Paul Westbrook";<br>> $this->locales = Array( "en_UK" );<br>> - $this->version = "20070602";
<br>> + $this->version = "20070910";<br>> <br>> <br>> if( $source == "admin" )<br>> @@ -62,6 +63,7 @@<br>> $this->pluginEnabled = $blogSettings->getValue(
<br>> "plugin_related_enabled" );<br>> $this->numRelatedArticles = $blogSettings->getValue(<br>> "plugin_related_num_articles" );<br>> $this->minWordLength = $blogSettings->getValue(
<br>> "plugin_related_min_word_length" );<br>> + $this->minNumKeywords = $blogSettings->getValue(<br>> "plugin_related_min_num_keywords" );<br>> $this->refreshInterval = $blogSettings->getValue(
<br>> "plugin_related_refresh_interval" );<br>> $this->extractKeywordsFromBody =<br>> $blogSettings->getValue( "plugin_related_extract_keywords_from_body" );<br>> $this->bannedWords = $blogSettings->getValue(
<br>> "plugin_related_banned_keywords" );<br>> @@ -116,8 +118,14 @@<br>> <br>> // Get the keywords<br>> $keywords = $this->getArticleKeywords($article);<br>> +
<br>> + // Make sure that there are enough keywords to make<br>> + // generating the list of articles worth while<br>> + if ($this->minNumKeywords != "" && count($keywords) <
<br>> $this->minNumKeywords) {<br>> + return $relatedArticles;<br>> + }<br>> +<br>> <br>> -<br>> foreach($keywords as $word) {<br>> // Build the list of articles that have this keyword
<br>> lt_include(<br>> PLOG_CLASS_PATH."class/dao/searchengine.class.php" );<br>> <br>> Modified: plugins/branches/lifetype-1.2/related/templates/related.template<br>> ===================================================================
<br>> --- plugins/branches/lifetype-1.2/related/templates/related.template<br>> 2007-09-10 19:45:42 UTC (rev 5924)<br>> +++ plugins/branches/lifetype-1.2/related/templates/related.template<br>> 2007-09-11 05:09:42 UTC (rev 5925)
<br>> @@ -35,6 +35,17 @@<br>> </div><br>> <br>> <div class="field"><br>> + <label for="width">{$locale->tr("related_min_num_keywords")}</label>
<br>> + <span class="required">*</span><br>> + <div class="formHelp">{$locale->tr("related_num_keywords")}</div><br>> + <input class="text" type="text" name="minNumKeywords"
<br>> id="minNumKeywords"<br>> + {user_cannot_override<br>> + key=plugin_related_min_num_keywords}readonly="readonly"<br>> + {/user_cannot_override}<br>> + value="{$minNumKeywords}" width="10" />
<br>> + </div><br>> +<br>> + <div class="field"><br>> <label for="size">{$locale->tr("related_cache_lifetime")}</label><br>> <span class="required">*</span>
<br>> <div class="formHelp">{$locale->tr("related_cache")}</div><br>> <br>> _______________________________________________<br>> pLog-svn mailing list<br>> <a href="mailto:pLog-svn@devel.lifetype.net">
pLog-svn@devel.lifetype.net</a><br>> <a href="http://limedaley.com/mailman/listinfo/plog-svn">http://limedaley.com/mailman/listinfo/plog-svn</a><br>> <br>> <br>> _______________________________________________
<br>> pLog-svn mailing list<br>> <a href="mailto:pLog-svn@devel.lifetype.net">pLog-svn@devel.lifetype.net</a><br>> <a href="http://limedaley.com/mailman/listinfo/plog-svn">http://limedaley.com/mailman/listinfo/plog-svn
</a><br>> <br>