[pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related: . class/action class/view locale templates
Ayalon
ayalon at blog.nl
Fri Sep 14 03:16:51 EDT 2007
Hi Paul,
1. Mine is set to 5, not really helping
2. 720 is set.
3. Also no body checking.
What i think is that the search engine that's build in is not really build
for so many queries at a time. Look at the wordpress related plug-in:
http://peter.mapledesign.co.uk/weblog/wp-content/uploads/related-posts.phps
They do the same thing, but instead of using the build in search engine of
wordpress they directly go into the database. If we would do these and save
those queries?! I know that it's maybe not the nicest option, but is this
something you can try? Personally I think it's the only solution for having
it work without taking to much mem currently.
Attached is my optimized related plug in, now it takes a bit less memory to
do the process, but still not perfect..
Regards,
Ayalon
Van: plog-svn-bounces at devel.lifetype.net
[mailto:plog-svn-bounces at devel.lifetype.net] Namens Paul Westbrook
Verzonden: donderdag 13 september 2007 22:53
Aan: LifeType Developer List
Onderwerp: Re: [pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related:
. class/action class/view locale templates
Hello,
Some of the options in the admin interface will help reduce the CPU
usage:
1. Minimum Keyword Length - Set this to a larger number (Will reduce
the number of search engine queries)
2. Cache Lifetime - This will increase the time between the list of
related articles is rebuilt.
3. Parse Body - Uncheck this, and the plugin will only get keywords
from the subject.
--Paul
On 9/12/07, Paul Westbrook <paul at westbrooks.org> wrote:
Hello,
This plugin uses a brute force algorithm. In order to calculate the list
of related articles for a given post, first it gets a list of all of the
unique words in the post. Then it for each keyword, it uses LifeType's
builtin search engine to find a list of the posts that have that keyword.
After it does that for all of the keywords, it lists the returned articles
based on the number of times that the article is returned by the search
engine. Then the plugin returns the top x articles.
Since this is expensive, the plugin implements a cache. The list of
related articles for a post is cached in the file system, so the next time
that it runs, the queries do not have to be run. The user can specify the
lifetime of that cache. For example, I have mine set to 1 month, so the
queries will only be run once every 30 days for each article.
There are serveral things that could be done to improve performance
1. Implement or use a library that will actually summarize the article
text. This shouldn't simply take the nth sentence, but should generate a
representative summary that contains the important keywords. This would
reduce the number of search engine queries that are run
2. Change the search engine to be able to return a list of articles
that contains at least one of a specified list of words. This would allow
the search engine to be run only once for each article
3. Use native tag support. When lifetype nativelly supports tags, the
plugins wouldn't have to use all of the text in the article, but could just
find posts that are tagged with the same tags.
--Paul
On 9/12/07, Ayalon < ayalon at blog.nl <mailto:ayalon at blog.nl> > wrote:
> Hi There,
>
> This is really a great plugin, but there are some problems with it.
>
> When i switch on this plugin my apache process rise to a memory use that's
> incredible. Is there something to change about it? My database is pretty
> large with a lot of articles and I have a lot of reads on the blogs.
Anybody
> an idea how to optimize?
>
> I know this is not really something for the list, but I tried to reprogram
> the plugin making a different way of searching but I got stuck..
>
> Regards
>
>
> -----Oorspronkelijk bericht-----
> Van: plog-svn-bounces at devel.lifetype.net
<mailto:plog-svn-bounces at devel.lifetype.net>
> [mailto: plog-svn-bounces at devel.lifetype.net
<mailto:plog-svn-bounces at devel.lifetype.net> ] Namens
> pwestbro at devel.lifetype.net
> Verzonden: dinsdag 11 september 2007 7:10
> Aan: plog-svn at devel.lifetype.net
> Onderwerp: [pLog-svn] r5925 - in plugins/branches/lifetype-1.2/related: .
> class/action class/view locale templates
>
> Author: pwestbro
> Date: 2007-09-11 01:09:42 -0400 (Tue, 11 Sep 2007)
> New Revision: 5925
>
> Modified:
>
>
plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
> action.class.php
>
> plugins/branches/lifetype-1.2
/related/class/view/pluginrelatedconfigview.cla
> ss.php
> plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> plugins/branches/lifetype-1.2/related/templates/related.template
> Log:
> Added a setting so the minimum number for keywords that are used to
generate
> the list of related articles
>
>
> Modified:
>
plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
> action.class.php
> ===================================================================
> ---
>
plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
> action.class.php 2007-09-10 19:45:42 UTC (rev 5924)
> +++
>
plugins/branches/lifetype-1.2/related/class/action/pluginrelatedupdateconfig
> action.class.php 2007-09-11 05:09:42 UTC (rev 5925)
> @@ -18,43 +18,45 @@
> Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
> USA
> */
>
> - lt_include(
> PLOG_CLASS_PATH."class/action/admin/adminaction.class.php" );
> + lt_include(
PLOG_CLASS_PATH."class/action/admin/adminaction.class.php"
> );
> lt_include(
>
PLOG_CLASS_PATH."plugins/related/class/view/pluginrelatedconfigview.class.ph
> p" );
>
> $apiKeyValid = true;
>
> class PluginRelatedUpdateConfigAction extends AdminAction
> - {
> -
> - var $_pluginEnabled;
> - var $_numRelatedArticles;
> - var $_minWordLength;
> - var $_refreshInterval;
> - var $_parseBody;
> - var $_bannedKeywords;
> + {
> +
> + var $_pluginEnabled;
> + var $_numRelatedArticles;
> + var $_minWordLength;
> + var $_minNumKeywords;
> + var $_refreshInterval;
> + var $_parseBody;
> + var $_bannedKeywords;
>
>
> - /**
> + /**
> * Constructor. If nothing else, it also has to call the
> constructor of the parent
> * class, BlogAction with the same parameters
> */
> function PluginRelatedUpdateConfigAction( $actionInfo, $request )
> {
> - $this->AdminAction( $actionInfo, $request );
> + $this->AdminAction( $actionInfo, $request );
> }
> -
> - function validate()
> - {
> +
> + function validate()
> + {
>
> $this->_pluginEnabled = $this->_request->getValue(
> "pluginEnabled" );
> - $this->_pluginEnabled = ($this->_pluginEnabled != "" );
>
> + $this->_pluginEnabled = ($this->_pluginEnabled != "" );
>
>
>
> $this->_numRelatedArticles = $this->_request->getValue(
> "numArticles" );
> $this->_minWordLength = $this->_request->getValue(
> "minWordLength" );
> + $this->_minNumKeywords = $this->_request->getValue(
> "minNumKeywords" );
> $this->_refreshInterval = $this->_request->getValue(
"interval"
> );
> $this->_parseBody = $this->_request->getValue( "parseBody" );
> - $this->_parseBody = ($this->_parseBody != "" );
>
> + $this->_parseBody = ($this->_parseBody != "" );
>
> $this->_bannedKeywords = $this->_request->getValue(
> "bannedKeywords" );
>
>
> @@ -98,28 +100,49 @@
> return false;
> }
> }
> +
> +
> + if( $this->_minNumKeywords == "" ) {
> + $this->_view = new PluginRelatedConfigView(
> $this->_blogInfo );
> + $this->_view->setErrorMessage(
> $this->_locale->tr("related_missing_num_keywords"));
> + $this->setCommonData();
> +
> + return false;
> + }
> + else {
> + $val3 = new IntegerValidator();
> + if( !$val3->validate( $this->_minNumKeywords )) {
> + $this->_view = new PluginRelatedConfigView(
> $this->_blogInfo );
> + $this->_view->setErrorMessage(
> $this->_locale->tr("related_invalid_num_keywords"));
> + $this->setCommonData();
> +
> + return false;
> + }
> + }
> +
> }
> - return true;
> - }
> -
> + return true;
> + }
> +
> /**
> * Carries out the specified action
> */
> function perform()
> {
> // update the plugin configurations to blog setting
> - $blogSettings = $this->_blogInfo->getSettings();
> + $blogSettings = $this->_blogInfo->getSettings();
> $blogSettings->setValue( "plugin_related_enabled",
> $this->_pluginEnabled );
> $blogSettings->setValue( "plugin_related_num_articles",
> $this->_numRelatedArticles );
> $blogSettings->setValue( "plugin_related_min_word_length",
> $this->_minWordLength );
> + $blogSettings->setValue( "plugin_related_min_num_keywords",
> $this->_minNumKeywords );
> $blogSettings->setValue( "plugin_related_refresh_interval",
> $this->_refreshInterval );
> $blogSettings->setValue(
> "plugin_related_extract_keywords_from_body", $this->_parseBody );
> $blogSettings->setValue( "plugin_related_banned_keywords",
> $this->_bannedKeywords );
>
> $this->_blogInfo->setSettings( $blogSettings );
> -
> - // save the blogs settings
> - $blogs = new Blogs();
> +
> + // save the blogs settings
> + $blogs = new Blogs();
> if( !$blogs->updateBlog( $this->_blogInfo )) {
> $this->_view = new PluginRelatedConfigView(
> $this->_blogInfo );
> $this->_view->setErrorMessage(
> $this->_locale->tr("error_updating_settings"));
> @@ -127,20 +150,20 @@
>
> return false;
> }
> -
> - // if everything went ok...
> +
> + // if everything went ok...
> $this->_blogInfo->setSettings( $blogSettings );
> $this->_session->setValue( "blogInfo", $this->_blogInfo );
> $this->saveSession();
> -
> - $this->_view = new PluginRelatedConfigView(
> $this->_blogInfo );
> - $this->_view->setSuccessMessage(
> $this->_locale->tr("related_settings_saved_ok"));
> - $this->setCommonData();
> -
> - // clear the cache
> - CacheControl::resetBlogCache(
> $this->_blogInfo->getId());
> +
> + $this->_view = new PluginRelatedConfigView(
> $this->_blogInfo );
> + $this->_view->setSuccessMessage(
> $this->_locale->tr("related_settings_saved_ok"));
> + $this->setCommonData();
> +
> + // clear the cache
> + CacheControl::resetBlogCache(
> $this->_blogInfo->getId());
>
> - return true;
> + return true;
> }
> }
>
>
> Modified:
> plugins/branches/lifetype-
1.2/related/class/view/pluginrelatedconfigview.cla
> ss.php
> ===================================================================
> ---
>
plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
> ss.php 2007-09-10 19:45:42 UTC (rev 5924)
> +++
>
plugins/branches/lifetype-1.2/related/class/view/pluginrelatedconfigview.cla
> ss.php 2007-09-11 05:09:42 UTC (rev 5925)
> @@ -38,6 +38,7 @@
> $pluginEnabled = $blogSettings->getValue(
> "plugin_related_enabled" );
> $numArticles = $blogSettings->getValue(
> "plugin_related_num_articles" );
> $minWordLength = $blogSettings->getValue(
> "plugin_related_min_word_length" );
> + $minNumKeyword = $blogSettings->getValue(
> "plugin_related_min_num_keywords" );
> $refreshInterval = $blogSettings->getValue(
> "plugin_related_refresh_interval" );
> $parseBody = $blogSettings->getValue(
> "plugin_related_extract_keywords_from_body" );
> $bannedKeywords = $blogSettings->getValue(
> "plugin_related_banned_keywords" );
> @@ -49,6 +50,7 @@
> $this->setValue( "pluginEnabled", $pluginEnabled
);
> $this->setValue( "numArticles", $numArticles );
> $this->setValue( "minWordLength", $minWordLength
);
> + $this->setValue( "minNumKeywords", $minNumKeyword
);
> $this->setValue( "interval", $refreshInterval );
> $this->setValue( "parseBody", $parseBody );
> $this->setValue( "bannedKeywords", $bannedKeywords
> );
>
> Modified: plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> ===================================================================
> --- plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> 2007-09-10 19:45:42 UTC (rev 5924)
> +++ plugins/branches/lifetype-1.2/related/locale/locale_en_UK.php
> 2007-09-11 05:09:42 UTC (rev 5925)
> @@ -10,12 +10,15 @@
> $messages["related_settings_saved_ok"] = "Related Posts settings saved
> successfully!";
> $messages["related_missing_num_articles"] = "Number of articles needs to
be
> specified";
> $messages["related_invalid_num_articles"] = "Number of articles needs to
be
> an integer";
> +$messages["related_missing_num_keywords"] = "Number of keywords needs to
be
> specified";
> +$messages["related_invalid_num_keywords"] = "Number of keywords needs to
be
> an integer";
> $messages["related_missing_min_length"] = "Minimum keyword length needs
to
> be specified";
> $messages["related_invalid_min_length"] = "Minumum keyword length needs
to
> be an integer";
> $messages["related_banned_keywords"] = "Keywords that should not be used
to
> find related posts (comma separated).";
>
> $messages["related_articles"] = "Number of related articles to return.";
> $messages["related_word_length"] = "Minimum length of keyword used to
> generate related article.";
> +$messages["related_num_keywords"] = "Minimum number of keywords required
to
> determine list of related articles.";
> $messages["related_cache"] = "Lifetime for the related article cache.";
> $messages["parse_body"] = "Parse the body of articles to generate
keywords.
> (This may cause generating related posts to take longer.)";
>
> @@ -24,6 +27,7 @@
>
> $messages["related_max_articles"] = "Number Articles";
> $messages["related_min_word_length"] = "Minimum Keyword Length";
> +$messages["related_min_num_keywords"] = "Minimum Number of Keywords";
> $messages["related_cache_lifetime"] = "Cache Lifetime";
> $messages["related_parse_body"] = "Parse Body";
> $messages["banned_keywords"] = "Banned Keywords";
>
> Modified: plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> ===================================================================
> --- plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> 2007-09-10 19:45:42 UTC (rev 5924)
> +++ plugins/branches/lifetype-1.2/related/pluginrelated.class.php
> 2007-09-11 05:09:42 UTC (rev 5925)
> @@ -25,10 +25,11 @@
> var $pluginEnabled;
> var $numRelatedArticles;
> var $minWordLength;
> + var $minNumKeywords;
> var $refreshInterval;
> - var $cacheFolder;
> - var $extractKeywordsFromBody;
> - var $bannedWords;
> + var $cacheFolder;
> + var $extractKeywordsFromBody;
> + var $bannedWords;
>
> function PluginRelated( $source = "" )
> {
> @@ -38,7 +39,7 @@
> $this->desc = "The Related plugin will generate
a
> list of related posts.";
> $this->author = "Paul Westbrook";
> $this->locales = Array( "en_UK" );
> - $this->version = "20070602";
> + $this->version = "20070910";
>
>
> if( $source == "admin" )
> @@ -62,6 +63,7 @@
> $this->pluginEnabled = $blogSettings->getValue(
> "plugin_related_enabled" );
> $this->numRelatedArticles =
$blogSettings->getValue(
> "plugin_related_num_articles" );
> $this->minWordLength = $blogSettings->getValue(
> "plugin_related_min_word_length" );
> + $this->minNumKeywords = $blogSettings->getValue(
> "plugin_related_min_num_keywords" );
> $this->refreshInterval = $blogSettings->getValue(
> "plugin_related_refresh_interval" );
> $this->extractKeywordsFromBody =
> $blogSettings->getValue( "plugin_related_extract_keywords_from_body" );
> $this->bannedWords = $blogSettings->getValue(
> "plugin_related_banned_keywords" );
> @@ -116,8 +118,14 @@
>
> // Get the keywords
> $keywords = $this->getArticleKeywords($article);
> +
> + // Make sure that there are enough keywords to make
> + // generating the list of articles worth while
> + if ($this->minNumKeywords != "" && count($keywords) <
> $this->minNumKeywords) {
> + return $relatedArticles;
> + }
> +
>
> -
> foreach($keywords as $word) {
> // Build the list of articles that have this keyword
> lt_include(
> PLOG_CLASS_PATH."class/dao/searchengine.class.php" );
>
> Modified: plugins/branches/lifetype-1.2/related/templates/related.template
> ===================================================================
> --- plugins/branches/lifetype-1.2/related/templates/related.template
> 2007-09-10 19:45:42 UTC (rev 5924)
> +++ plugins/branches/lifetype-1.2/related/templates/related.template
> 2007-09-11 05:09:42 UTC (rev 5925)
> @@ -35,6 +35,17 @@
> </div>
>
> <div class="field">
> + <label for="width">{$locale->tr("related_min_num_keywords")}</label>
> + <span class="required">*</span>
> + <div class="formHelp">{$locale->tr("related_num_keywords")}</div>
> + <input class="text" type="text" name="minNumKeywords"
> id="minNumKeywords"
> + {user_cannot_override
> + key=plugin_related_min_num_keywords}readonly="readonly"
> + {/user_cannot_override}
> + value="{$minNumKeywords}" width="10" />
> + </div>
> +
> + <div class="field">
> <label for="size">{$locale->tr("related_cache_lifetime")}</label>
> <span class="required">*</span>
> <div class="formHelp">{$locale->tr("related_cache")}</div>
>
> _______________________________________________
> pLog-svn mailing list
> pLog-svn at devel.lifetype.net
> http://limedaley.com/mailman/listinfo/plog-svn
>
>
> _______________________________________________
> pLog-svn mailing list
> pLog-svn at devel.lifetype.net
> http://limedaley.com/mailman/listinfo/plog-svn
<http://limedaley.com/mailman/listinfo/plog-svn>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://limedaley.com/pipermail/plog-svn/attachments/20070914/bdf7eab0/attachment-0001.htm
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pluginrelated.class.php
Type: application/octet-stream
Size: 13666 bytes
Desc: not available
Url : http://limedaley.com/pipermail/plog-svn/attachments/20070914/bdf7eab0/attachment-0001.obj
More information about the pLog-svn
mailing list