[pLog-svn] r3251 - plog/trunk/class/security

Paul Westbrook paul at westbrooks.org
Tue Apr 18 05:02:55 GMT 2006


Hello,
     The code after revision 2348 ran all the filters again.  There  
are two thoughts about whether this is the correct thing.
1) By running all of the filters again, we can proactively train  
filters on messages that have been blocked by other filters.

      This could cause a lot more processing.

2) By running only the filters that ran originally, gives them a  
chance to clean up.

     This fixes the problem that was happening with the bayesian  
filter, where the message was deleted after being mistrained.  In  
this case there wasn't a way to retrain the bayesian filter.


I am not tied to one of these two options.


--Paul



On Apr 17, 2006, at 7:36 PM, Jon Daley wrote:

> 	Yes, the part I was wondering about was your last sentence.  Did  
> the code used to do that, and now doesn't, or did it not ever do that?
>
> On Sun, 16 Apr 2006, Paul Westbrook wrote:
>> Hello,
>>  Originally, before change r2348, as soon as one filter wanted to  
>> reject the event, none of the other filters ran.  This caused a  
>> problem when I got a spam message that was not caught by the  
>> bayesian filter (So it trained it as ham), and then was later  
>> rejected by a following filter.  At that point the message wasn't  
>> around to be trained as spam.
>>
>>  With that change, all of the filters would be run again with a  
>> flag that indicates that the even is being rejected.  Then the  
>> filters can clean up anything that they did.  For example, the  
>> bayesian filter will learn it as spam, if it originally learned it  
>> as ham.
>>
>>  The problem with this fix was that all of the filters were being  
>> run to clean up.  Even the ones that never ran originally.  This  
>> potentially was causing a lot of unecessary code to be run.  The  
>> change that I just made fixed this to only run the filters that  
>> ran originally would be run to clean up.
>>
>>  Now that I think of it, it might be cool to give all filters a  
>> chance to do something with a message that has already been rejected.
>>
>>
>> --Paul
>>
>>
>> On Apr 16, 2006, at 7:18 PM, Jon Daley wrote:
>>
>>> 	Naive question: I understand the part about filters that already  
>>> ran should run again, but is the desired behavior that filters  
>>> that have not run yet should be run, or not at all?  I am not  
>>> sure what the code currently does, or if this addition changes  
>>> that behavior.
>>> On Sun, 16 Apr 2006, pwestbro at devel.lifetype.net wrote:
>>>> Author: pwestbro
>>>> Date: 2006-04-16 03:22:26 +0000 (Sun, 16 Apr 2006)
>>>> New Revision: 3251
>>>> Modified:
>>>>  plog/trunk/class/security/pipeline.class.php
>>>> Log:
>>>> Fix to pipeline where if one of the pipeline filters returns  
>>>> false, not all of the pipelines filters get run to clean up.   
>>>> Only the filters that ran will get called to clean up
>>>> Modified: plog/trunk/class/security/pipeline.class.php
>>>> ===================================================================
>>>> --- plog/trunk/class/security/pipeline.class.php	2006-04-15  
>>>> 10:44:44 UTC (rev 3250)
>>>> +++ plog/trunk/class/security/pipeline.class.php	2006-04-16  
>>>> 03:22:26 UTC (rev 3251)
>>>> @@ -134,6 +134,7 @@
>>>>            // Assume that this will be successful
>>>>            $this->_result = new PipelineResult( true );
>>>> +            $numPipelinesRun = 0;
>>>>            // if enabled, then check all the filters
>>>>            foreach( $_pLogPipelineRegisteredFilters as  
>>>> $filterClass ) {
>>>>                // create an instance of the filter
>>>> @@ -152,6 +153,7 @@
>>>>                    // break out of this loop
>>>>                    break;
>>>>                }
>>>> +                $numPipelinesRun++;
>>>>            }
>>>>
>>>>            // If one of the filters returns that this was not a  
>>>> valid result
>>>> @@ -162,6 +164,12 @@
>>>>                // want to do (i.e. report ip address to dns  
>>>> blacklist)
>>>>
>>>>                foreach( $_pLogPipelineRegisteredFilters as  
>>>> $filterClass ) {
>>>> +                    if( $numPipelinesRun-- <= 0 )
>>>> +                    {
>>>> +                        // We have run all of the pipelines in  
>>>> the initial loop
>>>> +                        break;
>>>> +                    }
>>>> +
>>>>                    // create an instance of the filter
>>>>                    $pipelineRequest = new PipelineRequest( $this- 
>>>> >_httpRequest,
>>>>                                                            $this- 
>>>> >_blogInfo,
>>>> _______________________________________________
>>>> pLog-svn mailing list
>>>> pLog-svn at devel.lifetype.net
>>>> http://devel.lifetype.net/mailman/listinfo/plog-svn
>>> **************************************
>>> Jon Daley
>>> http://jon.limedaley.com/
>>> Friendship is the inexpressible comfort of feeling safe with
>>> a person having neither to weigh thoughts nor measure words.
>>> -- George Eliot
>>> _______________________________________________
>>> pLog-svn mailing list
>>> pLog-svn at devel.lifetype.net
>>> http://devel.lifetype.net/mailman/listinfo/plog-svn
>>
>> --
>> Paul Westbrook
>> paul at westbrooks.org
>> <http://www.westbrooks.org>
>>
>>
>> _______________________________________________
>> pLog-svn mailing list
>> pLog-svn at devel.lifetype.net
>> http://devel.lifetype.net/mailman/listinfo/plog-svn
>
> **************************************
> Jon Daley
> http://jon.limedaley.com/
>
> Eagles may soar, but weasels don't get sucked into jet engines.
> _______________________________________________
> pLog-svn mailing list
> pLog-svn at devel.lifetype.net
> http://devel.lifetype.net/mailman/listinfo/plog-svn
>

--
Paul Westbrook
paul at westbrooks.org
<http://www.westbrooks.org>




More information about the pLog-svn mailing list