Very slow in using multiple Segment Combinations Filter

[Falcon]

Stan,

Been using this filter in this manner

411,501,402,321,600,,,,,, and it works perfectly. However if I use it in this manner ,,,,,411,501,402,321,600 (the segments examples only) it is virtually unusable because of slowness. My bulk segments CSV file is quite large ie 2000 lines so there is a lot of work to do. But why OK one way and not the other?

eg in the first mode took about 4 hours to process 1.8m tickets, in the second mode it takes about 4 hours to process less 100,000 tickets.

In respect of memory allocation I have tried the default, 256, 512, and 1024 allocations and it makes not a scrap of difference.

Thus is the filter designed to only work with the segments in the first position settings, or is it universal as to how the segments are placed in the columns?

[PadawanLotto]

Hey Falcon, I'm not real sure but my first guess on your second example ,,,,,411,501,402,321,600 is that the filter is checking the blank spaces throughout all of the lines and resetting for each blank space. Where as in the first example 411,501,402,321,600,,,,,, the filter determins that there is no more data after it reaches the first or second blank space. Like I say, that is just a guess on what is happening here, I have never ran the filter with blanks at the beginning, always at the end.

[Falcon]

Padawan,

Thanks. If it is resetting the blank spaces perhaps a wildcard or global segment of some sort might be needed.

I have read the main multiple segment thread a few times but I cannot properly understand the use wildcards if in fact they actually exist as ?-?-? but perhaps in this case it might need just a sort of step on to next column no filtering action perhaps rendered under something like 9-9-9 or 0-0-0.

r

falcon

[PadawanLotto]

Why would you want leading spaces such as ,,,,,411,501,402,321,600?

[stan]

there might be some room for performance optimization in that filter so i'll look at it

[Falcon]

Padawan,

Actually using a 4 segment wide array in my bulk filtering not 5 as in my earlier examples here and with only 4 segments not quite enough filtering is done so I thought it would be a good idea to sort of come at it from the other end with another set of 4 columns to see what I end up with. Basically experimenting with some techniques.

r

falcon

[Bobijohn]

Stan. Although this is a different issue it may offer a clue to the above. This was going to be the subject of a separate post another time. Please reference WNH/ History Differences. Running the segment simulator is relatively quick with one exception – that is if there is a 0 in the first segment e.g. 0-5-0. 0-4-1 etc. for a pick 5 or, 0-4-0, 0-2-2 etc for pick 4 game. In this case, the simulation will take several times longer than if a numeral were present and you can see the counter slowly going up to the 100,000 default setting.

Bobijohn

[stan]

[quote=Bobijohn]
Stan. Although this is a different issue it may offer a clue to the above. This was going to be the subject of a separate post another time. Please reference WNH/ History Differences. Running the segment simulator is relatively quick with one exception – that is if there is a 0 in the first segment e.g. 0-5-0. 0-4-1 etc. for a pick 5 or, 0-4-0, 0-2-2 etc for pick 4 game. In this case, the simulation will take several times longer than if a numeral were present and you can see the counter slowly going up to the 100,000 default setting.

Bobijohn
[/quote]

thanks for the tip, i'll check what's going on there

[stan]

[quote=Bobijohn:1211284266]
Stan. Although this is a different issue it may offer a clue to the above. This was going to be the subject of a separate post another time. Please reference WNH/ History Differences. Running the segment simulator is relatively quick with one exception – that is if there is a 0 in the first segment e.g. 0-5-0. 0-4-1 etc. for a pick 5 or, 0-4-0, 0-2-2 etc for pick 4 game. In this case, the simulation will take several times longer than if a numeral were present and you can see the counter slowly going up to the 100,000 default setting.

Bobijohn
[/quote]

the simulation is slower in this case because it first must find a random ticket that has zero numbers from the first segment. and since the first segment has usually most of the numbers it does take a few loops before random generator yields the required segment distribution.

[stan]

[quote=stan:1211275963]
there might be some room for performance optimization in that filter so i'll look at it
[/quote]

i checked the code and didn't find any redundant calls, it's pretty much optimized except for a few empty loops until a segment combination is found in the input line. this slowdown might show up a bit if you have tens of thousands of input lines...

[Bobijohn]

Stan. Thanks for checking into both of these issues.

Bibijohn