[quote=PadawanLotto]
My questions are, what do the prediction types mean as compared how to use the information? Also, what do the best pattern sizes pretain to? Is it the sum value ranges for each column? What is the minimum match ratios?
The predictor works ok but, it would be better if I knew what I was looking at. I used the predictor with the summary table set to history only. Back to my research.
[/quote]
the predictor divides all the past data into small sections of 5 to 30 draws (see the settings panel) and then compares these sections with the last xx sections of 5 to 30 draws in the table. the section size that produces the best results for the latest xx rows is chosen as the winner and is used for the actual prediction for the very last row in the table.
there are three ways for comparing data sections (patterns):
- each sequence/pattern of numbers e.g. 3,6,2,8,8,11,2 is normalized to 1,-1,1,0,1,-1 where 1 is for increase, -1 is for decrease and 0 is for same value. the past pattern matches the latest pattern if it has the same symbols at each position. the min match ratio specifies the minimum count of matching positions. e.g. if the min match ratio is 0.9 and the pattern size is 10 then the past pattern matches the latest pattern if at least 9 pattern positions have the same values.
- another prediction method calculates standard deviation of the differences of normalized patterns and the prediction is based on 100 matching patterns with the lowest standard deviation values
- the third prediction method calcuates standard deviation of value differences between the past pattern and the latest patter. the prediction is chosen from 100 matches with the lowest stddev values.
Trend Predictor
45 posts
• Page 2 of 3 • 1, 2, 3
by Trust » Tue Jul 04, 2006 10:06 pm
Hi Stan!
After loading all draws the predictor calculated on some of the
chosen columns-but the columns "Num Groups,Group Count,Movement
Sum,Hit 5-15 and 30,First Dig,Last Dig,Max in Col,Repeated 3,5 and 10,
Index Movement and Number Distance still gives the "Cannot parse data"
message.(With 450 draws in input file)
After loading all draws the predictor calculated on some of the
chosen columns-but the columns "Num Groups,Group Count,Movement
Sum,Hit 5-15 and 30,First Dig,Last Dig,Max in Col,Repeated 3,5 and 10,
Index Movement and Number Distance still gives the "Cannot parse data"
message.(With 450 draws in input file)
- Trust
- Posts: 144
- Joined: Tue Dec 14, 2004 5:02 am
by stan » Tue Jul 04, 2006 10:21 pm
[quote=Trust]
Hi Stan!
After loading all draws the predictor calculated on some of the
chosen columns-but the columns "Num Groups,Group Count,Movement
Sum,Hit 5-15 and 30,First Dig,Last Dig,Max in Col,Repeated 3,5 and 10,
Index Movement and Number Distance still gives the "Cannot parse data"
message.(With 450 draws in input file)
[/quote]
columns hit 5-15 have 'n/a' in the last 5-15 rows so the predictor cannot parse them.
the other columns have format e.g. "num1 - num2 - num3" or similar. the predictor cannot parse these either - it's not a single number.
you'll have to do some data reformating in e.g. ms excel if you want predictions for these statistics.
Hi Stan!
After loading all draws the predictor calculated on some of the
chosen columns-but the columns "Num Groups,Group Count,Movement
Sum,Hit 5-15 and 30,First Dig,Last Dig,Max in Col,Repeated 3,5 and 10,
Index Movement and Number Distance still gives the "Cannot parse data"
message.(With 450 draws in input file)
[/quote]
columns hit 5-15 have 'n/a' in the last 5-15 rows so the predictor cannot parse them.
the other columns have format e.g. "num1 - num2 - num3" or similar. the predictor cannot parse these either - it's not a single number.
you'll have to do some data reformating in e.g. ms excel if you want predictions for these statistics.
Expert Lotto Team
-
stan - Site Admin
- Posts: 6338
- Joined: Thu Sep 23, 2004 1:01 pm
by Midas » Tue Jul 18, 2006 3:28 pm
The following trend predictor analysis looked at stock market time series data, namely the weekly closings of the Dow-Jones Industrial Average, between July 1971 to August 1974. Data was taken from the New York Stock Exchange quarterly published statistics by Standard and Poor Co, NY.
The dataset had the following descriptive statistics summary.
[m]Time Series Sample Size 162
Minimum 752.58
Maximum 1047.59
Range 294
Mean 907.48475
Median 908.645
Variance 3672.350
Std Deviation 60.99992
Coefficient of Variation 0.06678
Std Error 4.761188
Skewness -0.033224
Kurtosis -0.4966
Hurst Exponent 0.58445[/m]T :finger:
The last twenty end values served as the validation set and the best performing trend algorithm selected namely, Pattern Matching Ratio (0.6, 0.7, 0.8, and 0.9) and Std Value, Std Trend with pattern sizes (2-6).
For pattern matching algorithm ratio sizes >6, the performance was not as good for this time series hence ignored. It appears the small pattern sizes (2-6) derived corresponds well with a low order moving average process where the latest values are of most significance.
The results correct for the trend predictors were Std Trend(15/20), Std Value (11/20), Pattern Match 0.6(10/20),0.7(9/20), 0.8(9/20), 0.9(9/20). Interestingly on two data points the standard deviation algorithms were correctly sensitive to “No Change†or “Same†criterion which suggested these algorithms had some inherent smoothing capability.
I notice the algorithms predict trend direction (Increase/Decrease/Same), however the magnitude of the prediction given the direction is not currently provided?. :rolleyes:
The dataset had the following descriptive statistics summary.
[m]Time Series Sample Size 162
Minimum 752.58
Maximum 1047.59
Range 294
Mean 907.48475
Median 908.645
Variance 3672.350
Std Deviation 60.99992
Coefficient of Variation 0.06678
Std Error 4.761188
Skewness -0.033224
Kurtosis -0.4966
Hurst Exponent 0.58445[/m]T :finger:
The last twenty end values served as the validation set and the best performing trend algorithm selected namely, Pattern Matching Ratio (0.6, 0.7, 0.8, and 0.9) and Std Value, Std Trend with pattern sizes (2-6).
For pattern matching algorithm ratio sizes >6, the performance was not as good for this time series hence ignored. It appears the small pattern sizes (2-6) derived corresponds well with a low order moving average process where the latest values are of most significance.
The results correct for the trend predictors were Std Trend(15/20), Std Value (11/20), Pattern Match 0.6(10/20),0.7(9/20), 0.8(9/20), 0.9(9/20). Interestingly on two data points the standard deviation algorithms were correctly sensitive to “No Change†or “Same†criterion which suggested these algorithms had some inherent smoothing capability.
I notice the algorithms predict trend direction (Increase/Decrease/Same), however the magnitude of the prediction given the direction is not currently provided?. :rolleyes:
- Midas
- Posts: 144
- Joined: Wed Apr 19, 2006 7:20 pm
by stan » Wed Jul 19, 2006 7:20 pm
[quote=George:1153229336]
I notice the algorithms predict trend direction (Increase/Decrease/Same), however the magnitude of the prediction given the direction is not currently provided?. :rolleyes:
[/quote]
it's possible to predict the actual value (in fact that was my first algorithm). but then it's much more difficult to validate the results and choose the best pattern size.
for example if the actual value is e.g. 100 and the predicted value is 110 - is the prediction valid or not?
I notice the algorithms predict trend direction (Increase/Decrease/Same), however the magnitude of the prediction given the direction is not currently provided?. :rolleyes:
[/quote]
it's possible to predict the actual value (in fact that was my first algorithm). but then it's much more difficult to validate the results and choose the best pattern size.
for example if the actual value is e.g. 100 and the predicted value is 110 - is the prediction valid or not?
Expert Lotto Team
-
stan - Site Admin
- Posts: 6338
- Joined: Thu Sep 23, 2004 1:01 pm
by Maryland » Wed Jul 19, 2006 9:48 pm
It would make more sense to me if I had a value. Guess I'm not to intelligent but right now the Trend Predictor is totally over my head. A value in the Trend Predictor would be something like a median in this little head of mine :vogel:. Is there anyone who really understands the Trend Predictor? I think Stan was asking for input when he released the plugin.
- Maryland
- Posts: 504
- Joined: Sun Nov 07, 2004 4:18 pm
by stan » Wed Jul 19, 2006 9:58 pm
[quote=Maryland]
It would make more sense to me if I had a value. Guess I'm not to intelligent but right now the Trend Predictor is totally over my head. A value in the Trend Predictor would be something like a median in this little head of mine :vogel:. Is there anyone who really understands the Trend Predictor? I think Stan was asking for input when he released the plugin.
[/quote]
i posted some instructions how to interpret predictor results - just ask if you need more details.
imagine you're looking at a chart - the chart points are the numbers in the predictor table (one column is one chart, for example winning numbers stats). the prediction result is an indication whether the chart line will rise/fall in the next draw (or when the next actual value is available)
It would make more sense to me if I had a value. Guess I'm not to intelligent but right now the Trend Predictor is totally over my head. A value in the Trend Predictor would be something like a median in this little head of mine :vogel:. Is there anyone who really understands the Trend Predictor? I think Stan was asking for input when he released the plugin.
[/quote]
i posted some instructions how to interpret predictor results - just ask if you need more details.
imagine you're looking at a chart - the chart points are the numbers in the predictor table (one column is one chart, for example winning numbers stats). the prediction result is an indication whether the chart line will rise/fall in the next draw (or when the next actual value is available)
Expert Lotto Team
-
stan - Site Admin
- Posts: 6338
- Joined: Thu Sep 23, 2004 1:01 pm
by Midas » Thu Jul 20, 2006 5:35 pm
Time series forecasting is a challenge in many fields. In finance, one forecasts stock exchange or stock market indices; data processing specialists forecast the flow of information on their networks; producers of electricity forecast the electric load and EL lotto specialists WN History or other useful statistical metrics. The common point to their problems is the following: how can one analyze and use the past to predict the future? In general, these methods try to build a model of the process. The model is then used on the last values of the series to predict future values.
Our challenge in time series prediction is the short-term prediction of these lotto numbers. Many methods designed for time series forecasting perform well (depending on the complexity of the problem) on a rather short-term horizon but are rather poor on a longer-term one. This is due to the fact that these methods are usually designed to optimize the performance at short term, their use at longer term being not optimized. Furthermore, they generally carry out the prediction of a single value while the real problem sometimes requires predicting a vector of future values in one step. I believe the pattern matching and std deviation algorithms are performing acceptably for the generic trend direction hence an averaging process on the overall trend but what prediction value is acceptable for an algorithm as Stan points out?.
I suggest to get a forecast value, the trend program(s) could iteratively select the best model by scoring of the pattern match versus its min prediction fitness error to get the best performing trend algorithms. I'm not sure how this would change the program, but the emphasis would be towards simultaneous prediction of magnitude value and direction.
Some useful fitness metrics are , Absolute Hits, Relative Hits, Rsquared,Mean Squared Error, Root Mean Squared Error, Mean Absolute Error, Relative Squared Error, Root Relative Squared Error, Relative Absolute Error, and Relative Error with Selection Range.
For consideration, please find attached the summary file for the above.
Our challenge in time series prediction is the short-term prediction of these lotto numbers. Many methods designed for time series forecasting perform well (depending on the complexity of the problem) on a rather short-term horizon but are rather poor on a longer-term one. This is due to the fact that these methods are usually designed to optimize the performance at short term, their use at longer term being not optimized. Furthermore, they generally carry out the prediction of a single value while the real problem sometimes requires predicting a vector of future values in one step. I believe the pattern matching and std deviation algorithms are performing acceptably for the generic trend direction hence an averaging process on the overall trend but what prediction value is acceptable for an algorithm as Stan points out?.
I suggest to get a forecast value, the trend program(s) could iteratively select the best model by scoring of the pattern match versus its min prediction fitness error to get the best performing trend algorithms. I'm not sure how this would change the program, but the emphasis would be towards simultaneous prediction of magnitude value and direction.
Some useful fitness metrics are , Absolute Hits, Relative Hits, Rsquared,Mean Squared Error, Root Mean Squared Error, Mean Absolute Error, Relative Squared Error, Root Relative Squared Error, Relative Absolute Error, and Relative Error with Selection Range.
For consideration, please find attached the summary file for the above.
- Midas
- Posts: 144
- Joined: Wed Apr 19, 2006 7:20 pm
trend
by jimbo » Fri Jul 21, 2006 3:27 pm
just my mind, and marylands.
i see what its saying, but it seems to what to make me go look at other charts. and study those charts for numbers. and it does confuse me at time. working and all, could the trend predictor just pick a group of numbers that fall into that area, something like other charts where i can click tab and sort by occ, if pattern size is 10, give the 10 numbers that did it, anything under 50% dont play, numbers that were used to give the % of increase or decrease, any numbers used to get the end result of data,
and then i can plug them into gererator and go for potluck.
brainstorming ideas
jimbo
i see what its saying, but it seems to what to make me go look at other charts. and study those charts for numbers. and it does confuse me at time. working and all, could the trend predictor just pick a group of numbers that fall into that area, something like other charts where i can click tab and sort by occ, if pattern size is 10, give the 10 numbers that did it, anything under 50% dont play, numbers that were used to give the % of increase or decrease, any numbers used to get the end result of data,
and then i can plug them into gererator and go for potluck.
brainstorming ideas
jimbo
- jimbo
- Posts: 211
- Joined: Wed Mar 29, 2006 1:57 pm
by Midas » Fri Jul 21, 2006 4:29 pm
Further thoughts on the "Scoring" or "Optimum Model Selection" method , the following describes a widely used calculation.
The Akaike information criterion (AIC) (pronounced ah-kah-ee-keh), developed by Professor Hirotsugu Akaike (èµ¤æ± å¼˜æ¬¡) (1927-) in 1971 and proposed in 1974, is a statistical model fit measure. It quantifies the relative goodness-of-fit of various previously derived statistical models, given a sample of data. It uses a rigorous framework of information analysis based on the concept of entropy(ε). The driving idea behind the AIC is to examine the complexity of the model together with goodness of its fit to the sample data, and to produce a measure which balances between the two.Its formula is AIC = 2k − 2ln(L), where k is the number of parameters, and L is the likelihood function. (Note this is summarised in alternate form below)
Usually however, normally distributed errors are assumed and AIC is computed as *** AIC = 2k + nln(RSS / n)***, where n is the number of observations and RSS is the residual sum of squares. See fitness functions previously decribed for RSS.
A model with many parameters will provide a very good fit to the data, but will have few degrees of freedom and be of limited utility. This balanced approach discourages overfitting. The preferred model is that with the lowest AIC value. The AIC methodology attempts to find the minimal model that correctly explains the data, which can be contrasted with more traditional approaches to modeling, such as starting from a null hypothesis.
Other model validation equations exist, such as known as the Schwarz criterion (also Schwarz information criterion (SIC) or Bayesian information criterion (BIC) or Schwarz-Bayesian information criterion) is an information criterion. The generic formula is: SIC =-2loglik=log(n)k
AIC in particular is the best, widely used in time series analysis. The maths may look awesum, but in reality all the numbers will be doing is pointing to the best trend predictor model.
2scoops, the file was sent to Stan, if you really want a copy please let me have your mailbox. Better still just google the items, they are all widely described on the web.
Jimbo/Mary, the idea for selecting pool of numbers occuring in overlapping tuples (x6,x5,x4,x3) through trend predictor and WN History (2) is very interesting, it could tie in nicely with Joseph and Stan's original methodology.
The Akaike information criterion (AIC) (pronounced ah-kah-ee-keh), developed by Professor Hirotsugu Akaike (èµ¤æ± å¼˜æ¬¡) (1927-) in 1971 and proposed in 1974, is a statistical model fit measure. It quantifies the relative goodness-of-fit of various previously derived statistical models, given a sample of data. It uses a rigorous framework of information analysis based on the concept of entropy(ε). The driving idea behind the AIC is to examine the complexity of the model together with goodness of its fit to the sample data, and to produce a measure which balances between the two.Its formula is AIC = 2k − 2ln(L), where k is the number of parameters, and L is the likelihood function. (Note this is summarised in alternate form below)
Usually however, normally distributed errors are assumed and AIC is computed as *** AIC = 2k + nln(RSS / n)***, where n is the number of observations and RSS is the residual sum of squares. See fitness functions previously decribed for RSS.
A model with many parameters will provide a very good fit to the data, but will have few degrees of freedom and be of limited utility. This balanced approach discourages overfitting. The preferred model is that with the lowest AIC value. The AIC methodology attempts to find the minimal model that correctly explains the data, which can be contrasted with more traditional approaches to modeling, such as starting from a null hypothesis.
Other model validation equations exist, such as known as the Schwarz criterion (also Schwarz information criterion (SIC) or Bayesian information criterion (BIC) or Schwarz-Bayesian information criterion) is an information criterion. The generic formula is: SIC =-2loglik=log(n)k
AIC in particular is the best, widely used in time series analysis. The maths may look awesum, but in reality all the numbers will be doing is pointing to the best trend predictor model.
2scoops, the file was sent to Stan, if you really want a copy please let me have your mailbox. Better still just google the items, they are all widely described on the web.
Jimbo/Mary, the idea for selecting pool of numbers occuring in overlapping tuples (x6,x5,x4,x3) through trend predictor and WN History (2) is very interesting, it could tie in nicely with Joseph and Stan's original methodology.
- Midas
- Posts: 144
- Joined: Wed Apr 19, 2006 7:20 pm
by stan » Sat Jul 22, 2006 12:28 pm
[quote=jimbo]
just my mind, and marylands.
i see what its saying, but it seems to what to make me go look at other charts. and study those charts for numbers. and it does confuse me at time. working and all, could the trend predictor just pick a group of numbers that fall into that area, something like other charts where i can click tab and sort by occ, if pattern size is 10, give the 10 numbers that did it, anything under 50% dont play, numbers that were used to give the % of increase or decrease, any numbers used to get the end result of data,
and then i can plug them into gererator and go for potluck.
brainstorming ideas
[/quote]
pattern size "10" doesn't mean there are ten numbers to play. it means there are several sequences of ten values that match the latest ten values. the predictor then counts in how many cases the past sequences were followed by an increased value and in how many cases the past sequences were followed by a decreased value. the bigger count is then chosen as the 'prediction' (increase/decrease)
imagine the following series of values:
5,8,9,3,6,9,7,8,9,1,5,2, 4,6,8
if the pattern size is 3 then the past sequences will be:
5,8,9 -> decrease
8,9,3 -> doesn't match
9,3,6 -> doesn't match
3,6,9 -> decrease
6,9,7 -> increase
9,7,8 -> doesn't match
7,8,9 -> decrease
8,9,1 -> doesn't match
9,1,5 -> doesn't match
1,5,2 -> doesn't match
5,2,4 -> doesn't match
2,4,6 -> increase
prediction will be 'decrease' because there are 3 past sequences that match the latest sequence (4,6,8) and those sequence were followed by a decreased value. only two past sequences were followed by increased value.
(this is just a simplified example, in fact the predictor 'trains' itself on the last xx values and chooses the pattern size with the best results. however you can force it to use only one specified pattern size)
just my mind, and marylands.
i see what its saying, but it seems to what to make me go look at other charts. and study those charts for numbers. and it does confuse me at time. working and all, could the trend predictor just pick a group of numbers that fall into that area, something like other charts where i can click tab and sort by occ, if pattern size is 10, give the 10 numbers that did it, anything under 50% dont play, numbers that were used to give the % of increase or decrease, any numbers used to get the end result of data,
and then i can plug them into gererator and go for potluck.
brainstorming ideas
[/quote]
pattern size "10" doesn't mean there are ten numbers to play. it means there are several sequences of ten values that match the latest ten values. the predictor then counts in how many cases the past sequences were followed by an increased value and in how many cases the past sequences were followed by a decreased value. the bigger count is then chosen as the 'prediction' (increase/decrease)
imagine the following series of values:
5,8,9,3,6,9,7,8,9,1,5,2, 4,6,8
if the pattern size is 3 then the past sequences will be:
5,8,9 -> decrease
8,9,3 -> doesn't match
9,3,6 -> doesn't match
3,6,9 -> decrease
6,9,7 -> increase
9,7,8 -> doesn't match
7,8,9 -> decrease
8,9,1 -> doesn't match
9,1,5 -> doesn't match
1,5,2 -> doesn't match
5,2,4 -> doesn't match
2,4,6 -> increase
prediction will be 'decrease' because there are 3 past sequences that match the latest sequence (4,6,8) and those sequence were followed by a decreased value. only two past sequences were followed by increased value.
(this is just a simplified example, in fact the predictor 'trains' itself on the last xx values and chooses the pattern size with the best results. however you can force it to use only one specified pattern size)
Expert Lotto Team
-
stan - Site Admin
- Posts: 6338
- Joined: Thu Sep 23, 2004 1:01 pm
by stan » Thu Nov 02, 2006 11:32 am
[quote=Clem9403]
I know this is an old reply, but has anyone been able to get the Trend Predictor to work? It sounds like a useful tool. I've not been able to download it just yet as I'm at work at this moment.
[/quote]
try the 'value predictor' instead. it can predict trends as well as values and its results are better. you'll find it at the beta versions download page at our web site.
I know this is an old reply, but has anyone been able to get the Trend Predictor to work? It sounds like a useful tool. I've not been able to download it just yet as I'm at work at this moment.
[/quote]
try the 'value predictor' instead. it can predict trends as well as values and its results are better. you'll find it at the beta versions download page at our web site.
Expert Lotto Team
-
stan - Site Admin
- Posts: 6338
- Joined: Thu Sep 23, 2004 1:01 pm
45 posts
• Page 2 of 3 • 1, 2, 3
Who is online
Users browsing this forum: No registered users and 5 guests