Probabilistic Prediction of Football Matches - A Discussion

[Rushian]

Hello all,

The following may sound (or should I say “read”) a bit academic but bear with me. Scanning through the number of different threads / systems, I realized that system picks are usually derived through one of the following three (at least) ways:

• Intuition (or gut feeling or experience): you know the ones, when you spot a price and almost immediately feel that this price represents value. Subconsciously, your brain has assigned a probability to that event happening (say Liverpool beating Everton on Sunday); it has transformed that probability to odds and compared it to the prices on offer before deciding whether it’s worth a bet.
• A set of pre-defined rules: systems which are based on rules such as:
  • If a home team that has won more than 3 out of their last 5 home games AND
  • An away team that has lost more than 3 out of their last 5 away games AND
  • Odds for a home win more than 1.50
  • Then bet on the home win
• A statistical model: systems which try to model things like the number of goals scored by the home/away team, or the result itself, or even the probability of over/under in a match using a number of predictors (whatever these may be). This includes anything from simple regression models to neural networks.
• Maybe there are some more so feel free to add …

The thought that occurred to me relates mainly to people developing systems using statistical models or rules. It may also be of interest to people who decide on bets based on their intuition but only if they feel capable of actually quantifying that intuition in terms of probability. So here goes:

What if all those people who have a way of estimating the probability of a Home Win / Draw / Away Win in a set of given matches, come forth and submit these probabilities in a thread so that we compare them across the different systems which are out there. This would have a number of advantages both for the modelers themselves but also for the rest of the punters because:

• Comparing the probabilities may help in finding improvements in the systems (e.g. someone may be consistently underestimating the probability of a draw). Of course one could argue that rather than comparing amongst ourselves, one should be comparing against the bookies prices (which is the ultimate test) but what if the majority of models underestimate (say) the draw, then maybe a complete change of viewpoint is needed, or an inclusion of an additional variable etc.
• The vast amount of information on matches from major leagues (and that would depend on what each of us is doing) which is spread around the different threads would be nicely summarized in a single thread, for the benefit of everyone visiting these threads.
• A possibly improved system could be developed based on an average of all the probabilities could be developed and tested in a thread of its own.

Moreover a couple of general points: probabilities hide enough of the background of any system which calculates them, in case somebody does not wish to share the specifics of his/her system. So if someone wanted to share the variables used in his/her model, that would be great, but not a prerequisite. Finally, this also applies to the people who use rules (see above) in selecting bets since these rules are closely related to what is known as a classification tree. Following a series of decisions (e.g has the home team won more than 3 of their last 5 home games: yes/no etc…), the number of games under consideration is reduced, until you reach down at a set of games which fulfill all the criteria. The probability estimate of a home win / draw / away win, is simply the number of home wins / draws / away wins out of those filtered matches.

So an example match would be:

Date	Home	Away	System / Model	Home Win (%)	Draw (%)	Away Win (%)
30/3/08	Liverpool	Everton	A Model	52.0%	25.0%	23.0%
			Another Model	48.0%	38.0%	14.0%
			A Third Model	36.0%	29.0%	35.0%

Anyway, I would be delighted if you could share your thoughts on this. Apologies if this is not the right forum to post it on, mods feel free to move/delete it.

Rushian

[lunatism]

Hi Rushian!
You've put to the forum a suggestion on many a system users mind I guess - a way to combine the cream of all systems out there into one super-system that can hopefully do the business better than any one single system could ever hope to. And it is a GREAT idea, and I would of course be willing to offer my help on this if at all possible.

I've always thought that the best way to fully predict the outcome of a game you need to take into account as many quantifiable factors as possible, and so that is what I do, there are soooooo many different things that happen to the raw data I import before the system shoots of the LSS selections in the form of "information". There are a lot of guys on the forum who use solely intuition, while for others, it's a mix of such and stats. I'm of the belief that it's possible to produce success without human intervention once you have spotted the patterns and taken the relevant steps to "codifying" it (for me...meant a few sleepless weeks making the most automated of spreadsheets to do all the donkey work). With regards to this, I wonder, do you already have any preconceived opinions about the "measure" of intution against, "rules" or statistical models?

I ask this because I guess it will make a difference to how you view information generated by one system compared to another.

As I explained at the start of my thread, I used a number of systems to generate ratings/percentages/scores - all successful at looking at certain aspects of a team. All show similar overall strike rates. But you are only interested in the percentages, right?

And which leagues?!! Big question here - i look at almost every world league, and i have the past percentages for all of them! So if you said post everything, that would be one HUGE list daily, so i guess that wouldn't be feasible really.

It will no doubt be a challenge to accomodate all systems (if you can get a lot of people interested in this idea), but it should be "fun".

Are there any patterns in particular that you are looking for?

On your last point, i think i speak for many who generate percentages by saying that it's often the way to keep it hidden. ESPECIALLY if you would be generating such stats for laying - as a few people know how to do it and you find you can't get your bets matched at reasonable prices.

Right - onto the Liverpool Everton game...here's what my system says (the same processing that makes the LSS stuff):

Date	Home	Away	System / Model	Home Win (%)	Draw (%)	Away Win (%)
30/3/08	Liverpool	Everton	LSS	30%	46%	24%

right, there's my input - let me know what you want from me!

oh ye - just wonder - have you developed a system that you will be posting details from too? Or are you just the "compiler"?

[Grex]

Hi Rushian,

Nice idea.... I think perhaps this was the general theory behind the sticky thread
Creation of a "systems calculator" tool for Punters Lounge
which never materialised (just noticed that you contributed to that thread, so you know all about it)

Anyway, I would be more than willing to contribute my system´s probability predictions (for what it´s worth ). But then I suppose it would be necessary to introduce some weighting system for each system, presumably reflecting its performance (i.e. yield).

The problem is that it may simply develop into a league table of system authors.... but perhaps that would not be so bad.... it would give the authors that extra incentive to fine-tune and improve their systems.

As a matter of interest, for the Liverpool-Everton derby my predicton is:
Home 53% Draw 25% Away 22%.... which makes the Home odds 98% value, the Away odds 112% value, and the Draw odds 90% value (Bet365 odds).... so the "value bettors" should go for Everton

[Hooloovoo]

I find the idea very interesting.

I am not an "intuition" punter, but relies on stats and pre-defined rules that I have set up and constantly evaluates and fine-tune.

I'm on this boat as well for what it's worth

Grex is probably right that this could develop into a some sort of a "systems league", and I agree that this could be a good thing.

Some comparator is needed, yield is fine with me.
Then we could have a chart with a bunch of specified matches, and everyone that has a system posts their system's probability percentages for each match.
Maybe three different tables are needed as system A may be best for picking home wins and system B shows to be best for picking draws, and so forth.
In that way the different strengths of different systems are not "hidden" in some general sum-up. Oh i don't know really, just rambling now

About what leagues to use i suggest we keep it simple, maybe only the Prem? Of course different systems performs differently between different leagues,
but maybe it would be too complex to start with if we were to include "the world".

I reckon it will be a challenge to come up with ideas for how to identify the different strengths of different systems and how
to combine them into this "super-system". We need some good ideas here. And i need to stop writing the word "different" all the time

[Grex]

Well, there seems to be some interest in this idea (well done, Rushian). As I said, the idea of having an automated "sniffy" was discussed in the sticky thread in "Systems & Strategy".... the technology was way above my head, and it would seem that it will take some time to get it up-and-running.

In the meantime, is it not possible to have a spreadsheet attachment to this thread where system authors could insert their probability predictions? It wouldn´t be too difficult to produce something in Excel that would automatically calculate an overall prediction for each match, possibly including a weighting system based on each system´s yield-to-date (I think the yield would have to be inserted manually - we would have to trust the author not to over-inflate his/her yield!)

As HooLooVoo (where did you get that name!?) suggests, I think we should initially concentrate on 1 or 2 leagues and see how it goes.

I am not a Techno, so I am probably talking a load of cr*p. Any suggestions?

[Hooloovoo]

Originally Posted by Grex

As HooLooVoo (where did you get that name!?) suggests ...

http://en.wikipedia.org/wiki/Races_and_species_in_The_Hitchhiker's_Guide_to_the _Galaxy#Hooloovoo

[Grex]

Thanks for "educating" me..... I was never into the Hitch-hikers Guide !!

"A Hooloovoo is a superintelligent shade of the colour blue. Little is known of them, except that one participated in the construction of the starshipHeart of Gold. At the launching ceremony one was temporarily refracted into a free-standing prism."

I suppose that also explains the "Prism Punter".

[robby]

lol, I add 42 cents to every bet I make, regardless of event just because of Hitchhikers! Yes, I'm a geek

[Hooloovoo]

Originally Posted by robby

lol, I add 42 cents to every bet I make, regardless of event just because of Hitchhikers! Yes, I'm a geek

The number 42 is everywhere, especially in movies. Sometimes you think every movie script writer has read "Hitchhikers..."

• The apartment Fox Mulder lives in in "X-files" has number 42
• The building Tom Cruise enters in MI-2 has 42 floors
• In Alien 4, the spaceship has 42 soldiers
• ...the list could go on

*BANG*

(shot by the Kill All Geeks Assassination Squad)

[Grex]

I have prepared a very rudimentary prototype Excel file for discussion purposes.... question is, how do you attach it to a post?? HELP please.

According to the Posting Rules at the bottom of the page, you MAY NOT post attachments.... aaaarrrrgggghhhh

[lunatism]

right - seems to be forming into some kind of systems competition then?

When we do decide on the leagues that would be best suited to this, will we post for every single fixture in that league? I think it would be fair to suggest we do - that way we don't leave out the shockingly awful completely wrong predictions...as it's those that will provide the greatest insight into all this i believe!

i also think that however many leagues we have, we MUST choose one that is notoriously hard to predict, alongside one that is historically "easy" to predict - that way we are judging a systems ability to see what "we can all see without help of a fancy pancy spreadsheet", and also on spotting what we do not!

Grex - i think you need some kind of permission to post...if you want, email me the file and i'll post it

[Grex]

Originally Posted by lunatism

Grex - i think you need some kind of permission to post...if you want, email me the file and i'll post it

I've sent it to your Hotmail address

[lunatism]

I can't remember how it is that you get the "post attachments" status, but there's something pretty simple that you need to do

had a look at the spreadsheet - that's all we need right at this minute, something easy to use that we can just stick stuff in, although in "the future" we could have a separate sheet for each system and then one combo sheet that has everything for side-by-side comparison (one day i'll make it if nobody else does)

I guess that for now we'll just determine how many systems will be looked at, the leagues we will look at, and what exactly Rushian's master plan is!

...another reason why i think we should post all predictions (from whatever leagues we chose) and not just a few select ones, is that we all know which of our picks are good/not so good...eg - i put all my good ones for LSS, etc. and i wouldn't really be able to input at all because of the sparsity of selections otherwise! But by saying all in any particular league, i can just copy these across from my world predictions, and not have to worry about filtering out the "unpredictables"...if that makes sense

[Hooloovoo]

My suggestion is that we start of with the systems from the four people currently in the spreadsheet. Anyone else interested can certainly join later.

We should definitely post all predictions for the selected leagues. What we want is to establish to what degree the probability estimates are correct. At this stage it's not about finding value bets.

[Rushian]

Apologies for the late reply! I'm glad that this has found interest among fellow punters although it's started going off the direction of a systems league (also helpful, I guess)!

A few points because I have to run, but promise to return with further thoughts tonight:

• I was thinking of focusing on a single league to start with, not necessarily a difficult one to predict, rather than a popular one so that more people take part. e.g the Premiership. But you would have to submit probabilities for all fixtures rather than the ones that are highlighted as picks by your system.
• My system is still under work, (and returns negative yield currently!) but I would be willing to share the probability estimates to see how I can improve.
• People filling an Excel sheet sounds like a good and practical idea.
• As for the "competition" thing, although I don't think that it needs to be this way (and it's not because I'll probably be getting the wooden spoon ) there are other measures that can be used which measure how close the probabilities lie to the actual result. I'll have to look more into it, but I am sure some people are familiar with information theory, entropy and the like...
• Forgot to add... the Liverpool - Everton match was just an example... Oh, and thanks for the kind sir which produced the spreadsheet indicating that my yield is +15% ... I wish!

Got to run now. I'll add to these initial thoughts later tonight. Thanks for the ideas!

[lunatism]

Right then, if it's just one single league, then Premiership is most definitely the most accessible for the largest number of people I guess, so I'd vote for that one then.

Negative yields can be turn around - so i'm sure it won't be long before you tinker your system just right until it's into profit!

i know the 'pool game was an example, but couldn't resist - it's a tough derby - perfect acid test to start off with!

Right now i've got Grex's spreadsheet open and i'm fiddling around with it to make it easier to input predictions. What do you think would be better, having the fixtures already there, so all we'd need to do was enter the percentages, or leaving it so that we each would have to put in the fixture and the percentage?

[Grex]

Originally Posted by lunatism

What do you think would be better, having the fixtures already there, so all we'd need to do was enter the percentages, or leaving it so that we each would have to put in the fixture and the percentage?

I think it would be better to have the fixtures already inserted.... if you don´t have time, Loon, I´ll do it this evening.

[lunatism]

Originally Posted by Grex

I think it would be better to have the fixtures already inserted.... if you don´t have time, Loon, I´ll do it this evening.

not a problem at all - a simple copy and paste from one spreadsheet to another

obviously i will only put in the coming weeks fixtures, as postponements and corrections may come about, plus, it's not like we'd have accurate predictions for that far into the future anyway, right?!

[lunatism]

how many of you guys use office 2007? because there is a function i "need" to use that isn't in office 2003

....no worries - found a way around it

[lunatism]

right - almost done - just doing the weighted average thing. So are we all submitting our our yields in for the Premiership?