I am not a statistician and so have very little understanding of the statistical maths used by Professor John Hattie to produce his book Visible Learning. I do however understand that some of Hattie’s work has been called into question regarding his use of statistical concepts.
It has been pointed out that Hattie’s use of the Common Language Effect Size (CLE) is incorrectly used. Some of the sources below also question Hattie’s use of effect sizes.
I have included a comment found in one of the links below which I think sums up the issues found with Hattie’s work. It resinated with me as a non maths /science person.
“Sometimes scientific critique is like a cannonball shooting a hole in a sail. The ship can continue sailing but has some repairs to do. At other times the critique hits below the waterline and the ship sinks (like Titanic). How does Topphols critique hit Visible learning? Do the main conclusions sail on or do they sink?” Jan Pålsgård
Based on the evidence that I have read so far I believe “Hattie’s ship can continue sailing but has some repairs to do”. This criticism of Hattie’s work is not recent (2012/2013) so if his work didn’t hold up due to statistical errors we would probably have heard more about it. Just because their are some errors in a particular aspect of Hattie’s work does not mean the work is to be dismissed.
My motivation for posting this information is to make people aware that if we are going to use educational research, particularly to the level that Hattie’s work has been, we should know as much as possible about how that research came to be.
I have provided a variety of sources to read in relation to the issues found with Hattie’s work. Be warned most of this is not light reading and involves, predictably, a lot of mathematical language around statistics.