Monday, 31 May 2010

The End Of Science

This was the cover title for a series of articles in Wired in July 2008. They were welcoming us to the petabyte age - where we will routinely deal with datasets containing petabytes of data. Inside the titles are less provocative the first is - The End of Theory.

In this article Chris Anderson looks at how tools like Google and the Cloud are chaging the way we look at data. He raises the question of how we have to dea
l with such massive amounts of data.

It forces us to view data mathematically first and establish a context for it later
Peter Norvig has gone so far as to change George Box maxim "All models are wrong but some are useful" to "All models are wrong, and increasingly you can succeed without them."

... faced with massive data, this approach to science - hypoyhesize, model, test - is becoming obsolete.
There are a couple of problems with this idea:
  1. It becomes very difficult to distinguish science and pseudo-science. The Bible Code and other such books suddenly become more convincing.
  2. There is still a model, or rather an assumption and that is that homology between already seen examples will apply to new unseen examples.
What is actually happening is you no longer look for the universal laws but what they actually produce. This is actually how science has worked before. That is the process by which Newton produced his Laws of Gravitation, but the explanation in a more fundamental sense of what gravity means had to wait for Einstein.

Anderson goes too far when he talks about the limits of biology:

Now biology is heading in the same direction. The models we were taught in school about "dominant" and "recessive" genes steering a strictly Mendelian process have turned out to be an even greater simplification of reality than Newton's laws. The discovery of gene-protein interactions and other aspects of epigenetics has challenged the view of DNA as destiny and even introduced evidence that environment can influence inheritable traits, something once considered a genetic impossibility.
This is over-stating reality. Biology according to pure Darwinian evolution is likely to be wrong and there are some epigenetic factors that follow a Lamarckian process but not all and Darwin and Mendelian inheritance is still true for most genes. We only know that it is not completely true because we know the mechanism of epigenetics, which is precisely what Anderson had said we do not need to know.

There is now a better way. Petabytes allow us to say: "Correlation is enough." We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot.
Sorry but he should read any statistics textbook about data-mining that show you can find any possible correlation depending on the way you partition the data. The higher the dimensionality of the data (more variables) the more likely this is to happen. So this paragraph is nonsense.

Finally:

Correlation supercedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic, explanation at all.
There's no reason to cling to our old ways. It's time to ask: What can science learn from Google?
To understand the importance of mechanism you need to read studies like those of Richard Peto looking at heart attacks and the use of aspirin as a primary medical treatment when patients are hospitalised. Why did they think to use aspirin? Because its mechanism of action is to prevent clotting.

Saturday, 29 May 2010

Conspiracy Theories

From the assasination of Lincoln to 9-11 nothing seems to attract the public imagination more than a conspiracy theory. Now there is an entire conspiracy theory literature and books are written about the dangers of conspiracy theorists. We live in the "brave new world of big brother" (people should read the books before slinging the names around indiscriminantly including me). Governments do want to hide some of the things they do and how can we tell when we are being lied to.

This blog entry was in fact inspired by the book "You are being lied to". Some of the articles are by respected authors such as Noam Chomsky and Howard Bloom but many of the authors are unknown to me, so why should I believe anything they say?

How can we tell if there is any truth to the claims? Well we need to use logic, science and reason and we need to find hard evidence.

Take for example the article by Jim Marrs - What is Missing from this Picture. I read the article about how much evidence is missing from high profile cases and you have to wonder. How can the police and the government lose so much evidence? What do they have to hide? The problem is that this is building a whole story on one or two pieces of information. To use a phrase from Terry Pratchett you are living on narativium. You have a story and you make the facts fit the story you want to believe in. So when you read articles like this you have to be careful not to be caught up by the story.

One example is the campaign to find illegal behaviour in the Clinton-Gore adminsitration and to connect this with the death of Vince Foster and Ron Brown (and even TWA 800). Now we know that certain right-wing groups were trying their hardest to smear Clinton after they had got the Republicans to employ a special prosecutor in an operation called the Arkansas Project. In the end it was the people behind the smears and the prosecutor who ended up as the losers. It is very easy to find coincidence and circumstantial evidence but if this is all you have then you have to be very careful at what conclusions you make.

For a Holywood take on conspiracy theories you can enjoy:

  • The Net - A hacker's identity and life is destroyed as she uncovers a conspiracy.
  • Conspiracy Theory - Mel Gibson hams it up
  • Enemy of the State - The NSA bad guy's birthday is 9/11
  • JFK - but read the websites about Oliver Stone's playing with history.

Best of all and a must watch for anyone interested in conspiracies, is Arlington Road. This is absolutely brilliant, especially as it showed how the US was vulnerable to home grown terrorism before 9/11.

Friday, 28 May 2010

Chariots of the Gods and Pseudoscience

I have always been a science-fiction geek and so from this I was always interested in stories of alien encounters and secret government cover-ups such as project bluebook and Majestic-12.

One of the early books I read probably when I was about 12 or 13 was "The Gold of the Gods" by Erich von Daniken. In this book von Daniken describes being shown into a huge system of tunnels that extended out under the Pacific Ocean and that were filled with gold artefacts and a library that was written on a huge collection of metal discs. The books has photos of the entrance to the cave system and of some of the artefacts.

This is one of a series of books that develops his theme from "Chariots of the Gods" that the meso-american civilisations were exposed to Alien technology and that we have been visited by Aliens who we think of as Gods and that UFOs are their chariots.

So here is more evidence of these visitations because there is a level of technology that does not fit with what we know of meso-American history. The problem is that it does not exist. The poor archaeologist who showed him the site then had to explain to the authorities where all these artefacts had gone as they did not actually exist. He had made up the extent of the tunnel system and the library. This is an example of pseudo-science, the hijacking of scientific language and presentation to make something appear to be scientific while not actually following scientific method. You cannot always trust what you read or believe the photos that you see.

For a clear debunking of von Daniken's claims you should read Ronald Story's book The Space Gods Revealed. Sadly it looks like Story's book is out of print but von Daniken's frauds continue to sell well. As wikipedia would say knowledge is not democratic. Just because lots of people believe it, does not mean it is true.

Thursday, 27 May 2010

The Curious Case of "The Total Synthesis of Taxol - K.C. Nicolaou"

This is an interesting case of scientific competition which is discussed in hushed tones amongst synthetic organic chemists. K.C. Nicolaou is one of the leading exponents of total synthesis of natural products. One of the early natural products to be produced synthetically was taxol which is the naturally occuring toxin in Yew trees. Taxol stops cell division and so it is a potential anti-cancer drug.

Nicolaou had been competing to be the first to produce a total synthesis against Robert Holton's group from the State University of Florida. Holton submitted his two papers on the synthesis to the Journal of the American Chemical Society on Dec 21st 1993. Nicolaou has submitted a paper on synthesis of related Taxoids three weeks earlier to the same journal on the 30th of November (this would appear in the February 1994 issue of JACs as the Holton papers).

Meanwhile Nicolaou submits a paper on the Total Synthesis of Taxol to Nature on the 24th of January which was accepted on the 31st of January and published in the 17th February edition of Nature. The paper before Nicolaou's in the journal was submitted 27th September 1993 and accepted 21st December 1993. So more usually it took 3 months to review a paper and not one week and even after acceptance Nature could take another six to seven weeks to publish not the two and a half weeks seen for Nicolaou's paper. Nicolaou would then go on to elaborate the synthesis in four papers published in JACS in 1995. Nicolaou included the synthesis of Taxol in his book Classics in Total Synthesis.

Now we cannot know whether Nicolaou was a reviewer for the Holton JACS paper but he would have been a logical choice unless Holton expressed that he should not be allowed to review because of a conflict of interest. Even if he did not review the paper, he was probably aware that it had been submitted but doesn't the rapid submission of the paper to Nature and its fast-track review and publication deserve some further investigation?

All in all this case raises questions about the publishing ethics of the journal Nature, the confidentiality of submissions to JACS and the integrity of scientists. Nicolaou has since gone on to be a highly acclaimed scientist with many significant international awards. Meanwhile Robert Holton's biography is rather more modest.

When Peer Review Fails - Fabricated Results

The idea behind peer review is that your peers are also experts in the field and so they should be able to spot weaknesses in the science and prevent the publication of fabricated results.

Here are a few cases where this has not worked.
This is likely to be the tip of the iceberg, these are serial offenders who had fabricated most of the data for their entire careers. It is much harder to find the one off fabrication or just the science that turns out to be wrong by error and irreproducible. Peers should be able to reproduce the experiment, that is what makes it science but this is not always the case as was seen with Cold Fusion.

Here is an article about retraction rates which shows Science and then Nature are at the top of the list. It also suggests a correlation between impact factor and retraction rates. Nature is affectionately referred to amongst the scientific community as the Journal of Irreproducible Results (cold fusion paper, memory of water paper ...)

Peer Review

The aim of Peer Review is to make sure that publications and grant submissions meet an acceptable standard as defined by a community of peers (The idea of trial by your peers goes back to Magna carta). The question is does it work?

Only 8% members of the Scientific Research Society agreed that 'peer
review works well as it is.' (Chubin and Hackett, 1990; p.192)

"A recent U.S. Supreme Court decision and an analysis of the peer review system
substantiate complaints about this fundamental aspect of scientific research."
(Horrobin, 2001)

"is a non-validated
charade whose processes generate results little better than does chance."
(Horrobin, 2001)

"Peer Review is one of the sacred pillars of the scientific edifice" (Goodstein,
2000)

"Peer Review is central to the organization of modern science…why not apply
scientific [and engineering] methods to the peer review process" (Horrobin,
2001).


Currently most peer review is carried out anonymously. That is the reviewers are anonymous but the authors of a paper or grant proposal are known to the reviewer. Often when you submit a paper you are asked for a list of potential reviewers, some of whom will be used to review your paper. These two features of the review process often combine to make success or failure of a paper submission dependent on how good the submitting author is at "networking". Well known names with a large circle of connections select members of this group as their reviewers and so they are more frequently published and become more famous as an expert in that area and have more connections. This is a "rich get richer" scenario, which makes it very difficult for new researchers to break into a field. Especially a field which is fiercely competitive and in which the leading members have large egos.

Some of the open source journals are proposing alternatives:
  1. Publish the reviews alongside the paper.
  2. Name the reviewers so they are no longer anonymous to the authors.
  3. Anonymise the papers so that the reviewers do not know who the authors are.
  4. Publish the papers online before peer review and let the community review them before they take a fixed form.
The consequences of (1) are to moderate the language of reviews. Reviewers are likely to be more careful about what they say if they know these reviews are going to be made public. Combining (1) and (2) is even better as it forces reviewers to act objectively as their reputation amongst the community matters. Idea (3) is hard to achieve as reviewers will be able to work out who the authors are from a knowledge of their field and the literature cited (all scientists cite themselves more than others). The problem with (4) is how active will the community be? If there is a very active community in a small field this might work, but there is still the question of the status of the papers in the depository before they reach their final form when they are effectively not peer reviewed.

The quotes above were taken from an e-mail advertisement for a conference on improving peer review.

References

Chubin, D. R. and Hackett E. J., 1990, Peerless Science, Peer Review and U.S. Science Policy; New York, State University of New York Press.

Horrobin, D., 2001, "Something Rotten at the Core of Science?" Trends in Pharmacological Sciences, Vol. 22, No. 2, February 2001. Also at http://www.whale.to/vaccine/sci.html and http://post.queensu.ca/~forsdyke/peerrev4.htm (both Web pages
were accessed on February 1, 2010)

Goodstein, D., 2000, "How Science Works", U.S. Federal Judiciary Reference Manual on Evidence, pp. 66-72 (referenced in Horrobin, 2000)


Wednesday, 26 May 2010

Extrapolation to Absurdity

Here is a typical newspaper article using statistics wrongly, this particular article is from The Independent about the number of people who can recognise Winston Churchill. The article describes a study carried out on behalf of the Royal Mint to celebrate 70 years since Churchill became Prime Minister for the first time. People were asked to identify famous 20th Century Prime Ministers and only 19% could not name Churchill but this increased to 32% of 25-34 year olds and 44% of those aged 16-24.

So assuming a linear model and extrapolating beyond the range of the dataset the report predicts that in about 80 years time Churchill will no longer be recognised. This is wrong for two reasons:
  1. The model is unlikely to be linear - the recognition factor is likely to tail off more slowly in the future as there will always be a core of Churchill recognisers that will be above zero (historians and politicians for example).
  2. You cannot extrapolate outside of the experimental range with confidence. So what they could say that in 8 years time the number of 24-34 year olds who cannot recognise Churchill will rise to 44% (those sampled who are now 16-24 and did not recognise him, still cannot do so in the future sample).