Why many research articles are wrong.

Nature is affectionately known amongst the research community (mostly those who have not had a Nature paper) as the Journal of Irreproducible Results. Nature has been responsible for some publishing disasters - cold fusion comes to mind but the other journals should not gloat.

Here are a couple of articles about why science is often wrong
The Truth Wear Out
Significance Chasing

Word Associations

How should you visualise gigabytes of textual data?

Word arcs
Word Spectra
Other Visualisations

Data Scraping and Privacy

It is amazing to see how far "reputable" companies will go at the request of others to pry into people's personal affairs.

This is what I have come to expect from Big Pharma, but not of the scrapers.

WSJ Article

Video Game Violence

How are you to objectively measure the effect of video games on level of violence?

Here are some articles about a study that tries to create an objective measure.

Techdirt: does-being-more-vocal-video-game-violence-debate-mean-you-have-better-argument.shtml
eurekalert: Press Release

So the way the "study" creates this objective measure is by weighing the number of publications produced by both sides of an argument. I wonder if they have ever heard of the field called publication bias.

Here is another study that finds a link between video games and mental health issues
Slashdot Article

Impact Factors for Bioinformatics

Publish or be damned, or in this case publish and be damned.

NAR 7.9
Bioinformatics 4.9
BMC Bioinformatics 3.4
PloS One 4.3
Briefings in Bioinformatics 7.3
BMC Genomics 3.8
Proteins, Structure Function and Bioinformatics 3.4

What are the chances?

Here are another pair of news stories from the 27th of February 2011. Both made the front page of the BBC website that day and both were about fatalities of people who had come from Tadcaster Yorkshire

http://www.bbc.co.uk/news/uk-england-york-north-yorkshire-12592527
http://www.bbc.co.uk/news/uk-12591428

So coincidences happen ridiculously often, you always need to think about the data.

Getting the most out of Google - part 1

Basic Searching

1. Phrases in quotation marks.
2. Default Boolean is AND.
3. Use OR explicitly.
4. Negate using the minus sign (should not contain). Minus must not be separated by a space.
5. Use a + to make an explicit inclusion of a short word ignored by Google e.g. +the.
6. Also search with synonyms by using the ~ character.
7. Number ranges are separated by .. e.g. 5..10. If the second number is missing it will be a maximum or minimum.
8. There are no stemming wildcards - so * only is a wildcard for a complete word.

Special Syntax

These allow you to focus your search onto certain elements of the web.

1. intitle: limits the search to the titles of webpages.
2. inanchor: searches only in hyperlinks
3. intext: searches only in the page text - not anchors, URLs.
4. insite: restrict the search to a particular domain.
5. inurl: limits the search to a particular page.
6. link: searches for pages that link to that URL.
7. cache: searches the cache - this is useful if the page has moved/been deleted.
8. filetype: searches for suffixes of that filetype e.g. ppt.
9. related: finds related pages.
10. define: finds related definitions.

To find more detail, there is the O'Reilly book - Google Hacks The problem is parts of it are badly outdated so it is perhaps best from a library rather than to buy.

The Joy of Stats and the Visigasm

If you have never seen one of Hans Rosling's talks or videos then you need to. He is an example of how you can make the presentation of data both interesting and also clear and simple. Along with Edward Tufte he has made data visualisation an art form. The BBC have recently made a documentary about him and there is also a  Wired article about the Data Visgasm.

Peer Review Again: NASA arsenic based life and the bloggers

A critique of the Science paper by NASA investigators saying that they have discovered a new form of life that does not use phosphorous in the DNA nucleotides has been published on a Blog by a researcher in the field. The response from NASA is to put their fingers in their ears and to say they are not listening because it is not criticism from a peer reviewed journal. This is exactly the same as the authors of "The Spirit Level" who also refused to respond to criticism that was not in peer reviewed journals.

Here is a wired article discussing the NASA response and comparing it to the arguments made by the Church in pre-enlightenment times. I especially like the title - The Wrong Stuff.

The Guardian has an interesting story tracker following events

Conspiracy and Coincidence

How common is the surname Aylward?

It does not seem too common to me. I cannot remember meeting anyone who has the surname but that might reflect regional variations in surnames. My own surname is not very common outside of its ancestral home in Leicestershire, Yorkshire and Lincolnshire.

So what are the chances of the surname being involved in two major news events on the same day?

Two BBC news headlines for the 25th of October involved Aylwards.

Violetta Aylward was a nurse who accidentally switched off the life support systems of a tetraplegic man in her care leaving him with brain damage.
Rebecca Aylward was found murdered in woodland after she had been attacked by two teenage boys.

What are the chances of that happening? It did happen and there is obviously no connection but chance patterns arise all around us, and it is difficult to escape trying to read too much into them.

Dirty Tricks against Lisa Murkowski

When Lisa Murkowski narrowly lost the primary for the Republican nomination to run  for senate in Alaska to Joe Millar a Tea Party supported candidate, she decided to run a "write-in" campaign. The last time this had been successful was Strom Thurmond in 1954, and Murkowski does not have the easiest name to spell. This campaign also did not have the support of the Alaskan Republican party or the national level party who both strongly opposed the move.

Internet presence is an important part of a campaign which is going to have negative media coverage and which does not have the big money support of one of the parties or their allied groups, although her campaign still aired commercials. Accidentally I typed Linda and not Lisa Murkowski into Google trying to find her campaign, and almost all the hits pointed to the headline "Linda Murkowski: Screw you, AK Republican voters, this seat is mine". This was on five or six bulletin boards as posts mostly posted on the 19th of September, one is a disgruntled Tea Party member but more than one is a campaign. So this caught my interest and I thought I might look further.

The first place for an attack is on a candidate's wikipedia entry, as this is where many people will look for background. On the day she announced the write-in campaign her page had been vandalised by an anonymous user from 108.122.217.233. This is a Washington DC address, so I hope it wasn't a sloppy congressional staffer. Murkowski's entry was then amended to include her having failed the bar examinations four times. This was inserted by InaMaka on the 4th of October along with a citation to the Tea Party who had exposed this fact. He had been active on her page since the 20th September. After repeated warnings about bias in his submissions especially in the run-up to an election InaMaka was blocked for a week.

Feynman on Magnetism

Here is a video of Richard Feynman talking about magnetism.

Now the reason this is more interesting is the debate on Sean Carroll's Blog about whether you think Feynman is being arrogant of being honest. Now his view is similar to that from Ian Stewart and Jack Cohen in their books The Collapse of Chaos and the Science of Discworld where they talk about "lies to children".

However, the question is older than this, and is really the same on that Plato was talking about with the cave allegory in The Republic. When we give explanations do we ever give complete and definitive explanations or are they all transitory - explanations for now that will be replaced by better explanations in the future. Does knowledge depend on context?

For me Feynman was forgetting that scientists always have a duty to explain and to educate and to not assume that the level of thought of the person you are answering is far below your own. This may be the case sometimes but I prefer to find that people can manage more complex explanations than you would expect. If you get to see the Horrible Science Theatre Show then there they emphasize the problem of scientists avoiding giving explanations. In this case Feynman also cheated by making it a "why" question when it did not start out as one, but his final explanation is very good.

Lee Smolin - The Trouble with Physics

The five great problems in theoretical physics

1. Combining Quantum Mechanics and Relativity in a background independent way as relativity requires a space time (background) that evolves. There is also the problems of determinism,different observational perspectives and continuity of space and time.
   
2. Resolve the foundational problems of quantum mechanics that deal with the problems of realism and determinism to avoid the problem of observers.
3. Determine whether the particles and forces can be united in a single theory. This means combining bosons with fermions with the introduction of a limited number of tuning constants.
4. Explain how the free constants of the standard model occur - why do they take their particular values?
5. Deal with the problems of dark matter and dark energy - the apparent contradictions of general relativity at the galactic scale with respect to acceleration and the need for a positive cosmological constant for the accelerating expansion of the universe.
   

Dissecting a Politician

The unlucky focus of my dissection is Nigel Lawson who was actually the MP for the constituency where I grew up. The book I wanted to take some quotes from is "An appeal to reason: A cool look at global warming".

Even before the main text starts he uses some political tricks. This is from the Foreword:

"... this book, despite being promoted by an outstanding literary agent, was rejected by every British publisher to whom it was submitted - and there were a considerable number of them.

As one rejection put it: 'My fear, with this cogently argued book, is that it flies so much in the face of the prevailing orthodoxy that it would be very difficult to find a wide marker'. The prevailing orthodoxy can be both stifling and intolerant. Those that have the temerity to question it have to become accustomed to being labelled 'deniers' - a loaded term ..." 

Now this is already loading the book for the reader. There is no sense of objectivity and reason here. This is saying here is my reasoned argument that has been rejected by closed minded orthodoxy and I am right and they are wrong. The prevailing orthodoxy might actually be correct and so it might be the right thing not to publish something that is wrong.

Next in the Introduction:

"By way of preamble, I readily admit that I am not a scientist. But then neither are the vast majority of those who pronounce on the matter with far greater certainty than I shall do here. Moreover (and this is frequently overlooked) the great majority of those scientists who speak with such certainty and apparent authority about global warming and climate change, are not in fact climate scientists, or indeed earth scientists, of any kind and thus have no special knowledge to contribute."

So, he is ignorant but that is alright because everyone else is ignorant as well. So his voice should count equally to all the others. Actually there are various levels of ignorance. If you are stuck in an isolated valley on a camping trip with a party of friends and you break a leg. None of the party are doctors, but who would you rather have treat you, your friend the vet or your friend the accountant? I might not be a climate scientist but as a scientist I know how to weigh up data and make scientific arguments and I know what limitations there are in the data. This is a use of an appeal to authority and underneath it all his authority is that he is a famous politician who now sits in the House of Lords where they debated these issues. I remember Lord Ackner giving a similar argument about his authority as head of the bar and why it should not be questioned. Authority should always be questioned.

Next in chapter one is the absolute howler:

"Nor, incidentally, does the fact that a scientific hypothesis has been published in a 'peer reviewed' learned journal provide ipso facto any evidence either that the science is 'settled', or that the hypothesis in question is likely to be proved correct. It does not even mean that the author's data and methods are available for scrutiny, or that his results are reproducible, as scientific journals, in contrast to most leading economic journals, do not require this."

That is wrong in so many ways. Firstly they do require the data and methods to be available for scrutiny. Most of the references cited come from Science and Nature which are both poor at making the methods and data available in the articles as they are in the accompanying web material as they have strict limits on article size. Real scientific journals should contain all the details. Here he is using an ad hominem argument against all science. He also cites Popper and how theories need to be refutable, but here he talks about proving a hypothesis. According to Popper you can NEVER prove a hypothesis, knowledge is always contingent.

I hate peer review as well and as he later points out it can lead to conventional wisdom dominating and a lack of  risk taking,  but if done well by open minded reviewers who take it that they might be wrong it does strenghten papers by improving the arguments. This is so long as it is not used as a political tool as it so often is. Economics is actually critiqued by Popper in the Poverty of Historicism where he says we cannot predict the future and so it is doomed to fail, as it is based purely on induction. What we observe now might not be what happens in the future.

Aristitotle's Causes

1. Material cause - what is the object made of


2. Formal cause - the definition of the object - ontological argument - Wittgenstein's pictorial logic


3. Efficient cause - how the object was produced


4. Final cause - why the object exists

Ethics 101

Aristotle

Good and bad are not opposites - good is the mean between excess or deficiency. So between rashness and cowardice lies courage or boldness, between miserliness or profligacy lies generosity.

Mill and Utilitarianism

The best acts produce the most happiness, but to restrain hedonism and selfish personal happiness different pleasures had different values. The problem is that this leads to intellectual snobbery. The problem of high art that you can see in James Martin's book.

Trolley Problem

A runaway railway trolley will kill five people if it continues down the track or one person if it goes down the alternative track. Should we change the signal? That by a simple calculus of utility suggests we should switch the tracks. What about if the one person is a world famous neurosurgeon? Now imagine there is a bridge over the track with a large man and you can stop the trolley by pushing the large man onto the track and save the five. Should you push them?

Metaphysics 101

Kant's left handed universe

Is space absolute or relative? Newton thought that it was but Leibniz argued that if you created another universe but with everything translated then it would be the same, as only the distances between particles (internal coordinates) matter.

Kant created a thought experiment to show Newton was right and Leibniz wrong by imagining a universe that contains only a left hand. Then if you create another universe this time with an identical left hand then while all the internal distances are the same these two universes are clearly not the same as they are mirror images of each other.

Now we have general relativity that shows that space is like a plastic sheet that folds and so the space we see are all those points that share the same time. The bending is caused by gravity and acceleration, which are equivalent.

The reason why relative and absolute are important comes when we consider,

Laplace's Demon

If the universe is composed of atoms (using atoms in the broadest sense to mean particles), and these are governed by the laws of physics then if there is a super intelligent demon that knows the positions and velocities of all the particles at a particular time. Then they can calculate the positions and veolicities for any other time and the future would be completely determined. There would be no possibility of having free will other than as an illusion. We would be destined to have the life we have.

Some argue that quantum mechanics removes this rigid determinism but this was the root of Einstein's famous saying that "God does not play dice". He means that the probabilities of quantum mechanics cannot apply to determinism and in particular the theory must retain causality. Events cause future events to happen. If causality is broken then there are no laws. Chaos is another example of what seems like random behaviour but it still maintains rigid determinism, it just requires an even higher standard of knowledge for the Demon.

Ryle's Ghost in the Machine

Descartes introduced the idea of dualism, i.e. mind stuff is different to physical stuff. This causes problems with understanding how the two interract. Gilbert Ryle thought this was the wrong view. For him they are an undivided whole and the body and its activities comprises the mind - a way of describing the machine's activities. In more modern terms we can imagine that mind "emerges" from the components of the body.

Epistemology 101

Epistemology

Epistemology is the theory of knowledge. The philosophy of understanding which determines the limitations of knowledge. The foundations of science depend on epistemology. Important questions are what is truth? How do we know? Absolutists such as Plato think that there is some perfect knowledge, an absolute truth that exists although we might never discover it. Relativists deny that there is an absolute truth and argue that knowledge depends on context.

Hume and Induction

The value of science is that it makes predictions about the future, usually these are based on past experiences. This assumes that the past is representative of the future. This process is called inductive inference. The question is have we observed all of the possibilities, so that our predictions are likely to be correct or are there any unknowns, either known or even worse might their be unknown unknowns (see Rumsfeld)? Hume said that it was not logically justified but this is how we actually operate.

Popper and Falsification

Popper argued that science cannot procede by induction and suggested that it develops by "conjecture and refutation". Scientists propose models of the world (theories and hypotheses) which are they then seek to falsify.  We cannot prove a hypothesis true, but any theory that cannot be falsified cannot be a part of science. This is a way of undermining pseudo-science such as astrology which does not try to provide a falsification.

Kuhn's Scientific Revolutions

In the structure of scientific revolutions Kuhn argued that "normal science" takes place in the context of scientific paradigms (fundamental world views) which define how we create our models/hypotheses. Sometimes these paradigms fail during revolutions which follow a period of crisis when existing science faces an increasing number of paradoxes. One example is the Copernican revolution in astronomy. The fluid model of electrical conduction is another important example as is the move to quantum mechanics from classical mechanics. At the minute both physics and biology are facing a period with an increasing number of puzzles and paradoxes and so it is possible that we are entering another paradigm change.

Pragmatism

This is the view of William James. The question we should always ask is "What practical difference does it make?" If the answer is none then the alternatives can be implied to be the same. So a belief is true if if helps us practically - is it useful, helpful applicable. This is the nature of Truth. Probably this is close to how most scientists think. Is it also applicable to religion? Can we just believe certain tenets of faiths because they make a practical difference to our experience of our lives?

Rumsfeld on unknown unknowns

Baggini gives a version of Rumsfeld's speech, which he points out to be philosophically correct but in a context that I cannot find on videos of the speech. I cannot find a video of the section "Reports that say that something hasn't happened are interesting to me, because as we know there are ...". All the videos I can find show Rummy to be evasive and suggest he does not merit the benefit of the doubt he is given by Baggini, as he ends the interview saying I am not going to tell you which they are (known unknowns or unknown unknowns).

The video is here but it might be edited:
Most complete - starts at "There are .."
He also seems to have used the same speech more than once.
Here is another video.


Here is the official transcript version used by Baggini:

Rumsfeld: Reports that say that something hasn't happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.

This is the problem of induction and the weakness of positivism. We just cannot know what we do not know. The typical example is swans are white, and experience said this was true until we found black swans. You can never prove a hypothesis you can only falsify it - see Karl Popper on refutations.

The term was used by the US military before Rumsfeld in giving guidance for war games and suggesting unknown unknowns should be incorporated into games so that they were unfair, because real life is unfair. For the Star Trek geeks an example of this is Kirk's response to the Kobayashi Maru simulation from the
Wrath of Khan.

Fallacies: from J Baggini - Do They Think You're Stupid.

If I don't do it somebody else will

"If we want to stop the defence industry in this country, we can do so. The result incidentally would be that someone else supplies the arms that we supply."

Tony Blair

If you carry out the action then you are directly responsible. You cannot be held less responsible because it was inevitable. This confuses cause and responsibility

Existentialist Fallacy

This is the confusion of can or ought with must. Just because someone carries out an action or expresses an opinion, this just means that it is a legitimate choice. It does not follow that everyone should make the same choice.