This is where bad science starts

This was originally published at my personal “No One’s Listening” blog, but I have decided to re-open my Optimiskeptic blog, and use this as the inauguration post. Unfortunately, I do not know how to transfer the original comments over, but you can find them here. They are helpful in clarifying and correcting some things I wrote in the blog. There is a much better comment system integrated into this blog, so I hope that helps a little.

I recently read this Gizmodo article that questioned whether or not the results of a 13-year-old kid, Aidan’s, science experiment was properly debunked.  Aidan’s science experiment is noteworthy for three reasons: (1) He claimed to have increased the efficiency of solar cell power generation by simply arranging them in a Fibonacci  (Golden Ration) pattern copied from the leafing pattern of plants found in nature, (2) his findings, published as an essay, received a “Young Naturalist Award” from the American Museum of Natural History (and a provisional patent, no less), and (3) this was reported on and praised as “genius” and a “breakthrough” by several noteworthy magazines such as Poplar Science, Slashdot, and The Atlantic Wire.

So here’s the problem. Aidan did not actually discover a more efficient way to convert solar energy into power as he claimed and these numerous publications reported. In fact, Aidan’s essay, while extremely well written, contains methodological flaws and incorrect conclusions. This rebutted published on blogspot by The Capacity Factor (and then subsequently removed) correctly and thoroughly explains why Aidan’s findings are inaccurate.  However, the Gizmodo article I read stated:

“Many of you in the comments remarked that 13-year-old Aidan Dwyer’s breakthrough was nothing new. Fair enough. But it’d be different if someone totally disproved it. One blogger claims to have done it. Can I get an expert in here?


NOW! I am no expert, so I can’t make any assertions as to the veracity of Mr. Blogspot’s claims. Graphs looks great, but what do you guys think?

Even The Atlantic Wire‘s article about the debunking did not provide much further explanation or analysis. It simply reported the facts of the story: Kid is praised for solar power “breakthrough.” Blog debunks findings. Here is a summary of the blog’s findings. Blog blames media for the mess.

Of course, I guess it is acceptable/safe for The Atlantic Wire to report the way it did by simply reporting the facts and to not interject with opinion or commentary.  However, Gizmodo articles, like the one I read, are specifically written in a commentary/opinion style.  Therefore, I was dismayed that Giz writer, Kwame Opam, felt that he needed to be an expert to verify The Capacity Factor’s findings.

You don’t need to be an expert to verify that Aidan’s findings are incorrect. I’ll admit, The Capacity Factor’s blog post was certainly a bit overlaiden with techno-jargon and can seem complicated and difficult to understand. But the flaws in Aidan’s science experiment are actually quite simple. They are so simple, in fact, that someone at the American Museum of Natural History should have noticed them. To be fair, the award might have simply been for his writing style and the fact that he praises nature, but for some reason I don’t buy that: “Young Naturalist Award” seems to imply the award is for a scientific achievement, not an essay-writing one.

Or someone at Popular Science, The Atlantic Wire, Slashdot, or Gizmodo should have noticed the flaws with Aidan’s work, yet it seems that they either glossed over the essay or did not understand what was wrong with it. In fact, any lay person with a basic understanding of science or 15 minutes of time to do a few Google searches (as I did for this blog post) and re-aquatint themselves with what they (hopefully) learned in high school should have caught the flaws in Aidan’s paper. Yet, not only did none of these media outlets stumble upon these flaws, even those reporting on the expressly admitted that they still did not understand the reasoning.

This would seem to me why so much bad science passes for good science these days. Please allow me to lay out my case, and perhaps hopefully re-explain The Capacity Factor’s debunking so that it can be understood by all the non-experts out there:

1. Nature generally doesn’t achieve maximum efficiency.

Nature usually operates by finding efficient minimums. That is, traits found in nature naturally develop and survive as long as they are the bare minimum necessary to survive – not the efficient maximum.  Nature can be extremely inefficient. It is faulty to assume that trees have a pattern of leafing that maximize their sunlight exposure. Rather, it would be correct to assume that trees have developed a pattern that give them the minimum necessary amount of sunlight exposure to survive long enough to reproduce. That, however, is not efficiency.

From Aidan’s essay:

I now had my first piece of the puzzle but it did not answer the question, Why do trees have this pattern?

Good science always starts with good research. See what others have done in the past, then find out where they went wrong, right, or can be improved upon. Aidan did excellent research about the history of the Fibonacci sequence, and Charles Bonnet’s observations of the sequence existing in tree branch growth. However, he did not appear to research the answer to his actual question: Why do trees have this (Fibonacci) pattern?

Aidan correctly notes that Fibonacci numbers appear almost everywhere in nature: Galaxies, moth wings, seashells, finger bones, Saturn’s rings, pine cones, honey bee colonies, etc. A few good articles on this can be found at: HowStuffWorks.com, World-Mysteries.com, University of Chicago’s Prof. Narain’s Golden Ratio Page, and MathIsFun.com. MathIsFun.com explains the answer to Aidan’s question best. In summary, the Golden Ratio is an irrational number, thus creating an ever expanding spiral pattern with few gaps.

The quick and easy answer is that the Golden Ratio is a pattern that naturally reveals itself in things that are growing. As MathIsFun.com put it:

Leaves, branches and petals can grow in spirals, too.

Why? So that new leaves don’t block the sun from older leaves, or so that the maximum amount of rain or dew gets directed down to the roots.

And here is where Aidan began to go wrong. Over time, plants grow. New branches and leaves are constantly forming and need to do so in such a way that they don’t harm previously existing ones. Solar panels don’t grow.  They are designed, arranged, and set up by humans at one time and don’t change in size, structure, or arrangement ever again unless someone comes along and changes them.

Aidan observed that Oak trees branch out in a Fibonacci pattern. But it is not necessarily to maximize sunlight exposure. Miximizing sunlight exposure doesn’t explain seashells, galaxies, or pine-cones. Rather, the Fibonacci sequence is a growth pattern. It is found in things that grow all over the place. You see, Aidan failed to factor in “constant changes over time” as the reason, thus setting himself up for failure with a faulty experiment.

If, like solar cells, plants just deployed themselves and didn’t grow from a tiny sappling to a gigantic tree, they would not maximize sunlight exposure by arranging themselves in the Fibonacci pattern as Aidan incorrectly assumed.

2. Optimal orientation is deductively derived using very few (and all possible) variables and therefore cannot be improved upon.

Inductive reasoning is a type of intuitive logic that involves observation. Observation is highly susceptible to misinterpretation. Deductive reasoning, however, involves provable mathematical constants: The conclusion always follows the premises. Inductive logic would be like looking at the heights of three men and noting which one is tallest and shortest based on simple observation. Deductive reasoning would be using math to determine the same thing: A > B, B > C, therefore, A > C.

Aidan decided to perform an inductive experiment to compare the efficiency of different arrangements of solar panels.  There is only one problem: This has already been mathematically proven using geometry and algebra, and no amount of observation will improve upon that math.

Aidan’s experiment compared solar cells arranged in a Fibonacci pattern like an Oak Tree, and compared the output to the optimum orientation of Solar Panels. Had Aidan understood why there is an optimum orientation for solar panels, he would have been able to conclude that it is mathematically impossible to improve upon this design. MACS Lab Inc., an environmental, health, and safety consulting service, explains the math behind optimum orientation pretty well on its website.

Basically, there are two factors in determining optimality: Latitude, and time of year. The goal of optimum orientation is to have fixed solar panels face the sun as directly as possible, allowing them to gather the maximum amount of sun rays taking into account the movement of the sun across the sky during a given day.

This orientation is called optimal because it produces a maximum exposure to the sun’s rays. This is mathematically impossible to be improved upon.  You cannot out-max a mathematically maximized number. Any deviation from the optimal angle will produce inferior results.

3. Increasing the number of something generally doesn’t change the outcome.

This is the most fundamental flaw with Aidan’s experiment, and the first thing everyone should have noticed. If solar cell A is placed at an angle that maximizes its own individual power production, and solar cell B is placed at any different angle, it will produce less power every single time. If this is true for one solar cell, this is true for 100 solar cells. You can’t add “smaller” numbers together to get a larger total than the same number of “larger” numbers.

Lets say you have 10 solar cells collecting power at an “optimal” level, that means each solar cell is maximizing its intake. Lets say they all are maximized at 5 watts (Watts is the unit that measures power – look at your electric bills. Notice that the more of these you use, the higher your bill is.). The total maximum power that can be collected is 50 watts.

Now, lets say you have an arrangement of 10 solar cells where any one of them is arranged so that it is not at “optimal” level. That means individually it will not collect 5 watts of energy. It might collect 4.5 watts, or 4.7 watts, or 3… but not 5. Logically, you will never get 50 watts of total energy if any single individual cell is collecting less than its maximum (5). In the case of Aidan’s experiment, all his solar cells are arranged at different angels, so every single one (except one) must be collecting a “suboptimal” amount of power. There is no magic mathematical formula that will allow these numbers to add up to a larger total than the ones arranged at the optimal angle. It is pretty simple logic, so you don’t have to be a math whiz to get this concept.

Of course, Aidan’s experiment goes one step further. His tree pattern actually uses more solar cells to gather light than his control group. Now he’s not even comparing the same thing! If 18 people can lift more weight than 10 people, there wasn’t an efficiency gain, there’s simply more people doing work. In the case of Aidan’s experiment, he compares the electronic generation of 10 solar cells in one pattern to that of 18 solar cells in a different pattern, then claims that the pattern was the cause of the 18-cell configuration gathering more electricity. Hmmm… that just doesn’t make any sense at all, does it?

4. Voltage is not a measure of power.

And here is where the entire experiment falls apart.

Most people don’t understand what voltage is, and warning signs are entirely to blame. You’ve all seen the signs: Danger! High Voltage, right? We all know that a certain amount of electricity can hurt you, and a higher amount can kill you. By saying Danger! High Voltage, we are lead to believe that voltage is a measure of the amount of electricity. It is not.

Voltage is a measure of the level of attraction (electrical potential) between a positive and negative.

The human body can build up to 25,000 “volts” of static electricity, but there’s almost NO power involved – if there were, a simple static shock would easily kill you. Obviously, when you experience static shock, there isn’t a lot of power transfer going on. Power is voltage times current. Current is basically the number of electrons flowing through a medium, and voltage is the “speed” with which they move (this isn’t entirely accurate, but it paints the right kind of picture for the uninitiated).

Danger! High Voltage signs aren’t warning about the amount of electricity. They are warning you, “Hey, this electricity really wants to get out of here – if you so much as walk near this circuit it might jump across the air and go into you!”

Think about a battery: A 12V battery is always 12V whether its fully charged or almost dead (this is actually not entirely true, but simplified for the sake of example – see comments below where someone gives a more accurate explanation). A constant electron flow comes out of it at 12V until it runs out of stored energy. Eventually the battery has almost no power left, but it has always had 12V of electricity. If Aidan were to measure the voltage of two 12V batteries – one brand new one and one nearly dead, they would both read 12V. Obviously that measurement isn’t going to tell him anything about the amount of power stored in the battery, is it?

From Aidan’s essay:

I measured the performance of each model with a data logger. This recorded the voltage that each model made over a period of time. The data logger could download the measurements to a computer, and I could see the results in graphs.

“Making voltage” is not making electricity.

I’m not entirely sure why Aidan thought that he could measure power intake by measuring voltage on his solar cells. I’m not entirely sure why the different arrangements yielded different voltage totals (see first comment below for explanation of this). I do know that solar cells are designed to convert energy from photons into potential energy in the form of electrons: “charging the battery.” Levels of voltage have nothing to do with how charged that battery is, however, and at no time during his experiment was Aidan actually measuring how much power was being converted by each of the solar cell arrangements.

Here is what I am sure of: No one bothered to correct him. Everyone who read Aidan’s essay and reported on it simply assumed that voltage was something it was not. This is possibly because no knew what unit of measurement was used to measure power, or no one bothered to take a few seconds to look it up.

Here is one way Aidan could have measured electric power being produced in his solar cells. Also, Chapter 11 and Chapter 13 of “Learn Physics Today” from Oracle’s ThinkQuest site explain what power actually is, and how it differentiates from voltage.

So then why did Aidan get different voltage readings at different times of day on the setups? Voltage on a circuit is measured using a voltmeter, or the voltmeter setting on a multimeter. This video explains how to use a multimeter. Readings can only be properly taken on an open or closed circuit that is powered. Now, keep in mind, this is not measuring the number of electrons moving through a circuit. This is measuring the force with which through those electrons are moving through.

Historically, voltage was called “tension” and “pressure.”  Think of it like water moving through a hose with a pump at the end. The water itself is the electrons (current). The more you turn up the pump, the greater the difference in water pressure between the pump and the end of the hose. That is your voltage reading.

Keep in mind, how high the pump is turned up does not necessarily indicate how much water is coming out the end of the hose. For example, if the hose is plugged at the end, the water pressure on that hose will be high (high voltage), but the actual amount of water (electrons) moving through the hose is zero. Zero electricity is being produced. You can adjust the pump up or down, thereby increasing or decreasing the water pressure (changing the voltage), but you aren’t changing the amount of electricity moving through the hose, because there is still none moving through it.

The key to all of this is the presence of water. If there is no water present, it doesn’t matter how high or low the pump is set – there will be no water pressure. Now, you understand why Aidan got voltage readings during daylight and none during nighttime.

As the sun came up, the solar cells began to convert sunlight into electricity – the water was being added to the pumping system.  All of the sudden he started to get voltage readings. There were electrons present which wanted to move from one end of a circuit to another with a certain amount of force – and Aidan was measuring the force with which they wanted to move – not how many electrons were actually present.  Because Aidan was completing the circuit with his voltmeter, the electrons were free moving through the circuit and not building up at one end of the other. Therefore, once the sun went down, there were no more new electrons being introduced to the circuit, and therefore the pressure, or voltage, dropped because there was no potential energy to measure anymore.

So, why did Aidan get two consistently different voltage readings from the two different circuits? Going back to the water pump example, any number of things can affect the operation of the pump, or how much pressure it can create. These might include temperature, how old the pump is, and when the pump was built. Who really knows what the specific factors were that changed the voltage on Aidan’s circuit (David Keenan does – it has to do with heat – see his comments below) – but those voltage levels he got would have been the same difference no matter what.  If Aidan placed his higher voltage circuit in partial shade up in Canada and his lower voltage circuit in direct sunlight down at the equator, he still would have received a high voltage reading off the one circuit and a low voltage reading off the other. Of course, now this would cease to make sense because obviously the solar cells in partial shade and way further north couldn’t possibly be creating more electricity than ones in direct sunlight at the equator. In fact, if Aidan had oriented his Fibonacci solar cells in new and random patterns, he would have still gotten the same voltage reading. Solar cell orientation had nothing to do with the voltage readings.

That is ultimately why Aidan couldn’t detect his experiment’s failure. Just like the Canada/shade equator/direct-sunlight comparison, the two solar cell arrangements Aidan was working with were equally dissimilar. The solar cells at optimal orientation had a huge advantage over the ones in the tree pattern – I already explained how this was so mathematically – but Aidan wasn’t measuring the amount of electricity being generated – he was measuring how fast that electricity could be generated on his circuit and one had nothing to do with the other.

5. No one stopped to think.

So, Aidan’s experiment was doomed from the start. The experiment was faultily designed using a fixed object to test why a growing object grows in the arrangement it does. It was already deductively (mathematically) proven that there was no possible better way to arrange fixed solar cells to increase electric generation. By increasing the number of solar cells being used in his experiment (necessary to arrange them in a Fibonacci “pattern”), Aidan was merely increasing the faultiness of it, not the efficiency of it. And finally, when push came to shove and Aidan measured his results, he measured the wrong thing.  Ultimately, this lead Aidan to incorrectly conclude that the arrangement of branches and leaves on trees allows them to gather the most possible sunlight they can. The answer to Aidan’t question is that trees could probably gather more sunlight with a different arrangement, but in order to grow, they must adopt a pattern of growth that allows them to continuously and evenly expand: The Golden Ratio found everywhere in nature. So Aidan was partly-right. At least Aidan was thinking.

All four of the faults with Aidan’s experiment were right there in Aidan’s essay for anyone to discover. Each of them could either be logically deduced with just a tiny bit of knowledge or discovered by doing a couple quick Google searches on the subject. In other words, all someone had to do was stop and think and they would have spotted these problems. Did anyone do this? No. The American Museum of Natural History didn’t stop and think. Popular Science didn’t stop and think. Slashdot, Gizmodo, The Atlantic Wire, UberGizmo, Inhabitat, EarthTechling, and TreeHugger didn’t stop to think either. They all just got caught up in the hype: “Oh, some kid claims to have increased solar cell efficiency! AMNH gave him an award, it must be true! The Fibonacci sequence is involved! Fascinating! Quick, report on it!” And most of these publications claim to be scientific publications. It is sad.

The only person thinking in this story was Aidan. As The Capacity Factor said in its debunking blog post, you can’t blame a 13-year-old kid for this experiment going wrong. All along the way he had to have been misinformed and misguided into building the faulty theories and misunderstanding the basic electronic principals that were necessary to properly conduct his experiment. But at least Aidan was thinking, questioning and searching for answers. None of the people who awarded him or wrote praise about him were.

And that, my friends, is where bad science starts. It starts when adults, professionals, and people perfectly capable of rational, logical, independent thought choose to turn off their brains, fixate on the conclusion of an experiment and choose to either affirm or deny that conclusion based on how happy or unhappy they are with it. This is why we have a divided country on scientific issues such as the nature and causes of global warming, the moral implications of modern stem cell harvesting, and even the very definition of when life begins. No one bothers to work their way through the science. One person publishes something that fascinates or revolts, other scientists jump on a bandwagon and affirm or deny without properly reviewing, media outlets report, even sometimes completely misreport the findings without examining the entire story, and opinionated people make up their mind about what is true or not true based on how they feel about the story without spending much careful time objectively examining the evidence.

Shame on you American Museum of Natural History. Shame on you media outlets who reported on this story. Shame on you commenters who praised Aidan’s findings as “genius,” “obvious,” or something else they weren’t. And congratulations Aidan. Your curious mind and extremely good writing skills gave us all something to think about and even accidentally fooled a lot of smart people into believing the impossible. Don’t ever stop thinking, Aidan.

To see original comments on original blog post, go here.

Followup: This is where bad science leads

If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.