How to properly conduct an experiment

[Matt Timion]

Experiments must be run in a certain way in order to reduce error. Error is introduced into any experiment as a result of conditions that are different from group to group. What this means that is that we must attempt to keep as many things the same as possible between experimental conditions.

I will explain the rules for participating in experiments, and then give an example.

Before testing a particular method, you must pick a stretch of highway/free which you can travel on uninterupted for a long period of time. If you can travel for 100-120 miles one way, it is better. The reason we pick highways and freeways over city driving is simple. There is too much potential error to be introducted into city driving. For example, if before testing you have all red lights, and after making a modification you have all green lights, it would be unknown if the increased gas mileage is due to the modification you made or due to the fact that you had zero idling time. Traveling on the highway reduces error that can be introduced due to idling, red lights, trains, traffic, etc.

You will first obtain a base measure of your gas mileage. To do this you must fill up your gas tank, travel the strip of highway/freeway and then travel back. You must then fill up immediately. Divid total miles driven by total gallons of gas and that is your gas mileage prior to modification. You then will make the modification to you car (fuel additive, engine contraption, etc) and the make the same trek, using the exact same method. Your new gas mileage will be compared to your old one. Since the only thing that changed was your modification, it can safely be assumed that your modification is the reason for gas mileage increases or decreases.

It is vital that you also maintain the same speed, have the same tire pressure, etc for both trips. Keep as much the same as possible.

Report your results on the thread dedicated to your experiment. When enough people have weighed in their results, statistical analysis will be run and the final report will be written up.

[theholycow]

WorkingOnWise,

1. I think most people on this forum get the most consistent results when they stop pumping without topping off at all. I vaguely recall reading about someone who uses a small bottle of gas and weighs it before and after the experiment or something. How do you know the "gallon or more" variation you saw was pump error?

2. Yes. Air temperature is very important, and I suspect the others are too.

3. Why would the weight change during the experiment? It should be the same.

4. Narrower tires have higher rolling resistance at the same pressure (assuming they're the same model tire), because they have to deform the sidewall more to make the same size contact patch (which you will get when you have the same pressure and weight). They do have less aerodynamic drag, though. I've posted a few lengthy posts explaining this in a little more detail; I really need to put together a canned post for it...
http://www.gassavers.org/showpost.ph...99&postcount=7
http://www.gassavers.org/showpost.ph...&postcount=151
http://www.gassavers.org/showpost.ph...46&postcount=8

[WorkingOnWise]

theholycow,
The pump error I found was by filling to the point that the nozzle tripped off, noting the quantity of fuel, the topping off the tank, right to the top of the filler neck. I did this on several pumps and several cars a few years back, and saw some pretty wide variations. During this time I had a 1978 Olds 98 that had the fuel neck behind the license plate. That particular car would take around 3 more gallons after the nozzle first kicked off. I blamed it on the fact that the filler neck was on such a narrow angle that the foam from filing didnt get enough time to dissipate before backing up into the fill nozzle.
I have also seen pumps that tried to pump fuel so fast that almost as soon as I hit the lever, the nozzle tripped off, and there was still a 1/4 tank to go on the gas gauge.

The weight could change because if I decide I want company on a test run one day, and not the other, I gain or lose up to 200 lbs, depending on who goes with me 150 miles, plus 15 miles to warm the tires and drivetrain, plus time to fill up 2 times makes for about 3 1/2 hours of time. Good time for bonding with my boy...or listening to my music that drives the rest of the fam crazy! lol
If the weight will skew the results too much, I can compensate with sand bags when I ride alone.
The goal for me is to test in a real-world but consistent environment what works, and how well.

Thanks for the info on tires. I'll be reading that next!
Keith

[flapdoodle]

Originally Posted by WorkingOnWise

theholycow,
The pump error I found was by filling to the point that the nozzle tripped off, noting the quantity of fuel, the topping off the tank, right to the top of the filler neck.

My filler is on the side, and I found that sometimes I can pump a little more fuel in if I bump the side of the car with my hip to make the fuel in the tank slosh. I have done this so many times it is just habit now. Sometimes it is enough that I can hear it sloshing.

The concrete around the pumps are sloped slightly for water runoff, so the side of the island I choose will determine it the filler neck is high or low relative to level.

[theholycow]

Originally Posted by WorkingOnWise

The pump error I found was by filling to the point that the nozzle tripped off, noting the quantity of fuel, the topping off the tank, right to the top of the filler neck. I did this on several pumps and several cars a few years back, and saw some pretty wide variations. [...]

But, did you try it and find it inconsistent on the same car, same pump, same day? That's the accepted way to call it "accurate" around here, and I'd be interested to hear if that assumption has been experimented and proven wrong. It wouldn't be the first time for me.

Thanks for the info on tires. I'll be reading that next!

I finally made the canned post, so I'll have it ready for next time.

[WorkingOnWise]

Ya know, I never did do a conscious test on the same car, same pump, same day to be sure. I figured if there is fuel at the top of the filler neck, it's pretty hard to say it isn't full yet...but not knowing where the air pockets might be on the top of my fuel tank, if the angle of the vehicle were right, the location of an air pocket can change too...so my method of full being full could easily be flawed too. Bummer!
Well, I have been playing with the idea of a installing a 6 gallon auxiliary fuel tank to use for fuel additive testing, so I only have to burn off 6 gallons of gas instead of 20 before I can test the next additive. Now with this discussion of full or not, I will definitely add the auxiliary tank, set up with a switch so I can flip between them as needed, like the F series ford trucks have had forever. Then, I can control the shape of the tank, and be sure that I start out with the exact amount of fuel each time, no matter what the pump might say or do. Now the trick is to find a good place to install in under a 1995 Villager!

[GasSavers_Roadkill]

All I will say if you are going to do a gas millage test , do it the old way of how much gas it took to fill up your fuel tank, then drive till it is empty note how much city and open highway driving one did. then fill up your fuel tank again, subtrackthe amout of gallens it took to fill it up again to get the total gallens of gas used. then divid the gallens by the miles traveled to get your real fuel millage. And please Stop using that stuped scan gage for it does not tell the real millage.

[Hateful]

I've yet to see any difference in mileage with the scangauge and my trip meter on the instrument panel; accept for a few miles where I Engine Off Coasted without having the gauge set to Hybrid. My gas logs have always been as described above. The tank MPG on the scangauge has been off before but it's still a good tool for testing.

[Daveedo]

Let me know if I missed it but I did not spot any reference to conducting a statistical power test of the experimental results. Basically from what I can remember from college (biostatistical analysis) is that a power test will help you determine if your results have some merit by analyzing how much variance is in your data (hopefully not datum cause then you'd have no way to figure out the power ).
If you have too much variance in your test results then then you may need to revise the test with stricter parameters or enlarge your sample field.

Just thought I'd throw that out there. I may go look to see if I still have that statistical analysis book cause its such great reading

[Draigflag]

Yea every fuel pump is different, most overestimate how much you put in, some are slightly under. You can test this by filling a small Jerry can that you know the exact size of. 5 litres is pretty common, fill it up and I bet you a small sum of money the pump will read higher that what you've allegedly put in. Now empty it and try it again at a different station, it will be different again. Annoying but true...