How to properly conduct an experiment

[GeneW]

A few more points....

1. Keep careful records, especially of external factors like ambient temperature, precipitation, number of passengers (including estimates of their weight), type of gasoline (including vendor) and whether any maintenance was recently done on the car (especially performance related issues like tune ups, ignition system changes, or fluid changes).

2. As one of my former supervisors at a Fortune 25 R&D lab said to me, "We let the data speak to us". Record your data with appropriate notes and do not draw any conclusions until you've formally finished the tests.

Sometimes things aren't what they seem. Sometimes there are unknown effects which matter more than what you tried to change. Sometimes these unknowns can only be inferred from the data itself.

Don't fall in love with any ideas, especially your own ideas. Be ready to discard anything if it doesn't make sense.

Richard Feynman put it best - real science involves putting anything into your report that would tend to falsify your results. If you poured a bottle of dry gas into the tank record it. If you had bunions one day that made your feet light on the pedal record that day, if you saw the gasoline tanker truck pulling out of the gas station record it. If you got stuck behind a lumbering garbage truck which drafted you. Etc.

Gene

[WorkingOnWise]

I'm going to do some formal tests on several products and methods this summer, as part of a business plan, and I have some questions that I think the collective wisdom on this forum can help with.
I have a 75 mile stretch of highway picked out that I can travel uninterrupted on pretty much anytime I need to, and I will run down this 75 mile stretch and back, giving me a 150 mile test trip.
I will begin and end each test run by driving a 15 mile highway loop then filling up at the same pump at the same gas station.
My questions are these.

1) Is topping off the gas tank, up to the top of the filler neck, really a good way to be sure there is the same quantity in the tank for every test? My thinking is that the fuel is aerated as it flows out of the nozzle, and this foam can give a minor variation in actual fuel quantity. I have seen a variation of a gallon or more when the nozzle shuts it self off. What do you think?

2) Do I need to try to match the atmospheric pressure, ambient air temperature, and relative humidity levels of my baseline tests? Or is this too anal?

3) How close should the vehicle weight be to the baseline to ensure measurable accuracy. The van weighs 3800 lbs. Is a 200 pound variation enough to skew the results?

4) Is there any measurable benefit to tires with lower rolling resistance? Maybe narrower tires that the OEM installed, as long at the overall height remains correct? If so, where can I get information on tires that give the best MPG, or that have the lowest rolling resistance?

Thanks
Keith

[theholycow]

WorkingOnWise,

1. I think most people on this forum get the most consistent results when they stop pumping without topping off at all. I vaguely recall reading about someone who uses a small bottle of gas and weighs it before and after the experiment or something. How do you know the "gallon or more" variation you saw was pump error?

2. Yes. Air temperature is very important, and I suspect the others are too.

3. Why would the weight change during the experiment? It should be the same.

4. Narrower tires have higher rolling resistance at the same pressure (assuming they're the same model tire), because they have to deform the sidewall more to make the same size contact patch (which you will get when you have the same pressure and weight). They do have less aerodynamic drag, though. I've posted a few lengthy posts explaining this in a little more detail; I really need to put together a canned post for it...
http://www.gassavers.org/showpost.ph...99&postcount=7
http://www.gassavers.org/showpost.ph...&postcount=151
http://www.gassavers.org/showpost.ph...46&postcount=8

[WorkingOnWise]

theholycow,
The pump error I found was by filling to the point that the nozzle tripped off, noting the quantity of fuel, the topping off the tank, right to the top of the filler neck. I did this on several pumps and several cars a few years back, and saw some pretty wide variations. During this time I had a 1978 Olds 98 that had the fuel neck behind the license plate. That particular car would take around 3 more gallons after the nozzle first kicked off. I blamed it on the fact that the filler neck was on such a narrow angle that the foam from filing didnt get enough time to dissipate before backing up into the fill nozzle.
I have also seen pumps that tried to pump fuel so fast that almost as soon as I hit the lever, the nozzle tripped off, and there was still a 1/4 tank to go on the gas gauge.

The weight could change because if I decide I want company on a test run one day, and not the other, I gain or lose up to 200 lbs, depending on who goes with me 150 miles, plus 15 miles to warm the tires and drivetrain, plus time to fill up 2 times makes for about 3 1/2 hours of time. Good time for bonding with my boy...or listening to my music that drives the rest of the fam crazy! lol
If the weight will skew the results too much, I can compensate with sand bags when I ride alone.
The goal for me is to test in a real-world but consistent environment what works, and how well.

Thanks for the info on tires. I'll be reading that next!
Keith

[flapdoodle]

Originally Posted by WorkingOnWise

theholycow,
The pump error I found was by filling to the point that the nozzle tripped off, noting the quantity of fuel, the topping off the tank, right to the top of the filler neck.

My filler is on the side, and I found that sometimes I can pump a little more fuel in if I bump the side of the car with my hip to make the fuel in the tank slosh. I have done this so many times it is just habit now. Sometimes it is enough that I can hear it sloshing.

The concrete around the pumps are sloped slightly for water runoff, so the side of the island I choose will determine it the filler neck is high or low relative to level.

[theholycow]

Originally Posted by WorkingOnWise

The pump error I found was by filling to the point that the nozzle tripped off, noting the quantity of fuel, the topping off the tank, right to the top of the filler neck. I did this on several pumps and several cars a few years back, and saw some pretty wide variations. [...]

But, did you try it and find it inconsistent on the same car, same pump, same day? That's the accepted way to call it "accurate" around here, and I'd be interested to hear if that assumption has been experimented and proven wrong. It wouldn't be the first time for me.

Thanks for the info on tires. I'll be reading that next!

I finally made the canned post, so I'll have it ready for next time.

[WorkingOnWise]

Ya know, I never did do a conscious test on the same car, same pump, same day to be sure. I figured if there is fuel at the top of the filler neck, it's pretty hard to say it isn't full yet...but not knowing where the air pockets might be on the top of my fuel tank, if the angle of the vehicle were right, the location of an air pocket can change too...so my method of full being full could easily be flawed too. Bummer!
Well, I have been playing with the idea of a installing a 6 gallon auxiliary fuel tank to use for fuel additive testing, so I only have to burn off 6 gallons of gas instead of 20 before I can test the next additive. Now with this discussion of full or not, I will definitely add the auxiliary tank, set up with a switch so I can flip between them as needed, like the F series ford trucks have had forever. Then, I can control the shape of the tank, and be sure that I start out with the exact amount of fuel each time, no matter what the pump might say or do. Now the trick is to find a good place to install in under a 1995 Villager!

[GasSavers_Roadkill]

All I will say if you are going to do a gas millage test , do it the old way of how much gas it took to fill up your fuel tank, then drive till it is empty note how much city and open highway driving one did. then fill up your fuel tank again, subtrackthe amout of gallens it took to fill it up again to get the total gallens of gas used. then divid the gallens by the miles traveled to get your real fuel millage. And please Stop using that stuped scan gage for it does not tell the real millage.

[Hateful]

I've yet to see any difference in mileage with the scangauge and my trip meter on the instrument panel; accept for a few miles where I Engine Off Coasted without having the gauge set to Hybrid. My gas logs have always been as described above. The tank MPG on the scangauge has been off before but it's still a good tool for testing.

[Daveedo]

Let me know if I missed it but I did not spot any reference to conducting a statistical power test of the experimental results. Basically from what I can remember from college (biostatistical analysis) is that a power test will help you determine if your results have some merit by analyzing how much variance is in your data (hopefully not datum cause then you'd have no way to figure out the power ).
If you have too much variance in your test results then then you may need to revise the test with stricter parameters or enlarge your sample field.

Just thought I'd throw that out there. I may go look to see if I still have that statistical analysis book cause its such great reading