For the most part, any electrical device will put as much heat into your house as it uses. An obvious exception is things like the hot water heater, with most hot water going down the drain.
In the case of a fridge, the only caveat is that most of them give off their heat from the back of the fridge, so most of this heat goes into heating an external wall. If possible, put your fridge in your house in such a way that it is completely inside your house, on all sides.
I'd like to rant about how dumb it is to place baseboard heaters on exterior walls, instead of interior walls, since a significant portion of that heat goes into heating the wall instead of the interior, but that's off topic so I'll shut up.
We have electric heat also, and in the wintertime we are very liberal about leaving electrical devices (such as the TV or computer) on, however in the summertime we are very careful to run these devices at an absolute minimum.
We also don't run the exhaust fan in our bathroom - we leave the door open during showers to help recover heat and humidify the house. Oops, off topic again.
-Bob C.