put a volt guage on the battery when it is running and see if you're putting out 14-15 volts running. It should. If so, alternator should be fine.
Cold weather is a battery's worse nightmare. If the battery is a few years old or more, it may be getting weak. Trying to start an engine in the cold is putting quite a shock to a battery that has seen its better days. Try turning on the headlights for a minute or so, then turn them off. Then crank the engine. That should get the juices flowing before putting such a shock to the battery. At least, that is the theory.