If light travels billions of miles from distant stars to reach us ...

Then why doesn't the heat from those stars also reach us? Is it just too small to be noticeable, or is it dissipated elsewhere?

When you blow a balloon up, the skin gets thinner and thinner. That's because it's the same amount of material but the surface area has been increased. It's the inverse square law where the intensity drops of with the square of the distance, double the distance and the intensity drops to 1/4 of what it was. A billion miles is a very long distance so the heat and the light would've dropped to 1 / a billion squared of what it was. If you think a billion is big, a billion squared is humungous. We see the inverse square law in practically everything, from gravity to light to magnetic fields. It's one of the reasons why we know that space is three dimensions aside from time, it would be the inverse cube law if space was four dimensions aside from time.

The heat from those stars do reach Earth just as the light does but as a star is much dimmer than our Sun, the heat is equally unnoticeable. Also, the atmosphere absorbs and re-radiates heat ( the greenhouse effect ) so it's not entirely transparent to infrared ( heat ). That's why we send our infrared telescopes into space.

Your instinct is correct. Here's why:
Light travels in distinct particles called photons, and so it arrives unchanged when we see a star at night.
However, heat is radiated as infrared radiation, and so it dissipates into space and in weakens as it travels farther from the source until it dissipates completely.
Think of a fireplace in a large room. The light from the fire reaches everyone's eyes no matter where he is in the room, but people standing just a short distance away from the fire cannot feel much of the heat because it dissipates throughout the room and loses its intensity fairly quickly because it is radiated outward as infrared radiation, in waves that weaken as they move farther from the heat source.