My water heater makes a rumbling sound. This generally happens when your hot water heater is reheating the water. It usually means that your water heater has a lot of hard water deposits built up in it. Sometimes flushing the water heater can help reduce the noise, especially if its done annually from the time the heater was first installed. Flushing of your water heater should only be done by a professional. If not done properly you could end up with the hard water deposits throughout your hot water piping system requiring the whole system to be flushed.