I had this identical problem. Every once in a while I would turn the ignition key and .... nothing..... Gradually got worse. The culprit was a loose or corroded battery connection on the starter itself. One of the service writer techs could actually see sparking when he was looking under the hood.
If it is the battery connection itself AT the battery, I would expect your clock to get reset every time you had this problem, since you are dropping electricity to the entire car.
__________________
|