I tried to start it immediately afterwards and it sounded like it had a low battery and couldn't start (instrument lights dimmed as I pressed the starter button, engine wouldn't turn over).
So current is going to the starter, just not enough: either more than the battery can supply or the battery supply isn't fully getting to the starter.
I ended up turning everything off and waiting a few minutes before I got it working again, so I thought it was a battery problem. Instead, when I tried starting it up a few hours later the bike turned on like it didn't have a problem, so it wasn't that the battery was dead.
I think more likely a bad/loose battery connection, either positive and/or ground. They can do illogical things like this. If not that then maybe your starter motor brushes, but they are pretty durable on these.
If the bike has power to restart several hours later I don't think charging is your problem. A duff battery would be the opposite and wouldn't hold the charge.