I was looking at train options from DC and Baltimore and it looks like there are only bus options. Google has you taking a metro or train from DC to New Carrollton and transferring to a bus. Similar options from Baltimore. Takes nearly two hours from DC. So if you don’t want to rent a car there are options, although not great ones.
If your grandson has never been to the area, I would recommend staying in DC to see the sights and then just going to Annapolis for game day. It’s a very nice town but only a few hours needed to see it. I can recommend the Maryland State House, which briefly served as the US capital. If you are just popping in for the game, then staying in/around Annapolis would be fine and you can tour the Naval Academy as well.