Bergamo is a jewel, especially the upper city. I urge you to read all about it.
I have never been appealed by Milan but that's a personal opinion. Personally, I'd stay in Bergamo for a couple of days and pop down to Milan for a day's sightseeing -though, frankly, I don't think there are that many sights that would excite you.
And, although Laramie's right about Bergamo hotel prices -especially weekends when they can be quite full- Milan's ain't that cheap either.
Laramie's also right about the drag of arriving late - Ryanair arrives at different times of the eveing depending on the day. But even at the earliest time of 20.00 it could mean that you don't get to your hotel until 22.00. The coach transfer from the airport to Milan centre(station) is 60 mins.