I've tried optimizing the MySQL tables as much as possible on the forum stuff, but we've got 5,000 topics and 30,000 posts and that seems to drag it down. I guess one option would be to archive the really old posts to another forum.
Upgading the forum software is on the list of things to do but so far I haven't found a good way to keep the events integrated with it.
However, Apache was just upgraded recently to 2.2, and since then I've noticed the forum homepage load a lot quicker.