from what ive seen with large image sequences (10s of thousands of files), and im guessing you have as well, is that there is some point where AE really starts to choke. not sure its its a wall, or exponential, but i have had luck reducing the number of files in a project and have had massive speed gains vs unopenable projects