I do know from the frame buffer question I asked Todd when CS6 was released, that AE still has a 2GB internal limit. This still causes me problems on my 6K IMAX frames when working in 16 bit and float.
So I’m guessing that for raytracing it can’t exceed that.
That’s one major architectural change that they need to bite the bullet and fix since it’s a 64-bit app and 2GB is nothing these days, even for a tablet or phone.