Are you doing this with 3d layers? i guess so since you talk about using a camera.

AE is actually very good at dont rendering layers out of view or fully covered by layers when using 2d layers, but when you use 3d layers this intelligence goes away and it seems like every layer is rendered even if outside of the cameraview. When i encounter AE projects that are very slow to render when using lots and lots of 3d layers i make sure to cut the layers in and out points when they are not needed anymore. 

I had once a 350+ 3d layer comp where i had all of the layers the full length of the timeline even if only about 10 was in the view at one time. This became sooo slow to render, especially with motion blur, that i had to go through the whole timeline setting in and out points on the layers when they went in and out of view.  Even an expression i did to set the opacity to 0 when they where outside didn't help, only when the layers was cut the rendertime went down so i could deliver on time.

Maybe you should try that too.

Robert