| hey tony, (and sent to the AE list for benefit of others wondering)
if you go with mac mini's try to get a few spare external power supplies from apple parts to have on hand as if they burn out / go dead - that would be something to replace right away with not waiting on replacement swap times. (also check out the prices and how much they may cost before you decide. one reason i never looked at Dell was they used proprietary power supply connectors on their motherboards - that means only their PS can be used and at their whim for price and supply of them)
as for heat - i don't know how much cooler mac mini's could be then a normal PC heat of i7 - heat is heat - it has to go somewhere...once i had the main AC go out a couple years ago and my studio with the farm it it went over 105 degrees in less then 5 minutes (from high 60-low 70's) - i've actually talked to my wife about moving to alaska or somewhere cold where all the heat could be of use- like i could exchange heating the office building for my rent or something.. would never have to pay to heat the studio in a cold climate - where in tropical florida i'm paying year round to cool instead. wife says NO to snow, so stuck for now it seems. : (
network gear - yeah they get hot too with heavy use and use quality ones as the power supplies and network routers themselves will melt/fry. been there done that when starting up 10 years ago and now only metal pro grade network stuff...
i've thought about using the mini's too time to time - but just afraid of how 24/7 would handle it... i have friends with web hosting that use them but they do not sustain 100% processor usage for days on end - web hosting is not anywhere as demanding as rendering - so they get hot - not sure if they can handle it (or if the power supplies can)- i once tried some "green" PC power supplies - more efficient types and they all were dead in 60 days - most died in the first 30 days - as the power saving components were not meant to be 24/7 it seems - i won't make that mistake ever again.
commercial grade power lines - yeah - many years ago when i was first starting out and tripping circuits under heavy draw (long past those days) i had to have heavy commercial grade lines put in - expensive and you will need more then you think. check the AMP load of one render node at 100% (they sell meters and stuff for that) now look at what the line can pull - multiply and get out the checkbook. years ago the electrician showed me what burned power lines in the wall look like - it's scary or was to me... the insulation was dried and flakey, brown/black like the outside of a burned marshmallow. watch those power limits per line. i routinely test my power pull on each commercial grade line now and have a patch cut out of the wall where i can touch the wires inside the wall run to see how hot they feel.
don't underestimate the power bills... summer has my electric bills for the studio now at $1600 per month - yeah... ouch. my electric bill comes in a special gold envelope. (kidding, but it should) the coming EPA regulations are going to shut down a lot of coal electrical plants - so with no new nuclear or other plants taking up demand - i don't know what is going to happen in the next couple years for electric rates... solar and wind aren't going to add anything to 24/7 demand needs. this year along i have seen a 25% increase and it gets worse as you are a high "user" of electric you keep going over the limits they have in tiers and into higher and higher KW rates. i.e. first 1000KW is one price, then next 1000KW is priced higher etc, then the next etc... i've had heavy use months go 50% above what the bill was last year. and the real big increases in power costs aren't even here yet. (never did i think politics and EPA regulations would affect my business but it is now)
UPS's are a must - and i just spent $6,000 on upgrades to mine (again)- need another $4,000K to finish the upgrade still... trust me the cheap ones won't last, the prosumer ones last a bit longer, but replacing batteries is expensive and the consumer types don't seem to handle it - i spent $4k last year on some "prosumer" ones to try them out and 1 year later 80% of them were dead... another waste of money... the APC 1500 ones (tall consumer/professional workstation types around $200) seem to last one to two years, but that's it and replacing batteries are just about as much as buying new ones.
just have to bite the bullet and go with the commercial grade and that is the expensive ones if you want them to last. usually i have to replace them every two years - so that money adds up too. the ones i get now are $600 each... hope to get two or maybe fingers crossed three solid years from them... but we'll see.
the only other reason to be careful with mac's is the software - lots of the c4d plugins (and other plugins are PC only) for your own studio this should be ok i suppose as if you are using macs you should have matching plugs for a mac farm ok.
good luck, let me know if you have any questions,
dann
___________________________________________________________________
___________________________________________________________________
On Nov 12, 2012, at 3:19 PM, Tony Romain wrote: I think where I'm at is trying to figure out what the main limiting factor is here versus building something from the ground up, piece by piece (probably PC based for cost).
Do you have a sense of specifically what the networking issues were? For cinema rendering, my understanding is that once the net render files have been distributed amongst the nodes, their really isn't a ton of network traffic, except for the rendered frames being copied back over to the server. However, I may be wrong on this… I don't know a ton of what goes on under the hood of these farms…
Have been exchanging emails with the owner of this company:
to see if he's heard any anecdotal info from any of his clients. He hand't heard anything specific other than describing one server set-up he'd heard of that utilized 160 mac minis for some type of test automation (not sure what that is), but they were all being taxed 100% of the time and seemed to run fine…
-- tony romain | principal/creative director
trance motion graphic animation and design 323 651 1114
My main concern would be the networking back end. Unless you have an applescript wizard on hand you may run into some problems, even with a networked rendering package like deadline. We tried to do something like this at a place I was working a few years ago and it was rage-inducing. The frustrating thing was "it should have worked" but it just didn't. There were always problems. maybe if you installed win 7 pro in bootcamp on all of them. but I guess that would defeat the purpose. Things have probably come along since then however
|