Also dont forget the basic rule that the larger the hole the slower your drill speed needs to be.
Some materials sush as stainless steel or Ti have additional requirements with things like feed rate/cutting speed and cooling/lubrication.