If the annual loan fee is incurred against a loan which is used to fund an income producing asset, it is 100% deductible when incurred.
It is not considered a borrowing cost to be written off over five years because it is one of those "ongoing" costs attached to keeping your loan going.