Sounds like you would be increasing your non deductible interest by pulling money out of your ppor offset? If thats the case it would be better to leave it there and borrow 100% for investment purposes IMO.
Is it a good time to invest? Its always a good time to invest if you can afford to but...