Thanks ozsupra. The house is actually currently PPOR, which I should have stated in my original post. This is the reason why I still want access to the shed, and why I put the break in the retaining wall.
The above shows roughly where I was planning on putting the fence (in yellow)...