The seeds will grow in to birches, no issues, but harvest those trees and leave the stumps and the saplings from the stumps give the error "need more room to grow"
On further investigation, the stump itself is causing the issue - it's seen as too close to the sapling; I cut down a single tree on the shoreline and left the stump - no other trees for a good distance away - and the sapling still showed "Needs more room to grow"