I like the idea of phone contactless in principle too, but wouldn’t use it exclusively for fear of dropping my phone on the way to the station/in the loo somehow. Wouldn’t want to create a single point of failure for travel/money/communication.
But I guess you’d (usually) have the card itself around too. And an Oyster card, which has auto-top-up from the same account as the card and the phone. Belt, braces and dungarees over the top.