So by “unreleased” I guess we are talking about songs which only exist in demo form, or which have only ever been performed live but never properly recorded by the artist for an album.
Sufjan Stevens performed an exceptional song called “Majesty Snowbird” around the time of his 2008/2009 gigs, but it remains unrecorded to this day. Sounds like possibly his best song to me 
Yeah this is always one of my examples, and agree it’s his best song.
Really dig this;
Think a demo of this was put out but nothing more;
Technically released but not properly;
This is wonderful
Was always a bit disappointed they never released Sandtrap
That other one something Terror Pocket not so much.
Gorgeous. Was that a Grace demo?
It’s a gorgeous song but always felt quite weird listening to it. Seem to remember Fraser telling off a journalist for mentioning it.
There are a handful of 2006-2010 Broadcast demos which were never fully recorded, for the saddest possible reason.
This is possibly the pick of them. Love that James is so generous to share these little snippets once a year on Trish’s birthday.
In my head it’s post Grace but I don’t know why I think that, sorry I’m not sure
The best song they ever did
Just googled it, sounds like Fraser was unhappy with it seeing the light of day because she considered it to be an unfinished song.
Had no idea they had been in a relationship.
I know what you mean, it’s unfinished and from what I’ve seen Liz never wanted it released. Just looked up her 2009 Guardian interview which is where she makes her feelings very clear about that but does say maybe she won’t always feel the same. It’s genuinely astounding, not saying that makes it ok it’s out there, but it is.
I love Felt Mountain era Goldfrapp and there’s a few unreleased songs from that era that exist only as live recordings but are cut from the same cloth as the album
Scandalous that this never got recorded properly
This one at least got the Peel Session treatment
The Mountain Goats have loads of good ones:
Probably the best known:
First thing that springs to mind. Only found on their first live album, Last Night I Saved Her From Vampires recorded at the Water Rats. @guntrip, various ex-Disers and myself are on this.
Not that you can hear us, obviously but we were at the gig.
They ended every show with this for a year or so as well. Great song.
Yep, very enjoyable chaos. Building a tower out of the drum kit whilst Jack’s still playing! 
i was just grooving to this when the postman turned up with my newly repurchased Prince The Hits/B-sides 3-discer!
I’ve written a bunch. They’re fucking excellent.