Lawyer Virginia La Torre Jeker in Dubai has a fascinating Two Part (and a coming Part Three featuring John Richardson) article Living in the Past: Citizenship Relinquishments: Am I Still A U.S. Tax Citizen?
She addresses the issue many of us are experiencing. We were told decades ago we were “permanently and irrevocably” relinquishing U.S. citizenship when we became citizens of other countries.
Part 1 addresses One Country, Two Citizenships.
She makes it clear reading this is not for the faint of heart. She says:

It may come as a surprise to learn that the United States of America, while one country, has two forms of citizenship. One form reaps many benefits; while the other form bears only fiscal burdens. One country – Two citizenships? Yes – it is true.

She also makes writes about the frightful trip back to the future many of us are experiencing as to whether we are or are not “tax citizens” of a foreign government.
While Part I deals with ways people can relinquish U.S. Citizenship for Immigration and Nationality purposes, Part 2 focuses on U.S. TAX citizens.
It’s complex but thorough. Stay tuned. I will post Part 3 when it is available.
Unfortunately, she does not cover the issue of those whoibecame citizens of other countries as minors or the issue of Accidental Americans.
UPDATE March 24, 2015 Virgina La Torre Jeker responded to a comment I made on this article. She pointed out she wrote an article last month about minors and about mental incapacity issues. Unfortunately, it confirms the nightmare the title of her article Help! I Want to Expatriate but they won`t let me.
In her response to my comment to her, on the subject of minors, she responded:

Another very difficult topic. While I am not an immigration lawyer, I understand that parents can apply for their minor children to become naturalized US citizens. From what I understand – No consent or understanding is required of the child in order for them to be naturalized as citizens. The parent can do all of this for the minor child. Again, this is my understanding & if it is incorrect, I would appreciate the relevant information. On the other hand, a parent cannot renounce US citizenship for his or her minor child. Only the child can do it and of course, not until he has reached a certain age and level of maturity so that the Dept of State is satisfied he has the mental capacity to know what he is doing when renouncing. I found this disparate treatment of the child’s rights and the parents’ rights with respect to the child’s US citizenship to be rather uneven and quite interesting.

Updated March 30, 2014 Part 3
was posted today.
It contains a lot of legal and technical information, including input from John Richardson. Ms. La Torre Jeker seems to concur with the position that those who relinquished U.S. citizenship prior to 2008 or 2004 are not U.S. “tax citizens.”
However, as John notes, many of the compliance fanatics are trying to turn their advice into a self-fulfilling prophecy.
My non-legal advice is don’t fall into the self-fulfilling prophecy trap!