All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
The Hadeeth of Rifaa'a that you referred to in the question was reported by Al-Bukhari and Muslim from 'Aa'ishah and it is evidence that the intention of the woman to return to her first husband does not affect anything.
Ibn Abdul-Barr said in At-Tamheed:
“The saying of the Prophet to the wife of Rifaa’a, “Do you want to return to Rifaa’a?” is evidence that the intention of the wife to return to her first husband does not affect the one who concluded the marriage contract with her [i.e. the second husband], and her having this intention does not make her included in Tahleel (marrying a second husband after an irrevocable divorce just so that she can return to her first husband), which is a matter that makes the person deserving of the curse (of Allah).” [End of quote]
This is because (Islaamically) divorce is in the hand of the husband and not in her hand, so her intention did not affect (the marriage contract).
Ibn Qudaamah said:
“Ahmad said: the Hadeeth of the Prophet to the wife of Rifaa’a, “Do you want to return to Rifaa’a? and the intention of the wife, is of no effect. The Prophet said: “Allah has cursed the one who marries a divorced woman with the intention of making her lawful for her former husband and cursed the one for whom she is made lawful.” And because the contract becomes void by the intention of the husband, because he is the one in whose hand is the disposal of keeping his wife or divorcing her. As regards the wife, she cannot invalidate the marriage contract, so whether she had the intention or not, is the same.” [End of quote]
On the other hand, this intention from her is not considered as asking divorce, and it is not permissible for her to ask her second husband for divorce simply because she wants to go back to her first husband, and the second husband is not obliged to divorce her for that; rather, the matter is in his hand, he may keep her if he so wishes, or divorce her, if he so wishes.
Hence, the matter is only related to the intention, it has nothing to do with hurting the feelings of the second husband, and it is not permissible for her to agree with him on Tahleel or to condition that on him (in the marriage contract).
Allah knows best.