The three sets was my original guess the first time around a long time ago but as I read then what Galland said:

From Galland´s " Die ersten und die letzten"

" On 28 January 1942 I was awarded as the second soldier in Wehrmacht the Brilliants to my Knight´s cross. I was happily surprised. Mölders had his after his 100th air victory, and I was still missing six. Hitler thought that my new position should not give me the expression that it had stopped me from getting the highest possible honour in the German army. Besides my 94 victories were all from the Western front so they were thought as more valuable.

My first brilliants I received in the usual ceremony by Hitler. A bit later I was travelling with Goering in his special train as he wanted to see my brilliants. " Are those the brilliants that Fuhrer gave you? Let me see!"
"No" he said. "These are not brilliants! These are pieces of glassa! Totally useless! The Fuhrer has been fooled by these. He knows on tanks and planes but nothing on brilliants. I have a little stock myself, you´ll be surprised, Galland! "
Later on I was asked to Karinhall, where Goering was happy like a child. Goering took the Fuhrer´s brilliants to his other, then his brilliants to his other hand and asked me to comapre: "Look-notice the difference? Who is the one who understands on brilliants?" I must say the Goering´s brilliants were absolutely fabulous. They were big and shone beautifully. Compared to these the Fuhrer´s brilliants were nothing.So now I had two sets of brilliants.
Goering also mentione dthat he´s going to talk with Fuhrer about this matter.
I guess he did because soon I was asked to Wolfsschanze. In the presence of Milch, Keitel and Jeschonnek Fuhrer told me " Galland, you are about to receive the highest honours in the German military, but so far you only had the temporary brilliants with you..."
I was truly schocked and frightened, because I had taken Goering´s brilliants to this occasion. Goering had not told Hitler about this. The situation became even worse as Hitler wanted to have the brilliants I had on.he must have noticed they were not the brilliants he had given, but he kept it in his hand and said:" Well, see the difference! These are pure glass while these are true brilliants!" Anyhow the Goering´s brilliants were much bigger and looked better-Fuhrer had no idea of what brilliants looked like?? But now I had the brilliants in three sets.

As the war was coming towards its end, my office in Berlin was bombed and I lost my brilliants. As Hitler heard of this, he gave me a new set of brilliants-the fourth ones."

As you were closest, Skipper, with the guess three times, I am willing to give you the position of the quiz leader!

__________________