Since the failure of solutions should not surprise the CEO, shouldn't he be unfamiliar with success?
In this sentence, we have this CEO who has implemented a series of impractical business solutions. That these solutions are "impractical" indicates that they are probably not going to be very successful. As such, no one would be surprised if these impractical solutions did not work—"pan out" fits perfectly in the second blank.
For the first blank, our clue is "unjustifiably." The CEO is so "unjustifiably _______ success." We know that the failure of his impractical solutions did not surprise anyone. Since no one was surprised, this means that everyone expected the CEO to fail. So, the CEO is UNJUSTIFIABLY assured of success—he thinks he is great, however this is unjustifiable because his actions and results do not prove his greatness. The reason your interpretation does not work is because of the word "unjustifiably." While he is indeed assured of success, this does not make him successful, as it is "unjustifiable."
Unjustifiable means "impossible to excuse, pardon, or justify." For example, look at this sentence: "Considering his huge student loans, the expensive watch he bought was an unjustifiable expense."
Is being 'unfamiliar with' success something that can be impossible to excuse or justify? No. On the other hand, being 'assured of' success is something that can be impossible to excuse or justify. For instance, if there is no reason to think the company will be successful because it is "floundering," then making impractical decisions based on a false sense of success is extremely 'unjustifiable!'