On this page you can read recommended steps of prototyping.
It's a copy from a forum discussion related to use WPF for GUI creation.
First I would design (on paper!) my GUI and all layout to get a feeling for my requirements. If the GUI mostly consist of buttons, labels and some few fields to enter text, I would go for something that it lightweight. In most games I developed there were only a few widget types that I really needed, so for a list like "buttons, labels, windows, listboxes, textboxes and images" a library like MyGUI would be a good candidate.
But let's assume that I need much more and WPF would be my preferred choice, the next step would be a small prototype that would render a simple 3D scene in the background and uses WPF to draw a GUI on top of that. My next goal would be to determine the performance impact of WPF, because I really like to be able to maintain a constant framerate in the game. It will just "feel" better because it will make it easier to have smooth animations in the game (I discussed this in detail in another thread in this forum).
OK, now let's say this also went well, the performance is satisfying and there were no major technical problems implementing a WPF GUI that works well with Mogre. Now the next important thing is event handling in the game. In my experience you always need to be able to route events like keypresses and mouse input to many different modules in your game. And you absolute need to make sure that events are only consumed by the right objects.
What I mean by this is, nothing is worse than clicking through the GUI and and affecting other objects that are hidden under a window. I had this ugly bug some time ago in a project where I was not able to detect this easily, so it was a pain in the a** to add all this event handling stuff that I needed to a GUI system that did not provide this feature and was written by somebody else who was not available anymore.
Once this is settled, there would be one last thing that I would test before making my final decision. Resource management. There needs to be an easy way to use game assets with the GUI system without too much pain. In many cases you use assets in different places and don't want to duplicate them for different systems. For example, let's assume you have some icons that are used by game objects and the GUI. You usually want this to "just work", so if you load a resource, like an icon, with the resource manager and can't just assign it to a GUI widget, like let's say a static image, without either converting it to another format on the fly or loading a second instance of the icon with the GUI system, it's also a drawback that can make thing more complicated than they need to be.
I just remember a good example in another project, a prototype that I was working on, where we were using Mogre and I also hooked some of my own code into the render loop that enables me to do some custom rendering on the D3D device. The idea was to do 2D rendering with our own graphics library and use Mogre for the 3D stuff. This worked pretty well, but when we needed to use the same assets that my 2D library uses with the Mogre 3D objects the trouble started. Suddenly I had to write a lot of code to manage all these assets in a way that both libraries could use them at the same time, I would have had much less trouble by just using the billboard system or the overlay system in the first place and stay away from using my own 2D library.