[FeS2-devel] correctly executed instructions

[Naveen Neelakantam]

Hello Ioana,

First of all, thank you for trying FeS2!  As for the multiprocessor 
version, we hope to get it out there this semester.  It should simply be 
a matter of dedicating time to do it, but well you probably know how 
that goes...

That the x86 instruction stats are inconsistent is strange.  That simply 
should not happen.  *pause*  Ok, I just checked the source, and it was 
possible if a very large number of stores execute incorrectly.  I've 
patched the source and _now_ it should not happen.  ;-)

Note that the following is possible: incorrect + correct < total.  The 
reason is that there are some (hopefully rare) cases where an 
instruction is unimplemented, has what the simulator thinks is an 
invalid opcode, has a memory operand without a valid translation, or 
took an exception or interrupt, .  These instructions events are 
counted, but the first three I listed are counted speculatively (i.e. 
could be caused by wrong path fetch).  That is why the example file 
simpletest_stats_parser.py defines "error rate %" as (1 - 
correctlyExecutedX86Instructions/totalX86Instructions) * 100.

As for the number of incorrectly executed x86 instructions, that number 
_needs_ to be low.  I've done some back of the envelope calculations and 
I estimate the simulation error due to incorrectly executed instructions 
to be 20x the "error rate %" described above (assuming the default OOO 
simulation parameters).  For my experiments I like the error rate be 
less than 0.1%.

Higher error rates do sometimes show up on benchmarks I've never run 
before.  Whenever that happens to me I take a deep breath and prepare 
for a couple of days of painful debugging.  The cause is usually a 
corner case bug in the ptlsim uop decoder.  That said, can you tell me 
more about what you are simulating?  What is the guest O/S?  What is the 
program?

You are also correct in that we only check the integer registers, eflags 
and instruction pointer when validating the simulation.  Instructions 
that use the XMM registers are modeled, but are not validated.  That's 
been something I've been wanting to fix, but to be honest probably never 
will...  x87 instructions however, are not modeled at all (i.e. they 
will show up as unimplementedX86Instructions).  That's not something I 
ever planned on fixing, because x87 is complicated and from my 
perspective obsolete.  For FP benchmarks you should be able to pass a 
flag that tells the compiler to use XMM instructions instead of x87 
instructions (i.e. for gcc use -mfpmath=sse).

Hope that helps!

Naveen

Ioana Burcea wrote:
> Hey there,
>
> First of all, thanks for making the simulator available. We're waiting for the 
> multicore/multiprocessor version ;)
>
> I'm thinking of using FeS2 for my own research, so I downloaded the source code 
> and tried out the simple test it comes with.
>
> The good news is that things seem to have worked. The not so good news is that I 
> have questions already :)
>
> I took a look at the stats generated in *.stats file. And I noticed you keep 
> track of the instructions that execute correctly and the ones that don't, as 
> well as the total. First, these numbers don't add up (i.e., the correct + 
> incorrect > total). Second, the incorrect ones are quite high ( ~ 30% of the 
> total). Is this normal (as in, does this surprise you or you expect this kind of 
> stats/numbers)? Do these numbers go down for a normal simulation, where you have 
> longer/bigger pieces of code/simulation?
>
> I also notice that for validation of instructions, you check only some of the 
> architectural regs (namely, you don't check the fp ones). So, the numbers can be 
> trusted mostly for int-oriented benchmarks.
>
> Could you please tell me more about the "accuracy" of the simulator? Or any 
> weaknesses you're willing to share...
>
> I appreciate your time and effort in answering: thanks,
> 	Ioana
> _______________________________________________
> FeS2-devel mailing list
> FeS2-devel at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/fes2-devel
>