SCCT – Hardware Speed Test 2015

Hello Chess Friends,

After some hot discussions...!)
I've decided to run a new hardware speed test !

And here are the results:

1st Hardware Speed Test

Conditions:
Intel Core i7-980X
Hyper-Threading OFF
Large Pages OFF
Arena 2.01 Chess GUI
Stockfish 290815 x64 modern
3 Seconds per move
128 MB hashtable size
600 Tactical positions
kN/s values are based on WSET 2015's bench position



Note: I tested Stockfish 290815 x64 with HT ON too:
388 points, 13898 kN/s, i7-980X 3.33GHz, 12 threads

----------------------------------------------------

2nd Hardware Speed Test

Conditions:
Intel Core i5-3210M 2.50GHz
Hyper-Threading ON
Large Pages OFF
Arena 2.01 Chess GUI
Stockfish 290815 x64 modern
3 Seconds per move
128 MB hashtable size
600 Tactical positions
kN/s values are based on WSET 2015's bench position




--------------------------------------------------------

Details,
This hardware speed testing is different than all, never played before !)
The goal of this test is that: the kN/s values are quite useful for measuring...
Note also Stockfish 290815 breakes the formula rules under the current conditions !)
As we see, Stockfish 2.01GHz 2 core performed better than Stockfish 4.02GHz 1 core

More Details,
Some experts have different views, but I hope after checking the current results:
- They will change their opinions... because we are in 2015 !)

For any questions about the current test, please feel free to ask...

Best,
Sedat

Dear Chess Friends,

Very soon I plan to run another speed testing !

In shortly,
I will test Stockfish 290815, 1 core, with same 600 positions, on i7-980X 3.33GHz
But this time, I will set Time Control: 11 seconds per move

Really I wonder a lot about what will be the results...

And what is your estimation ?)
SF 1 core 11 sec/move will be performed stronger than SF 4 core 3 sec/move ?)

Note: SF 4 core 3 sec/move i7 980X 3.33 GHz managed to get: 374 points

Have fun,
Sedat

The speed hardware test has been completed, so here is the final result:
Stockfish 290815 x64 1 core 11 sec/move i7 980X 3.33 GHz: 395 points

What I can say more,
This time, SF 1 core 11 sec/move performed clearly better than SF 4 core 3 sec/move
For example, SF 1 core 11 sec/move's performance is very close to SF 6 core 3 sec/move


Best,
Sedat

Next Speed Test:
I plan to test Stockfish 160715 with same 132 positions, on i7-980X 3.33GHz
But this time I will set 6 seconds per move + 6 cores

Note: Stockfish 160715 1 core, 30 seconds/move performed: 46/132
[You must be registered and logged in to see this link.]

Of course this time, I am not going to ask any questions, estimations...
Because it's a new way of testing...and it is very hard to guess...

But however,
Personally I expect to see similar points...maybe 6 core 6 sec/move will be slightly weaker...let's see

Best,
Sedat

The current test is completed

And here are the results:

Conditions:
Intel Core i7-980X 3.33GHz
Hyper-Threading OFF
Large Pages OFF
Arena 2.01 Chess GUI
512 MB hashtable size
132 Tactical positions

Houdini 4 Tactical x64 6 cores, 6 seconds/move performed: 83/132
Houdini 4 Default x64 1 core, 30 seconds/move performed: 52/132
Houdini 4 Default x64 6 cores, 6 seconds/move performed: 46/132

Stockfish 160715 x64 1 core, 30 seconds/move performed: 46/132
Stockfish 160715 x64 6 cores, 6 seconds/move performed: 44/132

More Details,
Used unsolved positions in 3 sec/move 1 core by the current chess engine versions
Note that (for better comparing) I decided to test Houdini 4 6 core 6 sec/move too
As I expected, SF 6 cores 6sec/move performed very close to 1 core 30 sec/move

Note also that,
I tested SF 6c 6sec/move with 128 MB hash too: 43/132
Also SF 6c 6sec/move tested with 1024 MB hash: 42/132

What I can say more,
We can not use any right 'formula' in case of comparing the hardware speeds !
It all depends on our used conditions...some engines perform better some worse...

But however (in case of measuring the hardware speeds)
Stockfish 290815's KN/s values seems to be not bad choice...
Of course, one of the best choices is that: running similar test suite positions
Or running engine matches in Auto232 mode or via LAN etc...

Hopes helps,
Sedat

Here is another new speed test !

Rybka 4.1 x64 1 core, 30 seconds/move performed: 26/132
Rybka 4.1 x64 6 cores, 6 seconds/move performed: 21/132

Crafty 24.1 x64 6 cores, 6 seconds/move performed: 6/128
Crafty 24.1 x64 1 core, 30 seconds/move performed: 5/128

More Details:
The current chess engines are tested with unsolved positions in 3sec/move 1 core
Crafty 1 core 3 sec/move managed to solve 4 positions, thats why the number is 128
Well, Rybka 4.1 6 core 6sec/move's performance is weaker than 1 core 30sec/move
But... Crafty 24.1 6 core 6sec/move performed stronger than 1 core 30sec/move !

I noticed also that (during all the time of analyzing),
Crafty 24.1's 6 cores CPU usage was at approx. 66%...not sure why...
If Crafty was with 100% CPU usage...probably then its points would be higher...

Best,
Sedat

Sf??

Admin wrote:

Sf??

Stockfish 160715 x64 modern

One new test more !)

Rybka 4.1 x64 1 core, i7 980X @4.02GHz, 222 kN/s, 6 sec/move: 9/132
Rybka 4.1 x64 2 core, i7 980X @2.04GHz, 215 kN/s, 6 sec/move: 7/132

Crafty 24.1 x64 2 core, i7 980X @2.04GHz, 5142 kN/s, 6 sec/move: 4/128
Crafty 24.1 x64 1 core, i7 980X @4.02GHz, 5124 kN/s, 6 sec/move: 1/128

Some Notes:
Used unsolved positions in 3 seconds/move, 1 core on i7 980X 3.33GHz
Exception Stockfish and Houdini, Crafty surpassed the formula rules too !)
As we see, the above mentioned engines did not follow the old formula ^0.76
It looks like, Rybka is more near to this old formula, but again in case of comparing the speeds:
- We can not use the strict number ^0.76
or it does not mean if same kN/s values, the playing strength should be same !
But however, the kN/s values are more useful for majority of the chess programs !

Continuing...
For example, I can say the same thing in case of doubling the processor speeds,
It will be wrong, if we use any strict formula ...I mean some engines gain 50 Elo, some 60 Elo etc...
Also it is similar issue, in case of increasing the time controls on same hardware...
For example, as far as I remember in one of my testings:
Rybka 30m+1s started to be stronger over Stockfish 1m+1s

In other words,
As I mentioned before, it all depends on our used conditions
For example here are some main factors, which are playing a very BIG role:
Hardware speeds, engine versions, opening books, time controls, etc..

Greetings,
Sedat