Hyperthreading DECREASES performance? There ought to be a ..

On 2009-10-28, General Schvantzkoph <schvantzkoph RemoveThis @yahoo.com> wrote:
> I'm surprised that Intel brought hyperthreading back in the I7, it adds
> complexity without providing much gain.

General, in my test case, running 8 parallel processes is 30% faster
with hyperthreading, than without. The gains are sometimes real.

My problem is that the scheduler that I have working for me now, does
it SUBoptimally when I have fewer processes/threads than logical
CPUs.

> The theory behind hyperthreading is that you can get higher
> efficiency by using a pipe to process two or more unrelated
> instruction streams. Unrelated streams have no dependencies between
> them so you can schedule instructions based on available resources
> without being constrained by the dependencies within a single
> stream. This is helpful if you can't use your pipe efficiently with
> a single stream. The hard problem in instruction scheduling are
> conditional branches which are very common in most code. If you have
> a long pipe and you guess wrong about the direction of a branch then
> you end up throwing away a lot of work. The P4 had a very long pipe,
> 28 stages in the last incarnations of that architecture. A wrongly
> predicted branch would end up costing you a lot of cycles, probably
> not all 28 cycles but close to that. If you were to run two separate
> streams down that pipe then it would look like two 14 stage
> pipelines rather than a single 28 stage pipeline. As a result a
> wrongly predicted branch is half as expensive, so in theory much
> less work would be wasted.


The threads we have, mostly mind their own business, so I would not
expect a lot of interdependencies between them.

They may read shared resources, like parameters and so on, but they
rarely, if ever, write to the same resources.

> The iCore7 has a much shorter pipeline to
> begin with so the cost of a mispredicted branch is less. Also branch
> prediction algorithms have been improved so the iCore7 guesses wrong
> less often. As a result the potential gain from hyperthreading is
> much less. Hyperthreading has a negative side also. Some resources
> are in short supply, in the iCore7 this is principally cache, in the
> P4 registers there weren't enough registers either. If you run more
> instruction streams simultaneously you end up contending for the
> scarce resource.

True. But what can we do, we run as many threads as we need (based on
events to be handled).

> The iCore7 is undercached to begin with. The Core2
> had 6M for two cores while the Core7 has 8M for four cores. If you
> also turn on hyperthreading you are down to 1M per thread. It's
> actually not as simple as that. It's a common cache so every thread
> is contending for the same cache blocks. If the cache was much
> larger this wouldn't be a problem but it's a pretty small cache even
> for four threads let alone eight so it is an issue. Imagine 8 people
> trying to share a two bedroom apartment, they would be throwing each
> others things in the street all the time. My experiments on my
> workload have shown that the different effects mostly cancel each
> other out so I keep hyperthreading enabled but I don't think it
> makes more than a 1 or 2% difference either way.

Based on my tests, I do hope for a 20% improvement, maybe I am wrong.

i

General Schvantzkoph wrote:
> The iCore7 is undercached to begin with. The Core2 had
> 6M for two cores while the Core7 has 8M for four cores. If you also turn
> on hyperthreading you are down to 1M per thread. It's actually not as
> simple as that. It's a common cache so every thread is contending for the
> same cache blocks. If the cache was much larger this wouldn't be a
> problem but it's a pretty small cache even for four threads let alone
> eight so it is an issue. Imagine 8 people trying to share a two bedroom
> apartment, they would be throwing each others things in the street all
> the time. My experiments on my workload have shown that the different
> effects mostly cancel each other out so I keep hyperthreading enabled but
> I don't think it makes more than a 1 or 2% difference either way.

So, for the price difference, would you say the i5 are comparable to the
i7, since they don't do hyperthreading?

On Wed, 28 Oct 2009 16:28:57 -0400, johnny bobby bee wrote:

> General Schvantzkoph wrote:
>> The iCore7 is undercached to begin with. The Core2 had 6M for two cores
>> while the Core7 has 8M for four cores. If you also turn on
>> hyperthreading you are down to 1M per thread. It's actually not as
>> simple as that. It's a common cache so every thread is contending for
>> the same cache blocks. If the cache was much larger this wouldn't be a
>> problem but it's a pretty small cache even for four threads let alone
>> eight so it is an issue. Imagine 8 people trying to share a two bedroom
>> apartment, they would be throwing each others things in the street all
>> the time. My experiments on my workload have shown that the different
>> effects mostly cancel each other out so I keep hyperthreading enabled
>> but I don't think it makes more than a 1 or 2% difference either way.
>
> So, for the price difference, would you say the i5 are comparable to the
> i7, since they don't do hyperthreading?

I prefer the Core2. I have a couple of Core2 boxes and an iCore7 box, the
best performer is the newer Core2 (with a 6M cache). I've done extensive
benchmarking on my workload, Verilog simulation and FPGA place and
routes. What I found was that the Core2 out performs the iCore7 on a
clock for clock basis by about 10% for Verilog simulations (which for me
is vast majority of my workload). The iCore7 outperforms the Core2 on
FPGA builds, clock for clock. However I have been able to run my Core2
system at 4GHz, the fastest I get my iCore7 system to run stably is
3.3GHz. My definition of running is that they are able to run 100% on all
cores simultaneously for days and days without problems, the overclocker
sites define running as being able to boot which is not the same thing at
all. In both cases I'm using a Thermalright Ultra Extreme 120 CPU cooler
which is the current king of the cooling hill. My Core2 is an 8400, my
iCore7 is a 920. The Core2 only has two cores, the iCore7 has four.
However when I run Verilog regressions on all four cores of the i7
simultaneously the performance drops so much, relative to two streams,
that the throughput on the four cores is no greater than the two cores in
the Core2. The problem with the iCore7, as I've mentioned before, is the
cache which is both undersized for four cores and has higher latency than
the cache on the Core2. Verilog simulations are extremely cache sensitive
which is why I'm seeing such terrible results from the 920. The FPGA
tools aren't nearly as sensitive so they aren't hurt by the cache
architecture of the iCore7.

Core2 motherboards cost half as much as iCore7 motherboards and the
processors are also considerably cheaper. If I were building another
system today I'd base it on the 8500 (which is a little faster than the
8400). I'd stick with the Thermalright heatsinks and I'd pick DDR2 1066
RAM which is the speed the memory system will run at when you overclock
the CPU.

General Schvantzkoph wrote:
> If I were building another
> system today I'd base it on the 8500 (which is a little faster than the
> 8400).

Overclocking aside; you'd still prefer the e8500 over any of the core2
quads (excluding the i5/i7 quads of course)?

On Thu, 29 Oct 2009 00:04:13 -0400, johnny bobby bee wrote:

> General Schvantzkoph wrote:
>> If I were building another
>> system today I'd base it on the 8500 (which is a little faster than the
>> 8400).
>
> Overclocking aside; you'd still prefer the e8500 over any of the core2
> quads (excluding the i5/i7 quads of course)?

Absolutely. Even if you don't overclock, the 8500 is a 3.16 GHz processor
at $189. The 920 is a 2.66GHz processor for $289, the 950 is a 3GHz
processor for $569. The motherboard for the Core2 is also a $100 cheaper
then the motherboard for an iCore7. As I stated earlier, clock for clock
the Core2 is about 10% faster then the iCore7 which means that on
individual threads the Core2 is 30% faster than the 920. In theory you
should get more throughput from the 920 because it has four cores but
I've found that isn't the case. Because of the undersized cache the
performance of the 920 on a heavy workload degrades enough that real
throughput is about the same as the two core 8500. The bottom line is
that a Core2 system will cost you $200 less than an iCore7 box and it
will give you better performance. An iCore5 will cost you less than an
iCore7 but it still costs more than a Core2 and it will slower. One final
thing, the Core2 motherboards have two gigabit Ethernet connections, the
iCore7 boards only have one.

If your workload is different then mine you might have different results.

General Schvantzkoph wrote:
> On Thu, 29 Oct 2009 00:04:13 -0400, johnny bobby bee wrote:
>> Overclocking aside; you'd still prefer the e8500 over any of the core2
>> quads (excluding the i5/i7 quads of course)?
>
> Absolutely. Even if you don't overclock, the 8500 is a 3.16 GHz processor
> at $189. The 920 is a 2.66GHz processor for $289, the 950 is a 3GHz

Firstly, I do really appreciate your input on this.

Secondly, I apologize I wasn't more clear, but in the previous post I
wasn't referring to the i5 or i7, I was asking your advice between the
8500 and any of the older model core 2 quads, specifically the 9550.

On Thu, 29 Oct 2009 12:25:03 -0400, johnny bobby bee wrote:

> General Schvantzkoph wrote:
>> On Thu, 29 Oct 2009 00:04:13 -0400, johnny bobby bee wrote:
>>> Overclocking aside; you'd still prefer the e8500 over any of the core2
>>> quads (excluding the i5/i7 quads of course)?
>>
>> Absolutely. Even if you don't overclock, the 8500 is a 3.16 GHz
>> processor at $189. The 920 is a 2.66GHz processor for $289, the 950 is
>> a 3GHz
>
> Firstly, I do really appreciate your input on this.
>
> Secondly, I apologize I wasn't more clear, but in the previous post I
> wasn't referring to the i5 or i7, I was asking your advice between the
> 8500 and any of the older model core 2 quads, specifically the 9550.

I haven't tried any of the older quads. Internally they are a pair of two
core die so you would expect the performance to be identical, however
they share a common FSB which might have some effect on their
performance. I would guess that they would be harder to overclock because
you have twice as much heat to get rid of and you have twice the loading
on the FSB. The other issue is the maximum amount of RAM. In theory you
can put 16G onto a Core2 motherboard but the price of 4G DDR2 DIMMs is
still prohibitive. As a practical matter 8G is the limit. If you really
need four cores I would build two separate dual core boxes, that way you
can have twice as much of everything.

General Schvantzkoph wrote:
> I haven't tried any of the older quads. Internally they are a pair of two
> core die so you would expect the performance to be identical, however
> they share a common FSB which might have some effect on their
> performance. I would guess that they would be harder to overclock because
> you have twice as much heat to get rid of and you have twice the loading
> on the FSB. The other issue is the maximum amount of RAM. In theory you
> can put 16G onto a Core2 motherboard but the price of 4G DDR2 DIMMs is
> still prohibitive. As a practical matter 8G is the limit. If you really
> need four cores I would build two separate dual core boxes, that way you
> can have twice as much of everything.

I really do appreciate all the input.
cheers.

On Thu, 29 Oct 2009 13:44:19 -0400, johnny bobby bee wrote:

> General Schvantzkoph wrote:
>> I haven't tried any of the older quads. Internally they are a pair of
>> two core die so you would expect the performance to be identical,
>> however they share a common FSB which might have some effect on their
>> performance. I would guess that they would be harder to overclock
>> because you have twice as much heat to get rid of and you have twice
>> the loading on the FSB. The other issue is the maximum amount of RAM.
>> In theory you can put 16G onto a Core2 motherboard but the price of 4G
>> DDR2 DIMMs is still prohibitive. As a practical matter 8G is the limit.
>> If you really need four cores I would build two separate dual core
>> boxes, that way you can have twice as much of everything.
>
> I really do appreciate all the input. cheers.

One more point. Multiple cores are only helpful if you are running a
server workload, anything more than two in a desktop system are useless.
In fact the second core in a desktop system is only marginally useful.
What is useful is single thread performance because most programs are
still single threaded. That's why I'm advocating overclocked Core2s
rather than any quad core system. The Core2 has a lot of headroom in it,
if you combine it with a really good cooler and a good motherboard it can
easily run at 4GHz or more. I used the Thermalright Ultra 120 Extreme
cooler and a Gigabyte motherboard. You should also pick RAM that runs at
1066 or better, you don't want to have to overclock your RAM so you will
want to chose RAM that is within specs when you overclock the CPU. This
is the motherboard I used,

http://www.newegg.com/Product/Product.aspx?Item=N82E16813128358

In comp.os.linux.misc General Schvantzkoph <schvantzkoph RemoveThis @yahoo.com> wrote:
> On Thu, 29 Oct 2009 13:44:19 -0400, johnny bobby bee wrote:
>
> One more point. Multiple cores are only helpful if you are running a
> server workload, anything more than two in a desktop system are useless.
> In fact the second core in a desktop system is only marginally useful.

I haven't had a single CPU workstation in at least 15 years of running
HPUX, Windows and Linux. Every time I sit down at someone's single CPU
machine it feels like it is crawling along.

This is due to the high amount of multi-tasking that I do and the
applications I runa -- and it leads me to totally disagree with
your blanket statement.

Multi CPU and or multi-core machines are very noticebly
zippier for many applications and workloads.

Kind of a moot point though, since it is getting very hard
to even find a new single-core machine.

Stan

On Thu, 29 Oct 2009 22:21:15 +0000, Stan Bischof wrote:

> In comp.os.linux.misc General Schvantzkoph <schvantzkoph.TakeThisOut@yahoo.com>
> wrote:
>> On Thu, 29 Oct 2009 13:44:19 -0400, johnny bobby bee wrote:
>>
>> One more point. Multiple cores are only helpful if you are running a
>> server workload, anything more than two in a desktop system are
>> useless. In fact the second core in a desktop system is only marginally
>> useful.
>
> I haven't had a single CPU workstation in at least 15 years of running
> HPUX, Windows and Linux. Every time I sit down at someone's single CPU
> machine it feels like it is crawling along.
>
> This is due to the high amount of multi-tasking that I do and the
> applications I runa -- and it leads me to totally disagree with your
> blanket statement.
>
> Multi CPU and or multi-core machines are very noticebly zippier for many
> applications and workloads.
>
> Kind of a moot point though, since it is getting very hard to even find
> a new single-core machine.
>
> Stan

Depends on your workload. You will notice that I used the term desktop
not workstation. I define a desktop as something you use for Internet
access, e-mail, word processing and as an X-server. I define a server as
a system that dedicated to a class of tasks, some compute intensive some
not, and that you access over a network, not something that you are
sitting in front of. A workstation is a system that is a hybrid of the
two, you use it for both interactive tasks and for heavy duty computing.
I agree with you that multicores are a requirement for workstations.
However I think workstations are a suboptimal way to work.

*nix is client/server oriented so it's very easy to utilize multiple
machines. It's one of the things that makes *nix so much more versatile
then Windows. I don't like the workstation model, I find it much more
limiting then the client/server model. I utilize a collection of machines
of various vintages to do my work. The newest is an iCore7 with 12G of
RAM, the oldest is a 500MHz PIII with 512M of RAM. I use an old 2.2GHz
first generation A64X2 for my desktop machine, it's performance in this
role is as snappy as it needs to be. I also use my laptop, (Core2 with
4G) as a desktop like system. I have a pair of 8G Core2s, 65nm 3GHz/4M
cache, 45nm 4GHz/6M cache, and an 12G iCore 7 that I use for compute
intensive applications. I also have am old single core A64 that I use as
a CVS server and a really old PIII that I use as an SSH server (so that
clients can upload and download things to me). My servers all run at init
3 so they don't have to waste any memory on the X-Server or a desktop.
I'm also running VMs on my servers which are dedicated to different
functions.

I do all my work by sshing into my servers (or VMs, they all look the
same to me) and running XEmacs over an X-tunnel, I use Gnome's multiple
virtual desktop feature to organize the X-Windows that belong to
different systems. If I need to run a GUI application like a wave browser
I just run that over an X tunnel also. On a LAN the performance of a
remote X-Window is indistinguishable from running it locally.

The great thing about Linux is that there is no limit to how much you can
scale your computing environment. You aren't forced to rely on a single
box for all of your needs which gives you a lot of flexibility. My
feeling is that if you need four cores worth of computing power today,
you are better off getting a pair of Core2 boxes rather then a single
iCore7. The only place where the iCore7 is better is if you need more
than 8G of RAM for a single job (I've done FPGA place and routes that
needed 10G). Another reason would be if you were severely space
constrained.

General Schvantzkoph wrote:
> http://www.newegg.com/Product/Product.aspx?Item=N82E16813128358

I think you've managed to change my mind. More so, than all the research
I've been doing so far.
Thanks, again.