Multi-Threading and the future of CPU's

[mirhagk]

As the Ivy bridge processors near being released, I think it's time to look ahead at where the future of processors are going. I remember hearing like 5 years ago that processors would near the limit, but it seems like Ivy bridge are still significantly better then the sandy bridge ones.

Processors with many cores may not need to be done soon, but we still have quad cores with hyperthreading being pretty widespread, so there are still 8 threads on many machines.

My question is: Do regular programs and games take advantage of this? I don't think everyday programs and games really use multi-threads, other than for network and/or disk threads.

[Tony]

for the most part, no, regular programs don't take advantage of multiple cores. Reason being is that parallel programming is hard to do right.

Some languages/libraries/frameworks/tools make the job easier. Professional games are likely to use engines that take advantage of the resources available to them.

There is also limited advantage to performance, in a way that there are only certain parts that are easily made to run in parallel. Here's an example: in a game, while different NPCs could run in parallel, their interactions with the player must occur in a single thread. If not, then to hit a player, each NPC will make its own copy of player's HP and hit their own copy. If the events occur close enough in time, one of the hits will be lost when two different HP values are written back into the player.

There are such bottlenecks around each shared variable. This is actually not that difficult to write with the use of monitors ( http://en.wikipedia.org/wiki/Monitor_(synchronization) ), but they turn parallel code into concurrent code, and funnel critical sections into a single thread.

Quote:

That is, at each point in time, at most one thread may be executing any of its methods. This mutual exclusion greatly simplifies reasoning about the implementation of monitors compared to reasoning about parallel code that updates a data structure.

Edit: of course as long as there are more processes than CPUs, then each additional core gives more runtime to those individual processes (or one could run more processes on the same hardware -- more servers, more concurrent connections, etc.)

[DemonWasp]

Depending on what you mean by "take advantage of", Tony is correct. If you mean "are written to use multiple threads", then nearly every program, especially GUI programs, are written that way. For example, on my work computer (Windows 7), there are only two unique processes that operate single-threaded: cmd.exe and notepad.exe .

Everything else (gvim, Eclipse, Chrome, Firefox, Outlook, ...) registers multiple threads.

Often, the split in threads isn't to improve processing throughput, but because of logical separation of threads, or for user-responsiveness reasons. For example, gvim probably has a thread for its UI elements, another for manually paging the backing file in and out, and another for searches and so on (complete guesswork, there). Others, like Firefox and Chrome, use extra threads to render multiple things simultaneously, asynchronously communicate with servers, etc.

Server processes, such as those for databases and web servers, are nearly always multithreaded for performance gains.

[mirhagk]

Well I know GUI usually runs in a separate thread so it remains responsive, and networking and saving and things usually run in parrallel.

But these are all things that were around with single core processors, and the reasons they are multi-threaded is to prevent blocking, not to increase speed. From what I understand multi-threading is very rarely done for speed's sake but to prevent blocking.

So do you guys think number of cores will continue to increase, and will it improve performance by much?

[Tony]

mirhagk @ Mon Jan 09, 2012 3:20 pm wrote:

So do you guys think number of cores will continue to increase, and will it improve performance by much?

Well something has got to increase in hardware, and it sure will not be clock rates. Pentium 4s were hitting up to 3.8 Ghz 12 years ago (which is slightly more cycles than Ivy Bridge's "normal" (though "Turbo" mode goes faster) clock rate for the top "Performance" grade chip).

Does your current computer perform better than one of 12 years ago? Undoubtedly. So something is responsible for better performance. Of course clock rates are a poor approximation for actual performance (e.g. RISC vs. CISC). Still, but some measure chips have not been getting any "faster" for the last decade. They _were_ getting smarter though, and we now have _more_ of them (more cores).

A lot of computation tasks can benefit from multiple cores -- faster spreadsheet updates, faster searches for text in documents. Of course this also requires much more complex code. Your procedural Turing code will continue to happily chug along as it did 10 years ago... it's just that now you can run 4 instances of that slow program, and they will not compete with each other for a shared CPU resource.

[mirhagk]

Yeah, but the biggest benefits of multiple processors with current software is the fact that you can run more programs, not so that those programs can run faster.

[Tony]

If a system ran only a single process and nothing else then you would be right -- that process will not get any additional benefit from an extra core.

On a real system though, there are typically many running processes. A CPU-bound* process would indirectly benefit in performance as it can exhaust 100% of the core it's running on, and it will not be interrupted as often, as other processes can also be scheduled on a different core.

Less interruptions == more cycles per unit of time == faster performance from user's perspective.

* This assumes a CPU-bound process -- something that can use up all of those cycles without blocking itself.

[mirhagk]

Yes I'm not arguing that Tony, most machines have many processes, which is why a quad-core HT is MUCH faster than a single core processor, even with identical other specs.

But what I'm saying is adding more cores generally just makes you able to run more programs at 100% speed, it does not make any of those programs faster unless multiple programs are on the same core.

I consider myself a HEAVY user of computers, yet I still only 118 processes open. So what happens once computers get to a point where there are in fact 128 cores? Those extra 10 cores would be completely useless

I don't see many people actually taking advantage of multi-threading, and while more frameworks are coming out to support it and make it easier, people tend to avoid them because it's still more complicated then simple linear programming.

It would suck to see a future where the majority of a user's resources go unused, yet we see this today still (360 has 6 threads yet a lot of games [mostly indie] only use a single one)

[Tony]

Yup, spot on.

Worlds first non-embedded dual-core processor is apparently IBM's POWER4 http://en.wikipedia.org/wiki/POWER4 (though no citation), released in 2001. Intel introduced hyper-threading in 2002, it would be a few years before multi-core started to become popular in consumer products.

This technology is incredibly new. When most of today's senior engineers have gone to school, there were no multi-core chips. Even for me, that second core was mentioned only in a single optional CS class (CS343 -- one of the best CS classes I've taken). I know that people avoid taking that class at all, because it actually is really challenging.

Multi-core is not a magic hardware solution that will make things faster on its own, the way faster clocks used to do. It's really up to you and me to figure out this new frontier on our own, and it's not even taught much (if at all) in schools yet...

[DemonWasp]

You may only have 118 processes, but you probably have a few hundred threads. My Ubuntu laptop (what I'm on at the moment) is running ~300 threads, and could therefore (hypothetically) scale to 300 CPUs. Your point does still stand, though, and my answer is that processes will start incorporating multithreading more, people will run more things on single computers (sharing CPU time), and peripherals will be much more important performance-wise (which we have been seeing for 20+ years now: relatively few programs are CPU-bound compared to the number that are I/O-bound).

[mirhagk]

Not only is multi-threading not taught in school, it is often discouraged because it makes code more complex. I think that multi-threading should be taught a lot more, and GPU-based programming (which is essentially just writing threaded code for a processor with thousands of cores) should be taught to show how to utilize every new core added to a system.

[DemonWasp]

Writing multi-threaded code is so inherently complex and difficult that it's a third-year course at UW. While some students may be able to handle it before then (potentially including a higher-than-normal proportion of CompSci.ca users), most cannot.

Writing GPGPU code is also in that category, and usually completely unnecessary. Focus on getting one processor to do the right thing first, then expand to multiple processors.

Complexity inhibits learning.