What version of linux

[Eglin]

Originally Posted by Muuss

So many people saying that Gentoo is nice, i m gonna try it...
i m yet hearing people here flaming me hehe :)

and mehltd, let my big <3 in peace plz ! 8)

That's cool. I'll try it too, eventually, but I think it is going to be well worth it to wait until they figure out how to use icc (Intel's optimizing compiler). _Every_ report I've seen on icc thusfar rates it as far superior to gcc.

The fact that gentoo, a distribution geared towards building from scratch, is having so much trouble adapting to another compiler is a bit distressing, but I still think it is going to be the best choice once the switch to icc is more fully fleshed out.

[Trumpcard]

icc is available in gentoo portage, so you can pull it down if you like...

* dev-lang/icc
Latest version available: 7.1.006
Latest version installed: [ Not Installed ]
Size of downloaded files: 61,920 kB
Homepage: http://www.intel.com/software/products/compilers/clin/
Description: Intel C++ Compiler - Intel's Pentium optimized compiler for L
inux

Why default to using icc though for portage builds rather than gcc ?

GCC is a commited opensource compiler, switching over 100% to icc , regardless of better optimizations, would be insane... You are bound to eventually run into licensing issues with Intel as icc isnt GPL.. icc is free for personal/noncommerical use, but relying on it would be dangerous should intel decide to yank the way they handle it or discontinue evaluation versions... Then you have a distro that requires a purchased compiler.

Heres the price should should you decide you want to purchase ICC

http://www.intel.com/software/produc.../pricelist.htm


You can put icc in your USE flags and it will try to use icc for any package that supports it....

[Eglin]

Originally Posted by Trumpcard

icc is available in gentoo portage, so you can pull it down if you like...

Yeah, but since the number of ebuilds that actually respect the use icc option is extremely small, that doesn't really help.

Why default to using icc though for portage builds rather than gcc ?
GCC is a commited opensource compiler, switching over 100% to icc , regardless of better optimizations, would be insane...

It's all about the performance, man. I think the insanity would be in _not_ switching if you can realize a 30+% performance gain. Code size could be a benefit for some, too. If the Intel compiler can take code (like eqemu's own distance calculations, for example) and automagically optimize it to haul major ass by going superscalar or hyperthreading or whatever the hell else the processor supports, then I sure as heck would want to make sure that the binaries I download were compiled with it! All evidence supports that vector distance calculations, like the kind that we make en masse, are faster when compiled by icc to the tune of more than double the speed. I've seen cases where certain optimization flags were completely prohibited by benchmarkers, because the performance gains were so dramatic that one can only assume that the compiler is being _too_ smart (i.e. 10x gains in whetstone). Additionally, the Intel compiler is the _only_ 100% ANSI compiler available. So, if I'm going to see gains of 100% in floating-point apps and 30% gains elsewhere, then I hardly think that switching default compilers is _insane_. Nor do I think that it is inappropriate for me to look to a distribution like gentoo, which touts its ability to build systems from the ground up supporting user-defined compilation metrics/options to ease the transition.

You are bound to eventually run into licensing issues with Intel as icc isnt GPL.. icc is free for personal/noncommerical use, but relying on it would be dangerous should intel decide to yank the way they handle it or discontinue evaluation versions... Then you have a distro that requires a purchased compiler.

I don't think so. For one thing, a license change wouldn't affect the status of software built with the compiler, only the compiler itself. So, it isn't like my system would just fall apart. Besides, intermingling gcc apps from that point on would not be any worse than adding icc apps at this point, so I don't follow your argument (unless you're suggesting that one not use icc at all). I can say that I certainly would never ever choose to spend 18 hours building a system from scratch just to get a 1-3% performance improvement (what you can expect over binary packages, assuming you know what you're doing). On the other hand, if I could rebuild using icc and get a 30+ percent improvement across the board, then the prospect would suddenly look much more attractive.

Heres the price should should you decide you want to purchase ICC

http://www.intel.com/software/produc.../pricelist.htm

Yeah, I've seen the details. To save others the hassle of checking it out - it is free for noncommercial use, <$100 for students and ~$305 for all others. That puts it roughly on par with the cost of other popular compilers. Certainly not what I would consider highway robbery.

You can put icc in your USE flags and it will try to use icc for any package that supports it....

Yeah, but that is limited to only a very small number of emerge ports ATM.

The bottom line is that icc seems to do a much better job than gcc ATM. Enough so, in fact, that I'm going to wait to start the 18 hour odyssey of building a gentoo system until it properly supports using icc. Of course, if it turns out that Mandrake starts packaging icc built binaries (a definite possibility, since it has always optimized builds for Pentium chips) before gentoo gets its stuff together, then I'll probably just go that route.

[Trumpcard]

I understand what you are saying, but switching over to a 3rd party, non open source compiler is risky, and ultimately against the opensource principles linux is built on. If you dont get the source code for it, then your linux distro shouldnt use it as a fundamental building block. I can see another SCO fiasco on the horizon...

Gentoo as a whole does not support icc in mass, even though icc is superior for INTEL compilation. I havent seen stats for other processors, but I would be willing to bet that the performance gains are much more minimal, and rewriting 1000's of ebuilds to work with a ultimately COMMERCIAL compiler for a single processor implemenation is alot of work and doesnt make much sense to me.

So your xmms opens 30% faster! Personally, I'd rather wait for the gcc folks to catch up. 3.5 is planning on including autovectorization which should drasticlly narrow the performance / optimization gap between icc.

So, overall, I see what youre saying, but I think it would be ultimately a bad idea to put the fate of a linux distro (and alot of initial work and possible rework) in the hands of a profit driven company... Who knows when Intel is going to hit dire straits and decide to milk everyone they can....

[Eglin]

Originally Posted by Trumpcard

I understand what you are saying, but switching over to a 3rd party, non open source compiler is risky, and ultimately against the opensource principles linux is built on. If you dont get the source code for it, then your linux distro shouldnt use it as a fundamental building block. I can see another SCO fiasco on the horizon...

I'm not following your argument at all. Where are the legal risks? At any rate, the freedom to recompile your source code is precisely what open source is all about. When a new chip or compiler or idea or whatever comes along, you can do your thing and reap the benefits, since you can do whatever you want with the source.

Gentoo as a whole does not support icc in mass, even though icc is superior for INTEL compilation. I havent seen stats for other processors, but I would be willing to bet that the performance gains are much more minimal

Yeah. I'm positive that you're right-on in that regard. I mean, it _is_ an Intel compiler :) Besides, there are features on the newer Intel systems that just aren't existent on other chips.

rewriting 1000's of ebuilds to work with a ultimately COMMERCIAL compiler for a single processor implemenation is alot of work and doesnt make much sense to me.

Your argument that builds would have to "support" icc in particular is not really appropriate, IMHO. The fact of the matter is that the Intel compiler (well, at least the parser) is fully ANSI standard compliant and has gone to great lengths to be command-line compatible with GCC. I don't think that anyone would argue against writing ANSI compatible C++ code, do you? Code that will compile under gcc but won't under icc is typically either making use of a nonstandard feature of gcc or abusing gcc's leniency in some way. This isn't just causing incompatibilities with commercial compilers; this is a portability problem that affects portability to other free platforms just as badly -- a problem that can most definitely be seen in even moving from one version of gcc to another (as can easily be seen in even eqemu's code). I think that the fact that many people are already running Linux kernels compiled with icc and the fact that 2.6 is definitely going to provide support for icc out of the box are pretty strong supporting facts for my case.

So your xmms opens 30% faster!  Personally, I'd rather wait for the gcc folks to catch up.  3.5 is planning on including autovectorization which should drasticlly narrow the performance / optimization gap between icc.

Well, that's your prerogative. I don't fault you for it. OTOH, I doubt that everyone would agree with you. GCC has gradually gotten _slower_ over time, and I'm not sure that platform independent optimizations alone can ever approach the benefits of platform specific ones. Further, it is going to be very hard to beat Intel at their own game. I can't compare gcc3.5 against icc7, but all indicators seem to point to ~20-30 percent improvement in fixed-point code (maybe due to more aggressive memory management), and as much as double the performance or more in floating point intensive code by moving to icc now. That is a big deal. 30% is a big deal. Twice as fast is a very big deal -- especially for something as simple as a compiler change.

So, overall, I see what youre saying, but I think it would be ultimately a bad idea to put the fate of a linux distro (and alot of initial work and possible rework) in the hands of a profit driven company...

Nobody is going to argue that putting "the fate of a linux distro ... in the hands of a profit driven company" is a good idea. I'm totally at a loss, however, as to how you suggest that refining the emerge build process and maybe making a few packages a bit more standards compliant is going to get us there. Again, I'd like to point out that the Intel compiler is ANSI compliant. Since the gcc compiler is striving towards compliance, it is reasonable to think that code that won't compile under icc will not compile under gcc much longer, either.

Who knows when Intel is going to hit dire straits and decide to milk everyone they can....

Umm... I don't know about the "dire straits" part, but I think it is fair to assume that the milking part started a while ago! I remember seeing 486sx chips that had math copros onboard that were disabled. I've seen Celerons that had portions rendered unusable, too. All this because it is often cheaper to destroy functionality than to introduce a new manufacturing process. Nonetheless, you're going to have to provide some scenarios where using icc would backfire for me to understand your stance.

Ultimately, I think that the possibility of a 20-200% gain in performance is going to be very attractive to a lot of people. If compiling my ray tracer with icc means that all my images render _twice_ as fast, you'd better believe that I'm going to look into it! I'm pretty sure that my users would want my binary distributions to be based around the built that runs twice as fast, too! Why on Earth wouldn't they? If using icc means that I can squeeze another year out of my aging systems, then I feel further compelled to look into it. We aren't talking about nominal differences here, we're talking big fun - and hardware is expensive.

[Trumpcard]

Im also at a loss as to what your big complaint with gentoo is then...

That's cool. I'll try it too, eventually, but I think it is going to be well worth it to wait until they figure out how to use icc

icc has been an option for gentoo for awhile now.. So Im confused as to what youre talking about here..

The fact that gentoo, a distribution geared towards building from scratch, is having so much trouble adapting to another compiler is a bit distressing

What troubles are you referring to? That only a few of the package builders ensure their makefiles can take advantage of icc or are capable of compiling out of the box with icc ?

The bottom line is that icc seems to do a much better job than gcc ATM. Enough so, in fact, that I'm going to wait to start the 18 hour odyssey of building a gentoo system until it properly supports using icc.

For INTEL systems, I'll agree with you.. How does it NOT support icc though? Do you mean that you the bootstrap system has to use gcc opposed to icc ? You can alter the ebuilds pretty leisurely if icc is cross compatible to build with if thats the case. Its very easy to manually play with and recompile glibc and the kernel if you desire, if icc is that simple, i doubt it would require that many changes to perform...

Im still confused why the linux community should rush to embrace ICC over GCC other than performance benefits. As far as ANSI compatible, so is GCC, but that doesnt mean that every package compiled on GCC is going to compile out of the box on ICC without code changes. I'd be amazed if EQEMU for example didnt require quite a free changes to compile on ICC for example (I'm more than willing to test it out).

Since you're not understanding my arguments, and apparently I'm not understanding yours, how about articulating what you expect gentoo/mandrake/debian/etc to do to 'figure out how to use icc'

Your comments make it sound like the linux community should drop GCC in favor of ICC, and thats what I'm argueing against, not that someone shouldnt investigate it's use.... I agree that better performance is ultimately a good thing, but how is it that everyone is scared away from ICC as you're making it sound?

[Eglin]

Originally Posted by Trumpcard

Im also at a loss as to what your big complaint with gentoo is then...

I certainly didn't mean to start a flame war, but my complaint would be that the system would take many many hours (possibly days) to install, justifying the time spent by declaring the result as optimal. I have read many many tidbits about using icc with gentoo and can say with absolute certainty that it is not a well supported option. This doesn't mean, of course, that individual packages can't be modified to compile. Nor am I trying to imply that it is not possible to finagle and coerce gentoo into building with icc. I do, however, think it is pretty rough that a system that is so geared up for building from scratch doesn't make it _easy_ to switch default compilers.

icc has been an option for gentoo for awhile now.. So Im confused as to what youre talking about here..

OK, maybe I'm missing something. This is entirely possible, since I'm obviously inexperienced with gentoo. My current understanding is that if you want to have emerge build with icc instead of gcc you must use the 'use icc' construct. The port must then support that same construct. This is most definitely not the same thing as specifying a new compiler.

What troubles are you referring to? That only a few of the package builders ensure their makefiles can take advantage of icc or are capable of compiling out of the box with icc ?

Not being able to properly specify a new compiler for emerge, for one thing. Correct me if I'm wrong, but it isn't enough for a package to compile perfectly under icc, is it? The makefile has to actually be modified to support the 'use icc' construct, right? How many packages are included in a modern Linux system? That's a lot of work to ask someone to do during an install process. Moreover, all of that must be done even if everything would otherwise compile perfectly.

Im still confused why the linux community should rush to embrace ICC over GCC other than performance benefits.

Aside from having better support for the ANSI standard, I think that performance is the only reason. Isn't it reason enough? If you have the choice between two programs with _exactly_ identical functionality, but one runs _twice_ as fast as another, why would you ever choose the slow one? It is that same rationale that would drive me to emerge an icc system. Hell, I want to build gcc w/ icc! Why not? Wouldn't it be nice if your copy of gcc built everything 30% faster? For a person who generally installs binary packages when available (which I suspect constitutes the majority of Linux users), the license of the compiler used to make the package doesn't matter in the least, so I don't see any reason to not use the one that generates the fastest/smallest code. As far as

[Trumpcard]

[quote]I

[Muuss]

Your comments make it sound like the linux community should drop GCC in favor of ICC, and thats what I'm argueing against, not that someone shouldnt investigate it's use....

I don't wanna interfer in that discussion about the interests of icc against gcc, or vice versa (even more cause i m far of being technically able to follow it :p), but if i had to make a choice among those 2, i would obviously go for gcc, simply because we're also running linux on Digital-Compaq servers and a few old sun workstations, all this with shared homes. Nobody would even accept to talk of the eventuality to modify their code each time they change of platform. Before going for performance, we have to respect compatibilities.
Just 2 questions tho, is icc able to produce some code that is compatible whatever your ix86 cpu is, ie P4 code working on a P2 for example ? And is P4 written source compilable on a P2 ?

[Eglin]

Originally Posted by Muuss

if i had to make a choice among those 2, i would obviously go for gcc, simply because we're also running linux on Digital-Compaq servers and a few old sun workstations, all this with shared homes.

I agree that it is nice that gcc runs on all of these machines nowadays. OTOH, shared homes or otherwise, you're still going to have to recompile for each platform. Good code should strive to compile with icc or sun's cc or hp's cc or ... just as well as with gcc, IMHO. The biggest problem with switching compilers is usually related to linking with external libraries. Since icc can, from what I understand, produce code compatible with gcc's ld (or MS COFF and others), I don't think this is going to be a problem.

Nobody would even accept to talk of the eventuality to modify their code each time they change of platform.

That's cool. Unfortunately, simply using gcc on all platforms isn't enough to satisfy that requirement (endianness, setenv(), and threads vs pthreads come to mind as examples of gcc behaving differently on sun than Linux).

Before going for performance, we have to respect compatibilities.

That's cool. My suspicion, though, is that incongruities would have more to do with your code than your compiler.

Just 2 questions tho, is icc able to produce some code that is compatible whatever your ix86 cpu is, ie P4 code working on a P2 for example ? And is P4 written source compilable on a P2 ?

I think that it is possible to use icc to write code for older x86 chips, if that is what you're asking. I am told that the performance benefits are still evidenced, perhaps due to more agressive memory alignment strategies. The compiler supports some optimizations, though, that are very chip specific. This is true for gcc and ms cl, too.

[Muuss]

hehe, its funny to see how you deturn the general sense into small pieces just to make it keeping more a less the same meaning, but by still isolating the parts where you can place your opinion and finally return them

I basically don't care of what gcc compiler people use here at job. Most of them produce code that is trashed the next week (university researchers, in voice recognition, numerical analysis and such...) and they make themselves the choice of their compilers.
One thing is sure, all of them finnally end up with gcc as soon as they start to work on unix like stations.

I think that it is possible to use icc to write code for older x86 chips, if that is what you're asking. I am told that the performance benefits are still evidenced, perhaps due to more agressive memory alignment strategies. The compiler supports some optimizations, though, that are very chip specific. This is true for gcc and ms cl, too.

Well no, that's not what i was asking. The question is : i wrote some optimized code for my P4, can i compile it on that other computer which only has a P2 ?

[Eglin]

Originally Posted by Muuss

hehe, its funny to see how you deturn the general sense into small pieces just to make it keeping more a less the same meaning, but by still isolating the parts where you can place your opinion and finally return them :)

I felt that your statements should be addressed individually.

One thing is sure, all of them finnally end up with gcc as soon as they start to work on unix like stations.

This is exactly contrary to your earlier statements implying that portability was of utmost importance. Not to mention the fact that it has no impact whatsoever on the fact that simply using gcc doesn't resolve all of the underlying differences in the various subsystems.

Well no, that's not what i was asking. The question is : i wrote some optimized code for my P4, can i compile it on that other computer which only has a P2 ?

So, you are asking if code specifically engineered to take advantage of features only found on a p4 (hyperthreading, for example) will run on a chip that doesn't support those features? Of course not. I'm not really sure how this could be unclear. I don't really see how this is an issue, though. It certainly hasn't hurt adoption of prior chip-specific features (like mmx or 3dnow! or sse or ...). If you don't have mmx avaliable, you just can't run mmx optimized code (obviously).

[Muuss]

Thanks thanks, you confirmed my thoughts. I was wondering if it was possible for the compiler to automatically produce the alternative code to make the software able to execute on a lower CPU (as u see, i have low qualities of coder :p)

This is exactly contrary to your earlier statements implying that portability was of utmost importance.

Nope, nope, that's a too fast conclusion coming from an isolated sentence. there's not only gcc and icc in the world, portability can also be assumed in a laboratory with another compiler than the one choosen in that other laboratory, or this one... The only conclusion i can make of what's happening here (100+ persons writing source code, great majority of them being doctors) is that gcc is in all the cases the elected compiler.

Tho, i totally agree that on each specific platforms, there's several alternatives to gcc, which can certainly provide better results, as i wrote before, i have no preference for gcc or another one...