Merging In The GNU D Language Compiler To GCC

[XorEaxEax]

I think you are largely missing the point here. D is not Java where you have to use the GC for everything; you can skip the GC for the heavy lifting. Furthermore since D is a relatively clean language, it compiles really fast and tools like rdmd exist to make D behave like a scripting language you may be able to forgo grafting on a high level language for the game logic (in which case you would have been using GC anyways). is that really so hard to understand?

I think YOU are missing the point I was responding to, which was garbage collection (as a GENERAL concept, not specifically in D) and it's impact on performance.

I was NOT commenting on whether or not D forces you to use garbage collecting for everything or not. Personally my 'best of both worlds' is if you are able to decide on a project by project basis if you want manual or automatic memory management, and any language which offers this gets a gold star from me.

[kayosiii]

I think YOU are missing the point I was responding to, which was garbage collection (as a GENERAL concept, not specifically in D) and it's impact on performance.

But that is not what you were saying... you said that using a high level language with GC was ok because it wasn't doing the "heavy lifting". And I said that you didn't need to use to use the GC to do the "heavy lifting" in D either. In both cases you have to deal with a GC running in your program you are just not using it to do the bulk of your memory management. Do you not see the mental disconnect here?

I was NOT commenting on whether or not D forces you to use garbage collecting for everything or not. Personally my 'best of both worlds' is if you are able to decide on a project by project basis if you want manual or automatic memory management, and any language which offers this gets a gold star from me.

Fair enough I can understand that stance; I also see value in having a standard automatic memory management option. Ideally D would be a little more flexible in this regards. Ideally C++ would make it easier to write reliable code.

[misiu_mp]

When it comes to memory management and real time code it is hard not to mention java and android. What at a glance looked like a nice idea is indeed a very, very bad one. The obligatory GC in android is slow and stupid (even though its better than the ordinary JVM). It gives no predictability about the memory usage, lifetimes of objects, and execution latencies, it doesn't even have a way of reliably telling you when objects are garbage collected ( cant trust finalize() ). If, in your pursuit of performance you would go the native way, you will be hindered by the probably worst native interface an interpreted language could wish for. The GC is basically only there to make life harder for the developer, who needs to come out of his way to implement pools, caches and jni wrappers etc, just to avoid GC and get the expected user experience. The end result is a ton of code, not-so-clean code, with quite a risk for leaks (its much easier to get those in java than most people think, mostly because you have to explicitly cache everything).

I don't know if its because it is hard to make a quick and efficient GC (in terms of cpu and memory) in general or just for android. In comparison, C++ and its manual reference counting sounds like a bliss (its quick, doesn't waste memory and is predictable). Recently, Objective C went a step further with Automatic Reference Counting built-in into the compiler. Apple is boasting about it on the first page of their developer site and I must say, it sounds tempting.

So yeah, my personal opinion is that garbage collection should stay away from anything less than a corporate mainframe (where given enough memory, GC can be made really fast, and for which throughput counts, not latency). My guess is this is also valid for D.

[movieman]

Garbage collection is one of those concepts that seems like a good idea in theory, but turns out to be a bad idea in practice. It burns CPU collecting things that you already know are out of scope anyway (e.g. objects that would be created on the stack in C++), it bloats RAM usage because you have to let the garbage accumulate before its collected unless you want to waste all your CPU time looking for your five bytes of unused RAM, and it introduces lots of exciting issues like objects going away at random when you didn't expect it (because you forgot to keep a reference) or not going away when you did expect it (because you forgot to delete every reference). It also prevents the efficient use of destructors to clean up other resources.

I've worked on big C++ projects and big Java projects and we typically spent about the same amount of time fixing garbage collection issues in Java as fixing memory management in C++; and most of the C++ memory management issues turned out to be in third-party libraries because we almost never call new or delete directly ourselves.

[ciplogic]

Garbage collection is one of those concepts that seems like a good idea in theory, but turns out to be a bad idea in practice. It burns CPU collecting things that you already know are out of scope anyway (e.g. objects that would be created on the stack in C++), it bloats RAM usage because you have to let the garbage accumulate before its collected unless you want to waste all your CPU time looking for your five bytes of unused RAM, and it introduces lots of exciting issues like objects going away at random when you didn't expect it (because you forgot to keep a reference) or not going away when you did expect it (because you forgot to delete every reference). It also prevents the efficient use of destructors to clean up other resources.
[...]
I've worked on big C++ projects and big Java projects and we typically spent about the same amount of time fixing garbage collection issues in Java as fixing memory management in C++; and most of the C++ memory management issues turned out to be in third-party libraries because we almost never call new or delete directly ourselves.

You simply missed the point of GCs and how things mostly work in it. First of all the bloating of RAM, may be true as much as working with C++ can have fragmentation. Are applications written in high level languages that work reasonably fast (like Pinta/Paint.Net). Flash and Silverlight have also a fairly fast runtime environment.
Some points where GC "would be nice to have" are today places like multi-threading where you sometimes you don't know the ownership of the objects. Another case is in runtime exceptions, when RAII can be easily made wrong.
RAII also work fairly poorly in spagetti codebases, when with GC, you can isolate a subsystem and when you remove the references to that subsystem, you will know that at least some of the memory is returned back.
At the end I think is a bit of missinformation about what GC does.
Most of GC algorithms will execute when is a threeshold, and will not do it randomly, but will start from your stack space. So if you don't have big addressable space, the GC is not expensive. Also, if you will use a lot of small objects, in C++ you will have tons of fragmentation of the heap, when most Java collectors will compact this memory and remove some leaks some programmers do (not you! )

[movieman]

First of all the bloating of RAM, may be true as much as working with C++ can have fragmentation.

C++ doesn't wait until your program has used a gigabyte of RAM before it clears things out. Perhaps D is smarter, but Java requires you to tell it how much RAM it's allowed to use and it will use most of that RAM regardless of how much is actually storing useful data, only triggering a garbage collection when it's required, or crashing if it requires more. No matter how much RAM I have on the machine, if I give Java application X a gigabyte and it needs a gigabyte and one byte, it crashes. But if I give it two gigabytes, it will now use well over a gigabyte before it garbage collects.

Not only does fragmentation rarely become a problem in C++ unless you have a poor design -- e.g. repeatedly calling new and delete rather than allocating temporary variables on the stack -- but the C library is smart enough to give memory back to the OS when it can (e.g. allocating large blocks with mmap rather than on the heap).

Some points where GC "would be nice to have" are today places like multi-threading where you sometimes you don't know the ownership of the objects.

That's just another case of hoping that GC will save your ass when you don't know what you're doing.

Most of GC algorithms will execute when is a threeshold, and will not do it randomly, but will start from your stack space. So if you don't have big addressable space, the GC is not expensive.

If you don't have a big addressable space, then you'll be doing garbage collections very often as you fill it full of garbage and have to find some more space to use.

Also, if you will use a lot of small objects, in C++ you will have tons of fragmentation of the heap, when most Java collectors will compact this memory and remove some leaks some programmers do (not you! )

Only if you don't allocate those objects on the stack; I'd guess at least 90% of all the allocations in C++ projects I've worked on are on the stack, where they can't cause fragmentation. The exceptions are primarily strings, lists and maps, which we don't use much.

Ultimately garbage collection creates new pitfalls for programmers who don't know what they're doing, isn't required by programmers who do know what they're doing and has much worse behaviour than traditional memory management in almost all cases. You have to go out of your way to bloat your memory usage and burn up CPU cycles in memory management in a C++ program whereas you have to go out of your way not to do that with a Java program.

I read an amusing thread on the web a while back where some web developer had allocated about 20GB of RAM to their Java server app because they couldn't handle the overhead of garbage collection. Which worked great until it actually filled up, and then their system froze for several minutes while it cleared out all the accumulated crap. That's the kind of excitement that garbage collection offers you, and when it happens you suddenly have to go back and rewrite significant amounts of your code to work around it.

[XorEaxEax]

Also, if you will use a lot of small objects, in C++ you will have tons of fragmentation of the heap, when most Java collectors will compact this memory and remove some leaks some programmers do (not you! )

Compacting is an extremely EXPENSIVE operation as it means MOVING memory blocks around in the heap. The reason this very expensive operation is done at all is because managed languages maintain their OWN heap. Now, as expensive as moving memory around is, it's even more expensive when the runtime has to ask the host operating system to resize it's entire heap because it's fragmented and can't fit a requested memory allocation. In short, it's a band-aid, not a feature.

When programming in a native environment you are allocating directly from the system (which has logic in place to limit fragmentation but not by moving memory around which again is terribly expensive). You are not maintaining your own heap (unless you actually want to, just like with a managed language you can setup your own heap by just allocating a large chunk of ram and create your own custom allocator/deallocator).

As for the garbage collector itself, it also eats cpu/ram for it's logic of deciding when data falls out of scope and when it is time to start garbage collecting, now in recent years there's been alot of work put into lowering this overhead but it's overhead nonetheless which does not exist in manual memory management where the programmer explicitly tells the system when and where to reclaim memory.

I'm not on the 'garbage collecting is evil'-train, I think it helps making it easier for the programmer and helps prevent memory leakage due to bugs, BUT it does come with a cost in performance. Whether the benefits outweigh the downsides is up to you and in practice likely varies from project to project. As always there's are no silver bullets in programming.

[kayosiii]

This gives some insight into the design differences between java and D.
http://www.digitalmars.com/d/2.0/safed.html
And this video gives you a bit of background in implementing webservers in D vs Java.
http://video.google.com/videoplay?do...65350602541568

[kayosiii]

This gives some insight into the design differences between java and D.
http://www.digitalmars.com/d/2.0/safed.html
And this video gives you a bit of background in implementing webservers in D vs Java including memory management. (it's long)
http://video.google.com/videoplay?do...65350602541568

[ciplogic]

C++ doesn't wait until your program has used a gigabyte of RAM before it clears things out.

As there are many C/C++/"native" memory allocators algorithms, similarly there are obviously many GC algorithms. In fact many things which are told here are in general 10 years old things that GC used to have and abstraction costs that people that are programming in a GC based language are used to make.
Typical missconceptions are written here:
- "C++ doesn't wait tilll has used a gigabyte of RAM before it clears things out": Java (or .Net) it doesn't wait either. The default GC algorithms are by default using a generational garbage collector, meaning when a "mini-heap" of a size of 4M is filled (or less, this is just an example) the GC occurs. Most of short lived objects are removed. This is important for many reasons to be understood: if you make fairly short lived objects or you get from other framework the same short lived objects and you will set some things out using them and leave them after, they will be released really soon. Nothing else trickier. In C/C++ if you get from a framework a reference of an object you sometimes don't have enough information to do delete by your own. COM solved this somewhat using reference counting, and some people are using smartpointers to do that.
- "Compacting is an extremely EXPENSIVE operation as it means MOVING memory blocks around in the heap." So this is why is not done by default but just for small objects! For example here is explained how the GC is improved for large objects heap (which was not moving) It has certainly a cost but is not that big.
- "Not only does fragmentation rarely become a problem in C++ unless you have a poor design -- e.g. repeatedly calling new and delete rather than allocating temporary variables on the stack -- but the C library is smart enough to give memory back to the OS when it can (e.g. allocating large blocks with mmap rather than on the heap)."
The poor design happens in a lot of cases, and all GC languages in question (D, Java, .Net ones) have "objects on stack" with the note that those objects in Java do not have the same semantics (are primitive types, structs and such). In fact even C++ have a different semantic for working with stack vs heap variables. You have to do the management. At the end, the "give back" memory happens in Java and .Net world too, it happen in D too (which use Bohem's algorithms if I know well). You can setup a default heap to not resize up and down it, but will certainly do give back the memory.
- "Only if you don't allocate those objects on the stack; I'd guess at least 90% of all the allocations in C++ projects I've worked on are on the stack, where they can't cause fragmentation. The exceptions are primarily strings, lists and maps, which we don't use much."
Is in the same category: you can allocate on stack in Java too. Can be annoying as there is no C# like "struct" semantic in it but it can work really well. For every time when you want to get a coordinate from a Point class, create a method int getX() and another int getY() and use the logic using those as being a full point. You don't need to heap allocate everything just because is Java (or C#)

And the conclusion:
- "I read an amusing thread on the web a while back where some web developer had allocated about 20GB of RAM to their Java server app because they couldn't handle the overhead of garbage collection. Which worked great until it actually filled up, and then their system froze for several minutes while it cleared out all the accumulated crap. That's the kind of excitement that garbage collection offers you, and when it happens you suddenly have to go back and rewrite significant amounts of your code to work around it. "
Garbage collection have a cost, for small heaps is too small to count, and certainly as data size grows, will increase the GC time. Also some years back, change may be different, people can experiment and improve.
The G1 GC algorithm in Java can have fairly good times: (source: http://stackoverflow.com/questions/2...-in-production) "I've found that it is very good at keeping within the pause target you give it most of the time. The default appears to be a 100ms (0.1 second) pause, and I've been telling it to do half that (-XX:MaxGCPauseMillis=50). However, once it gets really low on memory, it panics and does a full stop-the-world garbage collection. With 65GB, that takes between 30 seconds and 2 minutes. "
So "Stop the world" can take 2 minutes to free all the garbage. I don't want to be cozy but to work with this sizes a C memory allocator will have slow times to get the next malloc too. At the end, why people do not write web pages in C/C++, or at least not that often? Seems everything is just great in C++, people will argue that the performance is simply amazing, the memory manager do make things to finish even faster. Is here any C++ web designer can write a proper response to me?