By execution model I mean a process of having a .NET Framework application actually executed on the machine (CPU), starting from having its source code written. It contains all steps and actions necessary to happen in order to transform source code (like C#) into machine (assembly) code and execute it.

.NET languages standardization

As we all know, .NET is just an execution platform. There are, however, a lot of programming languages which allow us to write code which can then be executed by the platform-specific (not language-specific!) CLR. In order to give programming languages’ creators a set of rules which their language and its compiler must meet to be .NET-compliant, Microsoft defined a Common Language Infrastructure (CLI, also referred to as MSIL in the old days ) which is also standardized by ISO and ECMA.

The main goal of it is to define how the source code should be compiled to the Common Intermediate Language (CIL). As we already know, the CIL should have a standardized format, because it’s then JIT-compiled to a platform-specific assembly code. Some of the most common implementations of CLI are .NET Framework, .NET Core and Mono. I recommend you to check Matt Warren’s article on .NET Runtimes history as well

Common Language Infrastructure implies also some other aspects of the programming language and its compiler, including Common Type System, language-agnostic metadata and Common Language Specification. You can read more about it for instance here.

This all allows us to use various programming languages, like C#, VB.NET, F# or JScript .NET to develop .NET applications. Anyone can still implement their own own .NET language – it “just” needs to conform to the standards.

There’s also another term introduced somewhat around .NET Core – .NET Standard. Its purpose is to allow code sharing between various .NET implementations. It actually defines a set of APIs that each .NET platform should implement. More details here.

This whole standardization makes us arriving to the .NET applications execution model, which is how every standards-compliant CLR application is executed.

.NET execution model

Independently of the language of choice, .NET application’s execution model can be described as the following 4-steps process:

1. Writing source code in a programming language of choice, including the usage of its compiler
2. Compiling source code to Common Intermediate Language (CIL)
3. Compiling CIL into native code – details of this step in the previous post
4. Execution of the native code.

These steps are well-resented on the schema below and then described in details in the next sections.

Writing source code in a programming language of choice

As mentioned before, .NET applications can be implemented using any language of choice, which is compliant with CLI. Another important part is a compiler – it defines a general syntax of the language, kinds of data types that can be used by a programmer etc. For instance, current open-source compiler for C# and Visual Basic is Roslyn.

In the end, role of the compiler is to transform the source code to its CIL equivalent.

The simplest possible complete application written in C# can look as follows:

Compiling source code to CIL

After the source code is written, it must be compiled to CIL, which is an intermediate language understood by the CLR. On this level, the whole execution process becomes language-agnostic, which means that as soon as the code is compiled to CIL it is executed by the CLR and programming language’s features (e.g. its compiler) don’t play any more role.

CIL code looks more like assembly code, however containing some specific instructions. It’s still far more readable than native code. Apart from the direct transcript of source code, CIL contains metadata about the DLL/EXE assembly file.

The C# fragment presented in the previous section compiled to CIL by Roslyn looks as follows:

A bit more than the original source code, isn’t it?

Firstly there is a metadata instructions section which contains information about the assembly, execution framework’s version, types used in the code and external references. Next there’s the actual CIL code (equivalent to C# code presented above) listed.

Compiling CIL into native code

We’ve already covered this step in details in the previous post about Just-In-Time compilation, which is just a bit more fancy name of this process Now you can see how and where it fits into .NET application’s execution model.

Normal JIT compilation

By default, JIT compilation is done in the place where “Normal JIT Compiler” is drawn on the schema below:

IL instructions of a particular block of code, in the example diagram of a method Foo(), are JIT-compiled when the block is to be used for the first time. Then, this JIT-compiled code is stored as the JIT-ed equivalent of Foo() method’s code in the memory cache. Such native code is then executed and if the same block of code needs to be called again in the future, it will not be re-JIT-compiled, but instead the native code will be taken from the memory cache and executed again.

There’s a possibility to skip adding the native code into memory cache for future executions and it’s referred to as “Econo JIT Compilation”, however it’s not widely used and probably obsolete since few versions of .NET, so we won’t examine it here.

Pre-JIT compilation

The schema presented above changes a bit as soon as we want to use so called Pre-JIT compilation. We mentioned it already and we know that one of the techniques to allow pre-JIT (or ahead-of-time) compilation is using Native Image Generator (ngen.exe) which allows to transform CIL assemblies into native code files and to store them as a file on the disk (in the Native Image Cache). It allows to natively compile the whole assemblies – it doesn’t allow to compile only single methods as JIT compiler does.

Refer to the new version of the schema presented below to see how the process changes with NGen.

Normal and ahead-of-time CIL compilation modes provide different cons and pros, including:

• NGen delivers faster start-up time, especially in large applications used by many users in the same time, but requires more disk space and memory to store both the CIL and pre-compiled images,
• JIT compilation at runtime can deliver faster code, because it targets the current platform of execution; NGen produces the native images with instructions that can be executed on all possible platforms, meaning it must use the oldest from currently used instructions sets to be backwards-compatible,
• JIT is able to dynamically re-compile the code for a better performance depending on the execution conditions (e.g. by detecting a hot path in the executed code).

Choosing one of the modes depends on your use case.

Execution of the native code

Finally, the code blocks are run by the CLR, which asks CPU to execute compiled instructions sets. If you’re interested in how CPU exactly works and executes the instructions, I recommend you watching this YT video.

Summary

We’ve seen today how the .NET applications are executed step-by-step, starting from source code (like C#) up to being actually executed by the processor.

I also encourage you to read – if you haven’t already – all the previous posts in the .NET Internals series. Just click here to see them all

Let me know if you have any doubts or thoughts, I’m always open to your comments and opinions Especially the constructive ones pointing out some parts I omitted or didn’t describe precisely enough.

I hope it helped you learn something new

The post [.NET Internals 10] Application execution model appeared first on CodeJourney.net.

What makes managed applications portable?

Intermediate Language (IL)

Viewing MSIL

Just-In-Time (JIT) compilation

How Toyota brought us JIT?

Let’s start by some historical background In the early 1960s, Japanese engineers at Toyota had to reorganize their warehouse management, because they had very high storage costs – delivery of the parts from suppliers took a lot of time, because they ordered a lot in advance. Every delivery had to be handled by someone, so they needed a lot of employees. Ordered parts were stored in the warehouses for a long time, required a lot or storage space and maintenance.

JIT compiler

Following Toyota’s success , CLR creators introduced a just-in-time (JIT) compiler into the framework. Its role is to compile intermediate language code into assembly language according to hardware and operating system’s characteristics (depending on the machine on which the code is being JIT-compiled). Unlike unmanaged languages, in which the source code is compiled into machine language prior to program’s execution, IL is (by default) compiled to CPU instructions at runtime. That’s how .NET engineers made the machine code delivered on time to the CPU.

JIT compiler is a part of CLR (similarly to garbage collector). To make things simple, it can be said that JIT compiles a particular method’s source code as soon as this method is called for the first time. Of course, it takes into account the hardware, e.g. the CPU type and its instructions set to produce correct assembly code. Such machine code is then saved in the cache and re-used every time the same method is called again within the same application’s execution. It means that the parts of the code which are not used (called) during the program’s execution may not be JIT-compiled at all.

Just-In-Time compilation is actually a combination of two approaches: ahead-of-time compilation and interpretation, mixing advantages and disadvantages of both. However, compiling IL code into assembly one at runtime allows for several interesting features like dynamic typing (ability to retrieve the actual object’s type at program’s execution). That’s why JIT compilation is often treated as a form of dynamic compilation.

Another advantage of having JIT compilation is that the source code we write is “further from the hardware”, so it can be made more readable and human-friendly, whereas the metal-touching stuff is “hidden” in the IL (kudos to MSIL developers!)

Can we still compile ahead-of-time?

Summary

In one of the next posts we’ll dig into application execution model in .NET Framework, where we’ll see when and how exactly the JIT compilation process fits into the whole C#->CPU journey

so if you have your own opinion on JIT or any other remarks, don’t bother and share it in the comments!

The post [.NET Internals 09] Just-In-Time (JIT) compilation appeared first on CodeJourney.net.

(De)allocating objects on LOH

As we know from the second post, during allocating the memory only objects of size greater than 85,000 bytes are placed on LOH. There are also some exceptions, like arrays of double which are put on LOH (in 32-bit architectures only) as soon as they reach 1000 (or more) elements (not something around 10626 elements as could be expected). This is quite important to know to be aware what kind of objects have impact on heap fragmentation (more details below).

So we know when the objects are allocated on LOH, but when are they deallocated?

LOH is collected in the same time as the generation 2 collection occurs. It can be triggered if memory threshold for either gen 2 or LOH is exceeded. Conditions for garbage collection can be found in this post.

That’s why keeping large LOH may affect the GC’s – and the whole application’s – performance.

Garbage collection on LOH

LOH fragmentation

The reason why Large Object Heap is not compacted (by default) is because it’s used to store big objects (>85,000 bytes). Copying such amounts of data would seriously incur the performance of garbage collection process.

Anyway, memory of objects allocated on the LOH is reclaimed so it may eventually become fragmented:

We’ll see below how, but .NET keeps track of “Free space” memory blocks to know which chunks are available for new allocations on LOH. When allocating it looks for a block large enough to store the whole object.

However, imagine that there are 2 free space blocks next to each other. Both were marked free, but they represented different objects (maybe they were next to each other on the heap because one referenced the other). What do you think GC will do? Will it treat them as two separate free memory blocks, making less chances for the next allocated object to fit into one of them?

Fortunately not. GC has an optimization introduced which makes such adjacent free memory chunks “merged” together:

How does GC do it? Let’s see in more details.

Free memory representation on LOH

Instead of compacting Large Object Heap, garbage collector keeps the address ranges of not used large objects in a Free Space Table:

As you can see on the figure above, as soon as gen 2 collection run, address ranges of two unused objects were just added to the Free Space Table.

Now you can see that “merging” two adjacent free memory chunks is just a simple addition operation (or modification of one number in the table).

Allocating memory on LOH

As soon as a new large (>85,000 bytes or applicable array) object is to be allocated on the managed heap, GC looks for a single “Free Space” block to hold it. However, it’s rather unlikely that the particular object’s size will fit into one of the free memory chunks. In that case, a new object will be allocated on the top of the heap (just after ‘Object D’ on the figure above).

It may happen that the memory obtained from the operating system for LOH is already used (read here for more info about memory). Garbage collector then asks the operating system for more memory segments to be acquired for LOH. If it fails, gen 2 collection is triggered hoping that some memory blocks will be freed and then the allocation will be possible.

Let’s now think about it for a while. We said previously that LOH collection triggers gen 2 collection. So trying to clean-up the Large Object Heap every time an allocation on it is made would be a potential performance killer.

How does GC solve this issue? Well, in fact, after a lot of optimizations introduced to LOH management in .NET 4.5, the GC takes the following actions in order to make a new allocation of a large object:

• firstly, the GC tries to allocate new objects into one of the free space “holes” on the LOH (it’s quite simple to calculate whether any chunk is large enough to store the object knowing ranges of free space blocks from the Free Space Table).
• if the above fails – garbage collector prefers to allocate new large objects at the end of the heap. Even though it may involve asking the OS for more memory segments, it’s been found to be easier and less consuming operation than performing full GC hoping to free some memory chunks on the LOH first.
• only if the above fails (LOH cannot be “extended”) – GC triggers gen 2 collection hoping to free some additional space that could be used for a new allocation.

What’s worth noticing is that this actions order seems to be good for performance, but can sometimes be a reason of memory fragmentation.

Manual LOH compaction

As you should already know from the previous article, LOH can be compacted programmatically, by setting the GCSettings.LargeObjectHeapCompactionMode property. The simplest way to force LOH compaction is as the following snippet presents:

There might be some niche cases in which LOH compaction may be useful. More details and discussions can be found for example here.

Best practices working with LOH objects

We can try to simply design our applications to use the less possible number of large objects, but let’s not exaggerate. We are living in a world where 8GBs of 2133MHz RAM costs sometimes less than 100$, so the memory is generally cheap

In principle, the rule seems to be simple: large objects we allocate should be reused (e.g. cached) as much as possible.

We should just keep in mind that allocation of large objects can be costly, because of a need to perform gen 2 collection in some cases before the object is allocated.

An example of potentially problematic large object can be a ViewState used in ASP.NET applications, size of which can easily exceed 85K. There are some good articles explaining how to not stupidly incur ASP.NET app’s performance using it, for instance this one.

There are also a lot of tools which can be used to measure the memory state and performance of our applications (also its internal mechanisms like garbage collection, heaps compaction etc.) which we’ll surely cover in one of the next posts within the series

Summary

Today we examined – previously a bit forgotten – Large Object Heap. We saw how the information about free memory blocks on it is stored by .NET Framework and how new objects are allocated on it.

I think it’s one of the next internal concepts of .NET worth knowing and understanding, even though in the common scenarios and business applications you probably won’t get into troubles with LOH. However, it may be practical and useful to know when working with some more memory-demanding applications like games.

I hope this post clarifies some LOH topics for you.

Let me know if there are any topics you’d be interested in reading about. I’m here to provide some value to you, so I’m open for your criticism and suggestions

Stay tuned!

The post [.NET Internals 08] What about Large Object Heap (LOH)? appeared first on CodeJourney.net.

Unmanaged resources and finalizers

First of all we should all get familiar with what unmanaged resources are. These ones can be files or folders on the disk, database connections, GUI elements or network resources. You can also think of them as some external elements we need to interact with in our applications through some protocol (e.g. database queries or file system manipulation methods). They’re called unmanaged not without a reason – any unmanaged resources accessed from .NET code will not be automatically cleaned by the garbage collector. That’s how they are not(un)managed.

Maybe you’ve heard about destructors in C++ – remember methods called after class’s name (like its constructor) starting with ‘~’ character? This also exists in .NET, but it’s not what you’d expect from C++. In .NET, such methods are referred to as finalizers.

Finalizers and destructors in .NET

Nonetheless, you can define a finalizer in your class in one of the following ways:

 Using “destructor-like” syntax:

Using Finalize() method:

If you examine the IL code of both versions, it turns out that the destructor can be seen as a syntax sugar for the finalizer. In fact, the ~MyTestClass() method is translated into Finalize() method:

The only difference is that – as you can see on the IL above – Finalize() method created by the compiler from the destructor is implicitly calling the Finalize() method on the base class of the object (you can see it in the finally block in the IL generated). So in case of using a destructor-like syntax, we are actually creating a finalizer by overriding Finalize() method, whereas when implementing a Finalize() method directly we do it as we’d use the new keyword (hiding base class’s implementation of Finalize() method).

Will GC call the finalizer for me?

As you may know, finalizers methods are normally used to clean-up some unmanaged resources (e.g. to release a file handle or database connection). You may think that as soon as your object is ready to be collected (nothing references it anymore), garbage collector will firstly call the finalizer (to ensure everything is cleaned up) and then reclaim the memory used by the object.

Unfortunately not. If you think about it one more time – GC cannot know what’s inside your Finalize() method. You can put there the code which downloads the whole Internet just like that:

… and even though we have nowadays a bit faster bandwidths than the one from the gif , you can clearly see it wouldn’t be a good idea to block garbage collection because of some costly process being executed before the memory for the object is reclaimed.

In order to avoid the above-described scenario, which would slow down the GC’s work significantly, .NET contributors decided that the finalizers on the objects are called periodically on a separate thread, completely independently from the GC.

This way the main GC thread is not blocked. However, it adds a bit of complexity to the garbage collection process – continue reading for more details.

Finalization mechanism

As we said in one of the previous posts, one of the various sources of references to objects in our .NET application (GC roots) are “objects finalization references”. Let’s now make this mysterious term clear.

In order to prevent having a finalizable object (containing finalizer method) reclaimed before its Finalize() method is called (which happens independently from the GC as you already know), there’s a separate references list called finalization queue maintained by the GC. As soon as memory for a new finalizable object is allocated, a reference to it is also put on the finalization queue, which becomes a finalization root for this object (it’s not the same kind of root as “normal” GC roots – it’s treated a bit more differently).

The diagram below presents the memory state just after few new objects were allocated on the managed heap.

As you can see, objects ob2, ob3, ob5, ob6 and ob10 are all placed on the finalization queue, which means all of them contain Finalize() method.

Within these objects, ob2 and ob5 don’t have any references (roots) – it’s visible on the left side (heap). It means that these two objects are not used (are not referenced by anything) so they’re ready to be garbage collected. However, each of them contains a finalization root on the finalization queue.

As soon as the next garbage collection occurs, ob2 and ob5 cannot be reclaimed straightaway, as there’s a Finalize() method to be called on them first. Instead, when GC examines these two objects, it sees that they are not referenced from “normal” roots, but they have finalization references on the finalization queue. If that’s the case, GC moves finalization reference to another data structure called fReachable queue (also visible on the diagram above).

After, both queues and the heap look as follows:

That’s what I meant by stating that finalization roots are “special” kind of GC roots. Apart from that, ob2 and ob5 are still seen as “rooted” (as still being referenced) and because of that are not reclaimed by the GC. In that moment, standard generational GC rules apply, so if the collection below was a gen 1 collection, both objects will be promoted to gen 2.

Periodically, the finalization runs on a separate thread exploring the fReachable queue and calling Finalize() method on each object referenced from it. Then it removes the reference to this object from the queue. At this moment, finalizable object becomes rootless and can be reclaimed in the next garbage collection cycle.

Dispose for the rescue

In order to unify usage of finalizers and allow developers to correctly free their resources without unnecessarily prolonging their objects’ lifetime (as you saw above, finalizable objects stay uncollected for at least 1 more GC round than “normal” objects), programmer can (and should) implement a dispose pattern. The purpose of it is to allow the developer to free unmanaged resources “manually” as soon as they are not needed anymore.

The framework also provides a GC.SuppressFinalize method which tells GC that the object has been manually disposed and no more finalization is necessary. As soon as it’s called, the object reference is removed from finalization/fReachable queue. The goal is to prevent finalizing the object twice.

IDisposable

.NET Framework provides a System.IDisposable interface which should be implemented by a class which works with unmanaged resources.

Sample implementation of dispose pattern can look as follows:

Implementing IDisposable interface only forces us to add Dispose() method. Inside, we perform the resources clean-up and call GC.SuppressFinalize(this) to mark our object as finalized. We also implemented a separate method, in the example called CleanUp(bool disposing), which performs the actual releasing of unmanaged resources. But why do we need this bool disposing parameter?

It may be important to differentiate when the resources cleanup is called directly from code (we’ll see how it can be done in the next section) or by the finalization thread. As you already know, finalization is executed periodically on a separate thread. It means that if our UnmanagedClass needs any thread-specific resources cleanup, it’s not a good idea to execute it on totally independent, finalization thread, which is out-of-context at that moment.

That’s why we added a boolean parameter to the CleanUp method, which will be set to “true” as soon as Dispose() method is called directly from code. In that case we know that we can execute thread-specific cleanup code.

Additionally, we implemented the Finalize() method, which also calls CleanUp(), but passing “false” parameter’s value. In that case, as the finalization thread is out-of-context, we shouldn’t execute thread-specific cleanup code. We should treat Finalize() method as a last resort in case Dispose() has not been called because of some reason.

You can also check Microsoft dosc to see what’s their recommendation on implementing IDisposable interface.

Using statement

As mentioned above, the programmer can explicitly call Dispose() method on the class:

but there’s another – and preferred – way: wrapping usage of the class within using statement:

How does using statement ensure that Dispose() is called on our object created within it?

Let’s see the IL:

As you can see, the compiler translates using statement into try-finally block. In the finally part Dispose() method is called on our object’s instance.

This way the compiler ensures that even if there’s any exception thrown in the using block, the object will be disposed. That’s why we should always work with objects that may use unmanaged resources in such a way.

Summary

In this 7th post from .NET Internals series we examined how unmanaged resources are handled within .NET Framework and what’s the GC’s role in that process. We got to know two new GC-internal data structures: finalization and fReachable queues. In the end, we got familiar with dispose pattern and using statement.

I hope it helped you to clarify these topics a bit

See you next week!

The post [.NET Internals 07] Unmanaged resources: finalization, fReachable queue and dispose pattern appeared first on CodeJourney.net.

Heaps generations

As we know from this post, heaps used by .NET process to allocate reference objects are allocated on different kinds of heaps. In the previous article we also got to know that only Small Object Heap (SOH) is the subject of compaction process. Large Object Heap (LOH) is not compacted because of the performance reasons.

This is however not the only optimization introduced by GC contributors into Small Object Heap. SOH in .NET is also divided into 3 generations:

• Generation 0 – containing newly created objects, which did not have any collection performed on them so far,
• Generation 1 – storing objects that survived a single garbage collection (were not reclaimed during it because of still being in use),
• Generation 2 – keeping long-lived objects that survived two or more GC cycles.

LOH is not divided into generations. More details on LOH management in the article linked just before the “Summary” section below.

Thanks to such a division of managed heap, the actual collection is performed not on the whole heap, but only on one of the generations, where:

to perform a collection on generation X == to collect objects on generation X and all its younger generations

That’s why a generation 2 collection is knows as a full garbage collection. During this process objects present on all heaps (on all generations: gen 0, gen 1 and gen 2) are examined and reclaimed. Obviously, this collection is also the most costly one.

Collection survivors

In that place we should introduce a term collection survivor. It can be defined as an object which survives the collection (is not collected during GC collection cycle as it’s still referenced by something).

Knowing that, the following “promotion rules” can be defined:

• object survives generation 0 collection => object is promoted to generation 1,
• object survives generation 1 collection => object is promoted to generation 2,
• object survives generation 2 collection => object stays in generation 2.

Collection thresholds in generations

In the 5th post in the series, it was mentioned that one of the conditions on which the garbage collection can be triggered is when “the memory used by the objects on the managed heap exceeds some defined threshold”. In fact, this is not a single threshold, but several thresholds. More precisely, there’s a separate one per generation. As soon as size of all objects in a particular generation exceeds threshold’s value, the collection is started.

In the beginning, the thresholds’ values for each generation are initialized to the following values:

• gen 0: ~256K,
• gen 1: ~2MBs,
• gen 2: ~10MBs.

Nonetheless, these are only initial values and are adjusted by the GC at runtime. One of the conditions to increase a particular threshold’s value is then the survival rate in a generation is high (more objects from particular generation are either promoted to the next one or stay in generation 2), which makes the GC cycles to run less frequently (the condition of exceeding the threshold is not met that often).

This approach seems reasonable, as the main goal of GC is to collect as many objects as possible and reclaim their memory, not to “waste time” only to get to know that most of the objects are still in use.

In general, CLR tries to find a balance between two factors:

• not letting application’s working memory set to get too big,
• not allowing GC to take too much time.

Structure of generational heap

The diagram below presents how SOH actually looks like being divided into above-described 3 generations:

As can be noticed, the next object pointer (you can read more about it in this post) refers to the place after the last object in generation 0. That’s how it happens that newly created objects are always allocated in generation 0.

Generational GC cycle

The following schema very clearly presents how the full generational collection looks like (click to enlarge to see it clearly):

The first step is generation 0 collection. You can notice that objects H and D are not referenced by anything, so after the first cycle on gen 0 is performed, these objects are reclaimed and not promoted to gen 1. The same happens with object F, which is not being referenced to anymore when the collection on generation 1 is performed – that’s why it’s reclaimed during this step and not promoted to generation 2. In the end of the full GC cycle, only still-referenced objects stay on generation 2.

What’s also important to notice is that after each collection generation 0 becomes empty (contains no objects). Generation 1 may still contain objects promoted from generation 0 and generation 2 may contain long-lived objects.

Why is static data harmful for GC?

Having the knowledge presented in this post, we can deduce that the more objects are stored on a generation the longer collection process lasts. What we should also already be able to see is that generation 2 contains long-lived objects which are mostly static or global objects that survive garbage collection cycles (as there’s always a “live” reference to them in the application).

Of course, there are some reasons why static data can be justified (e.g. server-side apps storing some context), but normally using static data should be avoided as much as possible and reasonable. I think you already know why – it simply incurs the application’s performance by making garbage collector’s work harder and longer.

Cross-generation references

There’s also one very interesting issue in having the managed heap divided into generations. Can you imagine the case, when an object is already promoted to gen 2, but one of its reference-typed properties has just been initialized in gen 0? Consider the following code:

Let’s assume that an instance of GuyFromGen2 has already been created long time ago and it’s already promoted to generation 2. At this moment someone calls CreateYoungObject() method, which initializes _youngObject variable. This makes the “parent” instance of GuyFromGen2 holding a reference to its instance object located in gen 0. 

When there’s a generation 0 collection triggered (which, as we said, examines only objects in generation 0) at this exact moment, it would find _youngObject allocated on the heap having no gen 0 references to it (as it doesn’t check gen 2 references during gen 0 collection) so it would assume the object not being used and reclaim its memory.

Fortunately, .NET has a solution for this issue – card table data structure.

Card table data structure

Card table is used by .NET’s memory management system to store information about objects from older generation (gen 2 in that case) that reference younger objects (from gen 0 or gen 1). The execution engine adds an entry into card table as soon as any reference to gen 0 or gen 1 object is created “inside” gen 2 object.

For performance optimization reasons the card table stores this information per “bit”, which represents 128 bytes (or 256 bytes in 64-bit architectures) of managed memory. That’s how the card table can be imagined:

Then, as soon as gen 0 or gen 1 collection takes place, apart from examining references only from the particular generation’s roots, GC also checks which memory ranges have their bits set to 1, finds out which objects are allocated within these memory ranges and treats them as potential GC roots (analyzing what these objects reference).

What’s more, as Konrad pointed out in his comment, because of write barrier optimization the whole bytes (8-bit chunks, not only single bits as presented on the figure above) are marked as potential roots sources. So in fact, such cross-generations reference can mark up to 2048 bytes (for 64-bit architectures: 256 bytes x 8 = 2048 bytes) memory chunk as a potential GC roots source.

Every referenced object found this way is added to the list of objects still in use (if you still don’t know what are GC roots and still in use objects list, read the previous post now).

but… what about LOH?

If you’re curious about Large Object Heap (LOH) in-depth memory organization, I recommend you reading this article by Maoni Stephens. I can just mention that LOH has its own memory threshold which can be exceeded and it also triggers Gen 2 collection – from GC point-of-view, collection of objects allocated on LOH happens only during Gen 2 collection.

Summary

We went through the concept of having SOH managed heap divided into 3 generations. By understanding how objects are promoted between generations and the implications of being a long-lived object we saw the potential harmful impact of static data on our application’s performance.

Another interesting thing we pointed out was the card table data structure used to solve cross-generations references issue.

Next week we’ll see more on unmanaged resources and objects’ finalization.

Stay tuned!

The post [.NET Internals 06] Generational garbage collection appeared first on CodeJourney.net.

Finding objects to be garbage-collected

As we already know, .NET doesn’t allow developers to directly allocate the objects on the heap. Instead, the only thing the programmer needs to do is to use the new keyword – framework takes care of all the rest (placing the object on the proper heap, initializing it etc.). After that, you can simply forget about your object, because it is released (collected) automatically by the garbage collector as soon as it’s not needed anymore. But how does the GC know which objects it can collect?

In principle, all the garbage collector does is looking for the allocated objects which aren’t referenced by anything, which means that such objects are no longer needed. This makes such objects ready to be collected (their memory can be released – marked as being possible to be used again).

There are several sources of references to objects allocated on the heap:

• stack (as we know from the 2nd post),
• global or static reference variables,
• CPU registers,
• interop references (.NET objects used in COM/API calls),
• objects finalization references.

GC roots tree

All above-listed sources are called GC roots (also referred to as root references). As objects referenced from any of the roots can reference another objects (for example a customer may reference its orders collection), all these nested references are represented as a tree (graph) for every .NET application. Such references tree may look as follows:

As you can see, GC root is present on the stack, pointing to Customer reference object. It contains an ArrayList collection of orders, which is referenced by Customer object. The collection itself also contains references to its elements, so the tree grows up.

Each root contains either a reference (address) to some object or null value.

Unreachable (not used) objects

The GC roots tree described above is used by garbage collector to know which objects are no longer needed. As we see, the graph contains all objects that are still in use (are referenced by something else).

GC uses this information the other way around –  generally the GC classifies all objects that are not reachable from any root (not present in the graph) as garbage. Such objects are then collected (the memory allocated for them is released) by the GC during its collection cycles, which will be described in details in the next posts in the series.

Memory releasing (garbage collection)

As mentioned, garbage collection is a process executed in cycles, which will be covered in the later articles, but the process itself contains two main steps:

1. Marking phase – finding which objects are still in use,
2. Collection – actual collection process of not-anymore-used objects (including SOH compaction).

Let’s see both phases in details.

Marking phase

When garbage collector runs its memory releasing process (referred to as ‘collection’), it needs to know which objects on the heap are still in use by the application – obviously, such objects cannot be collected.

In order to obtain this information, GC goes through the whole root references tree and for each GC root moves along all its reference tree (all objects that are referenced from the particular root) in order to mark each object in this tree as “still in use”. That’s why this integrated part of garbage collection process is called marking phase.

Marking is a recursive operation, because as we said before, roots reference some object, which references another object, which references another object…

so in order to make sure that every object in use is added to the “still in use” list (if it’s not there yet), this recursive marking is done as can be seen below:

Let’s now see how it fits into the actual collection operation.

Collection

The actual collection process, including previously-examined marking phase, can be pictured with the following pseudo-code:

First of all, GC gets a list of all GC roots in the application (the list is maintained by JIT compiler and the runtime). Then it needs to iterate through each root and perform marking operation on it. Knowing that marking is a recursive operation, after this phase is finished, GC can be sure that all objects that have any root reference (either direct one or being pointed to by another objects) are added to the list of objects in use.

Next step is the actual memory releasing process of not used objects, represented as a call to Cleanup() method on the listing above. Memory cleanup itself is a very simple process – it’s just marking a particular memory block(s) as free (not used). There’s no overwriting of memory or anything like that (except ‘side overwriting’ which happens during heaps compaction – described below). Just simple boolean-like terminology is used for memory blocks – used (1) or not used (0). It works that simple for LOH , but collection on SOH (still don’t know what are LOH and SOH? Go and read here quickly!) also involves the compaction process. Let’s see what that is.

Heaps compaction

As we know from the first post, memory on the managed heaps can get fragmented, leaving holes of not used memory between live objects. That’s why the garbage collection for SOH process involves compaction, which is a process of removing the memory holes on the heap. It’s often referred to as a Copy Collection process, because it’s based on finding the dead objects or memory holes (it’s actually the same – holes of the memory that are not used by anything, present in-between live objects) and overwriting these holes with the live objects allocated later (next) on the heap – by copying chunks of live objects ‘down in the heap’.

The result of the compaction process is the contiguous SOH, where objects are allocated on top of each other with no or minimal possible number of memory holes.

Finally, addresses to the objects stored in reference variables are updated in order to point to the correct, new places in the memory occupied by particular object.

The best gain we get from the compaction is that the SOH is not fragmented, so the memory can be used efficiently. On the other hand it requires some CPU time, which may have some impact on our application’s performance.

By default, LOH is not compacted because of the size of objects stored on it (>85 kilobytes). Copying chunks of such objects would take too much time. Instead, used space and memory holes on the LOH are tracked and the allocation algorithm is adjusted to try to reserve the memory for new objects in the optimal way (by finding the best possible free slots on the fragmented heap).

What’s worth noticing is that starting from .NET Framework 4.5.1, LOH compaction can be enabled on demand by using GCSettings.LargeObjectHeapCompactionMode property.

When does the GC run?

It can be stated that GC works in a non-deterministic manner. It runs on a separate thread when one of the following conditions becomes true:

•  the OS sends low memory notification,
• the memory used by the objects on the managed heap exceeds some defined threshold (which is adjusted dynamically at runtime, based on current allocations),
• GC.Collect() method is called in the code.

If you see GC.Collect() directly used in code, in almost every case it means that there’s something wrong with the application’s implementation. Several great Microsoft engineers and open-source contributors have worked on automatic garbage collection in order to not trigger it manually

The exceptions for this rule I’d accept are maybe some gaming implementations, in which you work with huge objects and sometimes you’d like to affect the way when these objects’ memory is cleaned-up or usage in debugging – most of memory profilers (like dotMemory or ANTS Memory Profiler) force the collection before the memory snapshot is done by the user. This is reasonable, because when debugging memory leaks or issues you wanna see which objects remain allocated after GC does its cleanup.

What’s also worth mentioning is that when the garbage collection is run, all threads are stopped by default, except the thread in which the collection was triggered. However, this can be configured by changing the GC’s settings to work in workstation or server mode, as well as making it run concurrently or in the background. There are different use cases in which each settings combination should be used – you can find more details about it here and here.

Summary

Today we’ve covered the main blocks from which the garbage collection process in .NET is built. We went through marking phase, then the actual collection process – also including heaps compaction. We learnt how the GC roots tree is built and how garbage collector knows which objects it can clean-up during its cycle. We also listed cases in which the collection can be triggered.

In the next post we’ll examine a very interesting concept – generational garbage collection.

See ya next week!

The post [.NET Internals 05] Garbage collection: marking, collection and heaps compaction appeared first on CodeJourney.net.

Ready? Let’s start then!

What (or who) is Garbage Collection (Collector)?

As you could read in the previous posts on internal details of .NET,  there are a lot of conditions and assumption of memory organization in the framework. Data is stored on the proper structures, sometimes boxed and unboxed, but what’s the best about it all is that it’s almost completely invisible for us – developers.

In fact, providing this invisibility is a goal of Garbage Collector (GC). It should become our close friend (it’s not that scary as you could have heard – we’ll see that soon ), because this automatic memory manager does a lot of stuff for us, including:

• allowing us to write code without having to release memory (still remember the meme about C++?),
• performing heap allocations in an efficient way,
• reclaiming the objects that are no longer used and clearing their memory,
• ensuring that every new managed object gets clean content to begin with, so there is no work to be done in an object’s constructor to allocate the memory for all its members,
• providing memory safety, so one object cannot use the memory reserved for another object.

The term ‘garbage collection’ was firstly used in LISP programming language in 1959 and since then it represents the concept of automatic memory management in programming languages and frameworks.

Does GC allocate or deallocate the memory?

There’s often this confusion between allocating and deallocating the memory – what is actually responsible for that? In principle, the goal of garbage collector is to deallocate the memory – as its name suggests, it collects (frees, reclaims) the garbage (not-anymore-needed objects).

On the other hand, when there’s any memory allocation request sent by an application, such query is transferred to CLR, which is actually responsible for allocating the memory. However, as we’ll see in the coming posts, the deallocation of memory (which is much more complex process that allocating it) can be triggered on various conditions and it can happen that allocation of the memory requires deallocation to be performed first. It doesn’t mean that this is a sequence of events (i.e. before allocation the deallocation is performed), but that the number of allocations done by the CLR can affect the way and frequency of garbage collections (we’d like to have available memory when an allocation is to be done).

That’s how the CLR ‘asks’ GC for help (or ‘communicates with GC’ by notifying it about allocations) in actually allocating the memory (for instance by making GC running a collection or heap compaction process more frequently). Because of that reason, it’s usually simplified that garbage collector handles both allocation and deallocation of memory.

As we’ll see below, allocation of memory is as simple as 1 + 2 = 3 – continue reading to get to know why!

Memory allocation in .NET

As we read in the first post of the series, each process has its own virtual address space. Garbage collector is responsible for allocating the memory on the managed heaps (more about heaps here) created in this process’s space. It means that garbage collector only takes care of reference type objects.

Value types are stored on the stack in LIFO order and their memory is reclaimed as soon as the method in which these value types are defined exits. It would make no sense if GC had to manage these theoretically simple and (in a perfect world) locally declared variables.

Next object pointer

To make it simple, as soon as managed heap is created, it contains a pointer to the memory address where the next object will be allocated. This pointer is simply a number and is referred to as next object pointer. Initially, its value is set to managed heap’s base address:

As soon as the first-ever reference object is to be allocated on this particular heap, the memory is allocated in the place where the next object pointer currently points to. State of the heap after first allocation would be something like that:

As presented above, the next object pointer is moved to the address just after the allocated object. It ensures that there are no unnecessary memory gaps on the heap.

Complexity of memory allocation

In reality, the next object pointer’s value is a number in hexadecimal format (e.g. 0xF7279). When a new allocation is to be made, the only operation to perform is an addition operation. It is known how much space is needed for the new object (more details in the next section), so in order to reserve the proper memory chunk, number of bytes necessary for a new object is added to the next object pointer’s current value.

This is how the reservation of managed memory works. Because of that it can be said that allocating the memory for reference types is almost as fast as allocating variables on the stack.

If you’d like to see how you can check the memory address of an object, I recommend checking this article.

How is an object’s size calculated?

In the comments under the previous post, Jelena asked for explanation on how the size of an object is calculated. Here it comes

As we all know from the second post, object is sometimes allocated on the Small Object Heap (SOH) or Large Object Heap (LOH) depending on its size. For that reason before the allocation is made, the object’s size must be determined. How is this size calculated then?

It would make things easy to assume that size of an object is everything it contains (also other objects’ sizes). In reality, it’s not counted this way. Objects contained in another objects are allocated separately on the heap, so their actual sizes are not included in the parent object’s size.

Very good example is given in Chris Farrell’s and Nick Harrison’s book – Under the Hood of .NET Memory Management. If we consider the following code:

That’s how the heaps’ state looks like as soon as as instance of MyClass is created:

The size of MyClass object instance will include:

• general class stuff (e.g. some metadata – you can check it yourself in IL code with ILSpy as we did with boxing and unboxing),
• memory required to store the pointers (memory addresses) to the string and bytes array.

string Test will be allocated on the SOH, and byte[] data will be allocated on the LOH (it’s more than 85K).

In general, the size of an object is calculated by adding sizes of:

• general class data,
• pointers (for reference types class members) containing only addresses in memory (hexadecimal numbers),
• value types (for value types class members) – more details here.

Summary

I hope this post makes the role of garbage collector a bit more clear for you. We’ve also seen how easy the memory allocation is and what’s the general way of calculating an object’s size.

Even we don’t perform the actual allocations ourselves manually and may think like this guy:

We’ll see if that’s true next week, when we’ll dig into more complex topic, the real purpose of the GC – releasing the memory!

Stay tuned!

The post [.NET Internals 04] What is Garbage Collection? Memory allocation in .NET appeared first on CodeJourney.net.

What is boxing and unboxing?

In the previous post we learnt what are value and reference types and got to know that the former are stored on the stack, whereas the latter are stored on the managed heap. So why should we care? Isn’t it .NET runtime which correctly manages these data structures and what’s stored on them so we don’t need to worry about that?

In fact, no. What’s crucial to know and understand are the implications of moving data from the stack to the heap and otherwise.

Remember:

• when any value type variable is assigned to a reference type variable, data is moved from the stack to the heap and this is called boxing,
• when any reference type variable is assigned to a value type variable, data is moved from the heap to the stack and this is called unboxing.

Microsoft Docs examples illustrate these actions very well.

Consider the following example of boxing:

and the memory state as it executes:

In order to store “123” value in an object, the “box” is created on the heap and the value is copied inside it.

On the other hand, when unboxing happens:

that’s how stack’s and heap’s content changes:

Value “123” is taken out of the “box” and placed back on the stack.

Notice that when i value type is boxed into object o, there’s a reference stored on the stack and the actual memory is allocated on the heap (as for all reference types). As soon as the unboxing happens, the real data being on the heap must be copied to the stack (variable j). In both cases our goal is to deal with the same value (123).

As you can imagine, these operations produce some additional cost and affect the performance, which we’ll discuss in a moment.

Let’s see some IL

When analyzing such performance or memory management aspects in our code (C# in that case), it’s often worth to see how the Intermediate Language (IL) looks like.

We haven’t covered this concept yet, but as you probably know when C# code is compiled into DLL or EXE, these output files actually contain the IL code, which is later JIT-compiled and executed by the virtual machine (more details here). The .NET runtime must somehow know whether is should box or unbox the particular variable, as it requires some special memory allocation actions to be taken.

Let’s create some simple .NET Console Application with the following code in its Main method:

Let’s compile the application so we can find BoxingUnboxingTest.exe file in the output directory. Now we will use ILSpy to see the IL code inside the executable.

As soon as the EXE file is opened in ILSpy, we can go directly to see Main(string[]) : void method’s compiled content, choosing “IL with C#” view to make it simpler for us:

Notice the box statement just after the assignment of value type to reference type happens (object obj = num). Similarly for unbox.any statement just after assignment of reference type to value type (int num2 = (int)obj).

That’s how boxing and unboxing is represented in the IL.

When do we box and unbox?

The sample code above may seem naive and you may think “hey, but I never do such things”. That’s true in most cases, but values in our code are often boxed/unboxed even if we are not aware of that.

Non-generic collections

For instance, there still exists the old-school ArrayList collection:

which as you can see above has the Add method taking an object parameter. It means that when we’d like to add an integer to ArrayList:

boxing takes place:

Such issues were eliminated by generics and generic collections.

String concatenation

Another interesting example is the concatenation of strings with value types using “+” operator:

Such operation involves String.Concat method’s version taking two object parameters, so it implies boxing the integer first:

In order to avoid it, it’s enough to slightly modify the code by using ToString() method on the integer variable (for now ignoring ReSharper telling you it’s redundant ;)):

and there’s no boxing anymore, as now String.Concat method’s version taking two string parameters is used:

There could be many more cases to present, but the goal is that you get a strong feeling what boxing and unboxing is and when it occurs.

Performance implications of boxing and unboxing

As we already know, boxing and unboxing imply some cost. In case of simple stuff like concatenating the string once with some integer, the performance gain by introducing ToString on integer first is unnoticeable. That’s why, as I wrote before, even ReSharper tells you to not do it:

In that case it’s better to keep the clarity of the code without ToString method.

The perspective changes when you need to perform such operations in a loop for hundreds or thousands times. In this case the execution of code using boxing can be even 150% longer than its equivalent with no boxing (you can create a simple application and measure the execution time of code with and without boxing or check this article).

Boxed values also take more space in the memory than value types stored on the stack. Copying the value to/from the stack is also a cost. According to MSDN, boxing can generally take even 20 times longer that simple reference assignments, whereas unboxing can be 4 times slower than assignments.

So… why to use boxing and unboxing?

Despite all performance implications boxing and unboxing have, these concepts were introduced into .NET because of some reasons:

• there’s unified types system in .NET, which allows to “represent” both value and reference types in a similar way – thanks to boxing,
• collections could be used for value types before generics were introduced into .NET,
• it simplifies our code, like we saw for string concatenation and in most cases this clarity gives us much more than the performance we’d gain trying to avoid boxing.

Boxing and unboxing are so common that we can’t avoid them. We should be aware how it works in order to be able to minimize their usage, but it should always be considered reasonably. Don’t spend time on optimizing your code, checking IL all the time to get to the point with the smallest possible number of box statements used. Keep in mind that the clarity of your code, its self-explanatory structure and comfort of reading is sometimes much more valuable than some small or barely noticeable performance gain.

Summary

In today’s post we saw what is boxing and unboxing, how it’s represented in the IL code and what implications on performance it may have. I hope it clarifies these commonly mentioned concepts for you, at least a bit

If you’re interested in digging into nitty-gritty, low-level details of boxing and unboxing in .NET, I invite you to read this great post by Matt Warren about even more internal internals of boxing and unboxing

In the next posts in the series we’re going to start exploring the garbage collection. If you have any suggestions or ideas for the coming articles, let me know in the comments (some of you already did on reddit – new posts scheduled ).

See you (or better read you ) next week!

The post [.NET Internals 03] Boxing and unboxing appeared first on CodeJourney.net.

If you’re here for the first time, I encourage you to first read the .NET Internals series introductory post explaining the basics of memory structure.

Memory division

By default, when the .NET application is started and virtual address space is allocated for the process (as we saw in the previous post), the following data structures, represented as heaps, are created:

• Code Heap – storing JIT-compiled native code,
• Small Object Heap (SOH) – storing objects of size less than 85 kilobytes,
• Large Object Heap (LOH) – storing objects of size greater than 85 kilobytes*,
• Process Heap.

*side note: there’s an exception for arrays of double,  which are allocated on the LOH much before reaching 85K (double[] is considered “large” when it exceeds 1000 elements). This is for the sake of 32-bit code optimization. More details here and here.

Division of storing small and large objects on different heaps has its performance reasons, which will be covered in the next posts about garbage collection.

Elements placed on the heap have their addresses, which are a kind of pointers to the place in memory where the particular object is located.

Heap is however not the only data structure used by .NET process. There’s also a stack, which helps in tracking code’s execution and storing some special types of data. We’ll now see in details how stack and heap are used and organized.

Stack

Stack is a memory structure organized as LIFO (last-in-first-out) collection. If you think about it, it’s perfect for storing anything that may soon need to be used (easily popped out from the top of the stack). This nature of stack leads to its two main purposes: keeping track of execution order and storing value types variables.

Keeping track of execution order – call stack

Most of the code we write is wrapped into classes and methods, which can call another methods, which call another methods etc. .NET Framework must all the time keep track of this execution’s order. Moreover, it also has to remember the state of variables and parameters of the method calling another method, while this another method executes (to be able to restore the calling method’s state after the called method returns).

As soon as any method is called, .NET initializes a stack frame (kind of container) which stores all data necessary to execute the method: parameters, local variables and address of the line of code to go back to when the method finishes.  Stack frames are created on the stack on top of each other. Each method has its own stack frame. All that behavior is well illustrated on the figure below.

Stack used for storing the execution order is often referred to as call stack, execution stack or program stack.

Take a look at the following code:

In order to call Method2, the framework must save an execution return address which will be the next line of code to execute after Method2 finishes (line 4 in the example above). This address altogether with parameters and local variables of the calling and called methods are stored on the call stack, as the schema below presents.

You can also notice what happens when Method3 returns (its stack frame is popped out from the stack – it disappears).

Storing value types

Stack is also used to store variables of any of .NET value types, including bool, decimal, int etc. Full list of .NET value types can be found here.

Value types are basically the types which keep the data and memory in the same location. What’s also interesting to know is that all value-typed local variables of the method allocated on the stack are cleaned-up after the method’s execution finishes. It happens because the method’s stack frame becomes unreachable – stack has some pointer to the beginning of the top-level stack frame (current stack frame pointer), which is simply moved to the second stack frame from the top as soon as the method’s execution is finished (the data is still physically there, but it’s not reachable by default .NET mechanisms).

Heap

Heap is similar to stack, but if the stack can be imagined as a series of boxes stacked on top of each other, where we always push or pop the top one, heap contains boxes that can be accessed at any time. The boxes in the heap are placed in different places, not necessarily on the top of each other. To access one of the boxes we don’t need to pop out the top ones first.

Storing reference types

All other variables, which are not value types i.e. are either string or object (or deriving from one of these, so all classes, interfaces and delegates) are referred to as reference types. All reference types are allocated on the managed heap (SOH or LOH, depending on their size). However, even though an object is allocated on the heap, a reference to it (address on the heap) is stored on the stack.

Consider the following code:

The figure below presents how stack and heap would look like here in terms of allocated data:

The “OBJREF” stored on the stack is actually a pointer (reference) to the MyClass object on the heap.

NOTE:  MyClass myObj statement doesn’t allocate any space on the heap for myObj variable. It only creates “OBJREF” variable on the stack, initializing it with NULL value. By the time new statement is used, the actual memory allocation on the heap takes place and the reference’s value is set.

Value types vs reference types (stack vs heap)

The crucial difference between value and reference types is that when a value type variable is passed to another method as parameter or just assigned to another value type variable, its value is copied into the new variable. That’s why when passing a value type to another method, which modifies this variable (parameter), when the method returns the original variable’s value is not changed (it was its copy which was used inside the called method). Figure below shows this behavior.

It’s different in case of reference types. When reference type variable is assigned to another reference type variable (or method’s parameter), it’s not its value which is copied, but only the reference to it (address in the memory where the data is actually stored on the heap). In effect, the new variable (or method’s parameter) still points to the same place in memory, so modifying such reference type variable inside the called method will also be in effect outside of the method. This case is illustrated below.

Of course, storing some data on the stack and some on the heap (moreover: different types of heaps) has its precise purpose, which we’re going to explore in the coming posts

Summary

In today’s post we went through stack and heap, two data structures used for storing value and reference types and tracking execution order of .NET application. In the next posts in the series we will see what is boxing and unboxing and how it can affect application’s performance. We will also examine the garbage collection process in details.

Stay tuned!

The post [.NET Internals 02] Stack and heap – .NET data structures appeared first on CodeJourney.net.

Ever been surprised that there’s no need to worry about memory allocation and deallocation using high-level programming languages such as Java or C# after leaving the university* ?

Still remember (old) C++ times with delete statement?

By this post, I’d like to introduce a new “.NET Internals” series on the blog. I will be publishing a new post on .NET internal concepts every Wednesday. No end date for the moment

Posts within the series will be mostly about memory management and performance aspects of .NET applications. All discussed concepts are applicable to most of the modern programming platforms, but the examples will be based on .NET Framework.

If you’re interested in such topics, I encourage you to check this blog every Wednesday starting today

*there is 1 assumption and 1 fact here:

• assumption: you used to program in C/C++ (before C++11) on the university,
• fact: that’s not true you don’t need to worry about memory management using high-level programming languages; wait for the next posts to get to know why.

CLR

Each application targeting .NET is managed by CLR (Common Language Runtime), which is a part of .NET Framework, which must be installed on the computer on which the .NET application is launched – the exception are .NET Core self-contained apps, which we are not discussing in this series. By default, the CLR is loaded by each .NET process as a hosted process (so the CLR runs as a part of .NET application).

This CLR “layer” is responsible for various things while our application is running:

• executing CIL (Common Intermediate Language),
• ensuring type safety,
• handling exceptions,
• managing threads,
• garbage collection.

Memory structure

Already running application is represented as a process in the operating system. As software developers we know that such process saves some data into the memory, which is determined by the amount of RAM available on the computer. However, process never operates on this “open memory” – only OS has direct access to it. When the CLR is launched within .NET process, it requests some amount of RAM from the operating system. This memory chunk is called virtual address space. Each process has its own virtual address space, but all processes running on the same machine share the same physical memory.

32-bit OS

Do you remember 32-bit editions of Windows XP, which could only use 4GBs of RAM (with even less visible to the user)? The reason for that is that in 32-bit operating systems total virtual address space is 4GBs, usually divided into two equal chunks – first one owned by the OS and the second one prepared for user processes. The diagram below presents.

64-bit OS

In 64-bit Windows theoretical amount of virtual address space is 16 exabytes (2⁶⁴ bytes). In reality, only a part of it is used: 8-terabyte chunk is used for user space and portions of 248-terabyte are used for the OS:

Sharing memory between processes

What seems to be critical to understand is how this whole virtual memory is shared between processes. As we said before, multiple processes share the same memory, but accessing different (separate) chunks of it. Let us illustrate once more:

As can be seen, each process reserves different pages of memory when it’s needed. Even though the default 2GBs of virtual memory reserved for a single process may seem to be a lot, every developer needs to be aware that when an allocation (reservation of memory chunk) is made (manually or by memory management system) the virtual memory manager needs to find a single, continuous virtual memory block large enough to store the requested amount of data. This rule leads to the presence of “holes” in the memory. This virtual memory fragmentation is well-presented on the diagram below:

However, this requirement of finding a continuous block to be allocated refers only to the virtual memory. Physical memory is divided into pages, which can be stored in totally different places – as you can see on the diagram above with Notepad.exe application using continuous blocks of its virtual memory, but committed to noncontiguous pages scattered in various physical locations.

If the process requests for too much memory, which cannot be continuously allocated in the virtual address space (your process is out of virtual memory) or no more virtual memory can be committed (assigned) to the physical storage, it can result in OutOfMemoryException thrown by .NET code.

Summary

This was the introductory post of my new .NET Internals series. We saw what are the basic concepts of the memory structure, learnt what is virtual address space, how physical memory is shared between processes and why Windows XP was able to handle only 4GBs of RAM

See you next Wednesday!

The post [.NET Internals 01] Basics of memory structure appeared first on CodeJourney.net.