In this article, I’m sharing a compiled list of the most popular .NET interview questions I was asked during all those interviews. They represent real questions asked on job interviews for a software developer with almost 10 years of experience. Hope you enjoy it

Technical .NET Interview Questions

1. What’s the difference between a class and an interface?
2. What’s the difference between an abstract class and an interface?
3. What is the difference between a class and a struct?
4. Describe the difference between overriding and overloading in C#
5. Can a structure have virtual methods?
6. Explain the difference between GET, POST and PATCH in HTTP APIs
7. Is there any limit go the GET request’s length? If yes, where is such a limit configured? What about POST?
8. How would you secure your WebAPI?
9. What is the flow of generating, obtaining and passing JWT token in a .NET application?
10. When using JWT tokens in an API, how does the server ensure it talks to a legitimate client?
11. Is the API security compromised when the user’s JWT token is stolen?
12. What is the difference between IEnumerable and IQueryable in the context of Entity Framework?
13. What are the dependency injection scopes in ASP.NET?
14. How can dependency injection be helpful in testing?
15. What is IHttpClientFactory used for and which issues does it help to eliminate?
16. What is wrong with creating HttpClient manually?
17. What would you choose for implementing communication between microservices: REST or a message broker? What would you consider when making such a choice?
18. What would you do as a backend developer when frontend developers are blocked because they are missing an API endpoint from your side?
19. What is the size of stack in .NET?
20. What are value and reference types? What is the difference between them? Give a practical example
21. Which objects go on Large Object Heap?
22. What is JIT compilation?
23. Describe the strategies for delivering a message in a message queue
24. When is Dispose method called on a class implementing IDisposable?
25. What is the difference between Dispose and Finalize?
26. What is SOLID? Can you elaborate on each SOLID acronym?
27. You are assigned a task to process a large piece of text by inserting and changing many parts of it. Which of the following would you use for that task: string, StringBuilder, StringWriter or StringReader?
28. What’s the difference between Repository and Query Object patterns?
29. What is the Saga pattern?
30. What is CQRS and in which scenarios may it be beneficial?
31. What is the difference between an event and a command (like in CQRS)?
32. What are the advantages and disadvantages of using a Repository pattern?
33. What is Clean Architecture? Who invented it and how it works?
34. What is Onion Architecture?
35. What is Modular Monolith?

Experience-related .NET Interview Questions

36. Describe how you realized authentication & authorization in your last project
37. Explain your last project’s architecture
38. Explain a challenging task you faced in your last project and how you solved it
39. How would you approach diagnosing a performance issue on production, which cannot be reproduced locally?
40. Explain the “event-based architecture” you implemented in your last project. This was something I explicitly mentioned in my resume, so the advice is: prepare to elaborate on every experience you mention in your resume
41. How would you choose tech stack for a new full stack web application (backend + frontend)? Describe your decision-making process: what would you take into consideration when choosing a tech stack?
42. How would you decide whether to use an already-existing component (e.g. a library) vs implementing your own?

Final tips

These were not all of the questions I was asked, but the ones that recurred most frequently. As a programmer with almost 10 years of professional experience, I wasn’t asked many basic programming questions. The recruiters were mostly focused on my experience. They wanted to know how I solved problems at work in the past and what’s my approach to tackling issues. Rather than asking me theories about a design pattern, they wanted to know how I applied it in practice and whether I was happy with deciding to use it in the first place.

My general advice is to be patient and iterate. After each technical interview, try to write down as many questions as you can remember. This will help you be better and better prepared with each interview you take. Don’t forget to update your LinkedIn profile and keep building a quality network, even if you’re not on the job lookout right now. It comes in handy!

If you happen to be in a work search right now, I keep my fingers crossed! I hope these .NET interview questions will help you get that dream job

If you’re interested in knowing something about programming job search in current market conditions, feel free to leave a comment

The post Real .NET Interview Questions (2024/2025) appeared first on CodeJourney.net.

If you ever had an API action defined like that in your controller:

and fetched this data using TypeScript in the following way:

at some point, you probably also experienced the desynchronization of backend (C#, in our example) and frontend (TypeScript) types definitions. What if someone has changed the C# version of UserViewModel, but no one corrected its TypeScript’s equivalent?

Your TypeScript fetching code will tell nothing about that. There will be no error, even though the fetched data doesn’t match the expected UserViewModel type.

I’ll try to address this issue in this article Let’s see how typing API responses with zod can help us here.

Synchronization of Backend and Frontend API Typings

First, why would we want to keep the backend and frontend models in sync?

For me, that’s the purpose of using TypeScript. We want our code to be as well-typed as possible. For instance, we normally want the data displayed to the user to be fully typed. TypeScript enhances our programming experience by providing us with typing information. Thanks to that, we know what is what and what contains what. We also express what types of data we expect in particular cases.

The APIs mostly return JSON data, which can be anything. Because of that, it is much easier to have the data returned from the API fully typed in TypeScript. Thanks to that, we know what properties are available on the data models received from the API and whether we can use and display them to the users.

The sample code used within this article is available on GitHub. We will use ASP.NET Core (C#) and React (TypeScript) apps as examples.

Models synchronization example

As we saw in the beginning, a classic example is an API controller that returns a strongly-typed data:

The returned data type is a collection of UserViewModel objects. Here’s the C# definition of this type:

Its equivalent is also defined on TypeScript side:

Usage in TypeScript

Cool. With this simple code, we can create a usersService.ts file and fetch our users’ data from the API. Notice how we make this call strongly typed:

Everything looks legit. We can use the data retrieved from the API in UsersList component and everything is nicely typed:

The data is even perfectly displayed:

So, what can go wrong here?

The Problem – Typings’ Desynchronization

Let’s say that a backend developer implements a requirement to rename “loyalty points” into “fidelity points”. Easy. (S)he renames LoyaltyPoints property in the C#’s UserViewModel to FidelityPoints.

The new C# model looks as follows:

Nice! The backend dev is a very good programmer, so (s)he even launches the React web application to make sure that everything still works correctly and there are no errors in the dev console:

After a quick look, everything looks awesome. The users list is displayed, there are no errors in the console. Apparently, these test users don’t have any loyalty points assigned – that’s why the empty values in “Loyalty points” column. What’s more, translators will update the column’s translation later. We are good! Let’s go on prod!

I guess you already know what went wrong here. API definition changed, but TypeScript didn’t inform us about that Our UserViewModel still uses the old property name:

However, it still works. When rendering the UsersList, we simply get undefined in place of loyaltyPoints:

In the end, this is all JavaScript there. What’s interesting, the renamed fidelityPoints property is already there at runtime:

but no one cared about it

With the current solution, we will never be informed soon enough about API models changes in our React application. In the best case, we’ll get an undefiend or null error when clicking through the app. However, it’s usually an end user who finds such problems on production. This is definitely not what we want

We can solve this problem by typing API responses with zod. Let’s now see how to do that.

The Solution – zod

Our remedy – zod – is quite a decent npm package with with ~600k weekly downloads. Its GitHub page advertises the library as TypeScript-first schema validation with static type inference.

You can definitely do many things with zod. It can be used together with libraries like react-hook-form to perform complex forms validation. However, in our case, we’ll treat zod as a solution for better typings in TypeScript.

Adding zod to React app

First, let’s install zod into our React application:

npm i zod

First schema definition with zod

With zod, we define our types in a slightly different way. Instead of creating a type or interface directly, we first create a schema. In our case, we can define a UserViewModelSchema using z.object creator function:

Few interesting parts here:

• Line 2: notice how zod helps us to define types like Guid with built-in schemas like uuid()
• Line 8: first, I used AddressViewModelSchema here. This is a custom schema of an AddressViewModel object, which is another type used internally in UserViewModel. You can use such custom schemas in other schemas. Also notice the nullable() call here, which makes the address property nullable

First step done – we have our UserViewModelSchema. But can we use it instead of UserViewModel type? Not really. Schema is used for validation purposes only. We still need the UserViewModel TypeScript’s type.

Inferring type from zod’s schema

Fortunately, zod comes with a handy z.infer function that allows us to infer the type from the schema.

Finally, the userViewModel.ts file looks as follows:

We can use the exported UserViewModel type as previously used type. It’s an equivalent, inferred from UserViewModelSchema.

Validating API responses with zod schema

One last step is to make use of UserViewModelSchema. Let’s modify the getAllUsers function from usersService to validate the data received from the API against our schema:

Notice the usage of z.array. This function call tells zod to validate an array of objects meeting the rules defined by UserViewModelSchema, not a single object.

Now, let’s run our React app and see what happens when we click the “Fetch users” button:

This is awesome! Exactly what we wanted – a schema validation error for API response. Notice how the error message precisely points to the missing (or wrong, in other cases) property. It tells us we expected a number called loyaltyPoints, but instead we received undefined. The reason for this error message is that the loyaltyPoints field is Required in our schema.

After renaming loyaltyPoints to fidelityPoints in UserViewModelSchema and updating the UsersList component accordingly, everything works fine again.

We are now fully typed and prepared for the future, in case an issue with desynchronization of frontend and backend typings happens again

Summary

Today, we’ve seen how typing API responses with zod can help us detect frontend and backend models desynchronization. Schema validation throws errors when the data doesn’t match its expected shape.

Remember that zod is an extended library with many options. I recommend exploring them on your own. An interesting feature we didn’t cover in this article is strict mode, which doesn’t allow additional fields not present in the schema definition when validating the data object.

The open question remains whether to use schema validation on production. One could think that it’s better to not throw any validation errors, because JavaScript may just work. However, I think that throwing an error is always better than silently letting things through. An error lets programmers, automated tests or manual testers to detect the issue before the end user does

You can explore the complete code presented in this article here.

The post Typing API Responses With Zod appeared first on CodeJourney.net.

NHibernate.Exceptions.GenericADOException: could not load an entity: 
...
 ---> Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myappdb' on server 'myapp.database.windows.net' is not currently available.  Please retry the connection later.  If the problem persists, contact customer support, and provide them the session tracing ID of '{EB501CF2-2F21-4E28-9042-2B83EEE57B91}'.

The database is not currently available was very interesting to me. I struggled a bit with solving that, so if you want to know how I did it – continue reading

The problem

My .NET 6 app is an Azure App Service, and it uses SQL Server database. Quite standard Azure stuff.

The issue was that when opening the ASP.NET web app after some time of inactivity, there was an HTTP 500 error. When I checked the logs, the full error looked as follows:

NHibernate.Exceptions.GenericADOException: could not load an entity: [MyApp.Infrastructure.DomainObjects.ApplicationUser#37931c5a376748d7a49f215ca30283a2][SQL: SELECT applicatio0_.applicationuser_key as id1_3_0_, applicatio0_1_.UserName as username2_3_0_, applicatio0_1_.NormalizedUserName as normalizedusername3_3_0_, applicatio0_1_.Email as email4_3_0_, applicatio0_1_.NormalizedEmail as normalizedemail5_3_0_, applicatio0_1_.EmailConfirmed as emailconfirmed6_3_0_, applicatio0_1_.PhoneNumber as phonenumber7_3_0_, applicatio0_1_.PhoneNumberConfirmed as phonenumberconfirmed8_3_0_, applicatio0_1_.LockoutEnabled as lockoutenabled9_3_0_, applicatio0_1_.LockoutEnd as lockoutend10_3_0_, applicatio0_1_.AccessFailedCount as accessfailedcount11_3_0_, applicatio0_1_.ConcurrencyStamp as concurrencystamp12_3_0_, applicatio0_1_.PasswordHash as passwordhash13_3_0_, applicatio0_1_.TwoFactorEnabled as twofactorenabled14_3_0_, applicatio0_1_.SecurityStamp as securitystamp15_3_0_ FROM ApplicationUsers applicatio0_ inner join dbo.AspNetUsers applicatio0_1_ on applicatio0_.applicationuser_key=applicatio0_1_.Id WHERE applicatio0_.applicationuser_key=?]
 ---> Microsoft.Data.SqlClient.SqlException (0x80131904): Database 'myappdb' on server 'myapp.database.windows.net' is not currently available.  Please retry the connection later.  If the problem persists, contact customer support, and provide them the session tracing ID of '{EB501CF2-2F21-4E28-2342-2B836E35E891}'.
   at Microsoft.Data.ProviderBase.DbConnectionPool.CheckPoolBlockingPeriod(Exception e)

There were two characteristics of this error:

• it always appeared after some time of not using the web app
• I never reproduced it locally

So it must have been something wrong with Azure.

Transient issues in Azure databases

First thing I did, obviously, was checking Azure App Service logs. This is what I found:

The first recommended step got me curious. To reduce the impact of connection issues caused by future reconfiguration, please implement retry logic in your code.

So I started reading. There’s even official Microsoft documentation for that. It basically says that in the cloud environment you’ll find that failed and dropped database connections happen periodically.

I must admit it seemed quite new to me. However, I started digging into that. I used Polly to implement the retry logic in my app. I found few places where there was an NHibernate query for fetching users executed and wrapped it into the retry policy.

Nothing worked. I was still getting the error. Then, after some talks with my coworkers, we found the real reason behind that…

Solution: Azure database tier

When talking about the issue with my teammates, someone mentioned that the databases in Azure are serverless by default.

I logged into the Azure Portal and checked the “Compute + storage” settings my SQL database my app used:

Indeed, the tier selected was “General purpose (Scalable compute and storage options)” with “Serverless” compute tier selected.

I changed it to “Basic (For less demanding workloads)”:

and… it worked All errors saying that the database is not currently available disappeared. I also removed all Polly-related code for retry policies.

A little explanation

Why do serverless databases in Azure behave like that?

I finally found this explanation in the Microsoft documentation:

If a serverless database is paused, then the first login will resume the database and return an error stating that the database is unavailable with error code 40613. Once the database is resumed, the login must be retried to establish connectivity

That blew my mind! It means that the database is not currently available exception is expected. There’s also another part there:

Database clients with connection retry logic should not need to be modified. For connection retry logic options that are built-in to the SqlClient driver, see configurable retry logic in SqlClient.

Apparently, users of ORMs like Entity Framework should not have to worry about those issues. Unfortunately, NHibernate doesn’t have such built-in transient errors retry logic. That’s why I experienced this particular issue and couldn’t find any straightforward solution online. Simply not many people (if anyone at all? ) experienced this before.

If you want even more details and context, check out this article: SHOULD I USE SERVERLESS FOR ALL MY AZURE SQL DATABASES?

What if I want to use serverless Azure database?

Well… if you really want to use the serverless features of Azure, you might be struggling with NHibernate. As I mentioned, apparently there’s no built-in transient failures retry mechanism in the ORM. There’s one promising solution here. However, currently it’s build for NHibernate 3.3.1.4000, so it doesn’t work with the current version. There’s a Pull Request and an issue opened for upgrading it to the newest version and .NET 6/7. Let’s hope it is done soon.

Otherwise, you can try using Polly as I did initially. However, I didn’t manage to make fully working.

If you have any other idea or solution – let us know in the comments below.

Conclusion

I hope this article saves you some hours of debugging, so you can quickly solve this database is not currently available error in Azure. I wish I found this article on another website few weeks ago

The post How To Fix SqlException Database Is Not Currently Available On Azure (0x80131904) appeared first on CodeJourney.net.

How will this possibility change the way we write C#? Why is it being introduced? I’ll try to address these questions today

Interfaces today

As we all know from the current version of C#, interfaces allow to define a kind of contract, which must be respected by a class implementing it. Today interfaces can only contain definitions of methods (with no body). These methods are then actually implemented by concrete classes. If a class doesn’t implement all of its interface’s methods, compiler produces an error.

What’s problematic in this approach? Assuming that you have the following interface and class(es) implementing it:

With time, you may want to extend IDeveloper interface by adding a new method, for instance:

Now you have a problem, because the new method must be implemented in all places where the interface was used. Compiler gives you an error as long as it’s not done:

If your interface is used in many places, also by another teams or developers (which is often the case in reality), this is a breaking change

It may be really painful, depending on how extensively your interface is used. In any case, all classes based on the IDeveloper interface must implement the new LearnNewLanguage(string language) method. Only then the code compiles.

One of the reasons for introducing into C# default interface methods is to avoid such breaking changes scenarios

Default Interface Methods

As we can read in the feature’s design notes, an inspiration for it is a similar concept already alive in Java – Default Methods.

The main idea of this new C# feature is a possibility to provide a default implementation of an interface’s method.

Realization in C# 8.0

In our example using C# 8.0 we’ll be able to implement it as follows:

In that case, our class implementing the IDeveloper interface can legally not implement this new method and there will be no compilation error:

Notice that it only works when BackendDev is contextually treated as IDeveloper. It means that a class doesn’t inherit members from the interfaces it implements:

What about multiple inheritance?

As some of you probably realized, this new feature exposes C# to a multiple inheritance problem (also known as Diamond of Death). So far C# didn’t have this issue, because we can only inherit from a single class. We can implement multiple interfaces, but so far the interfaces couldn’t provide methods’ implementation. Because of that, there was no Diamond of Death possible. Soon it’s gonna change

Diamond of Death in C# 8

In case of C# default interface methods we may illustrate this issue with the following code:

Please note that the syntax or places in which compiler generates errors may still change in the final feature’s version.

To better illustrate this Diamond of Death example, take a look at the following diagram:

The question is: which method should be called in such case?

Solution: most specific override rule

C# 8.0 will solve this issue by introducing the most specific override rule. It means that in our case illustrated above, the compiler will issue an error and won’t let us do this.

To avoid this ambiguity, we’ll be forced to implement or override LearnNewLanguage(string language) in IFullStackDev interface. Only then it will be considered the most specific version of the method in the interfaces hierarchy we have. It will then be used when calling the method on objects contextually treated as instances of IFullStackDev. We could also implement it directly in the Dev class and use the class (not interface) type. It would then call the most specific method from the class. This way multiple inheritance issue will be eliminated.

Compiler for the rescue

There are a lot of other potential ambiguity scenarios that may take place after introducing into C# default interface methods. You can check all of them with potential solutions well-discussed on GitHub.

The general assumption is that all code which may lead to issues related to default interface methods implementations is detected by the compiler. In effect, we should get appropriate errors when compiling the source code. That way, programmer can fix all potential conflicts directly when these arise.

What about .NET Framework?

What’s interesting with default interface methods is that it requres changes in the runtime. According to current assumptions, it means that this feature will not work with .NET Framework! Only CoreCLR and Mono stack runtimes are going to receive these new updates.

Summary

As the motivation for introducing into C# default interface methods the language team members mention possibility to interact with some Android and iOS APIs, which already support such feature. They also claim that C# will get the traits programming language feature with default interface methods.

In my opinion this is the most controversial from all C# 8.0 new features. It completely changes something which was obviousness for the developers for years. Probably we’ll see less breaking changes issues when the interfaces are extended, but looking on the list of issues and edge cases it produces… Maybe assuming it already works in a similar way in Java we should just accept it?

Another interesting, .NET-specific aspect is that this feature will not work with .NET Framework. It’s probably the very first moment when the difference in pace between .NET Core and Framework is so explicit.

I’m also very curious about the performance implications of default interface methods. I’ll for sure be following this issue on GitHub to see how it evolves

What do you think? Is it a good idea to introduce default interface methods into C#?

The post C# 8: Default Interface Methods Implementation appeared first on CodeJourney.net.

Let’s see how this tiny feature is supposed to make our life easier

Goal of slicing

The main purpose of introducing slicing into the language is to make working with arrays and strings easier.

Currently, when you want to retrieve a fragment of an array (X elements from the array, located for instance in its middle), you can use LINQ, i.e. combination of Take and Skip methods. You can also create some extension method to provide kind of slicing.

To illustrate the current way, using LINQ to retrieve only 2nd, 3rd and 4th elements of an array looks as follows:

C# community and designers decided it’s not convenient enough, so are now adding to C# 8 indexes and ranges (in order to provide slicing)

Index

C# 8.0 comes along with a new object – System.Index. It’s a structure internally and looks as follows:

What’s interesting here, is that the constructor takes the value of the index and boolean flag fromEnd. Let’s see how we can use it in practice:

While the first printed value is nothing special, as you can see we can now use the hat operator (^) in order to index from the end. The following also works:

It means that the ^ operator is a syntactic sugar for Index. I allowed myself to check the IL produced:

As we suspected – an instance of Index is created, passing 3 as value and true as fromEnd (see the constructor parameters above).

Range

Another concept which uses Indexes is a new structural data type – System.Range:

As you can see, it really uses Indexes. We can even try creating it on our own:

This doesn’t really make extracting data from the arrays better than Skip-Take LINQ, does it?

Fortunately, there’s another friendly C# language element introduced called range expression. It can be written as x..y and used directly for indexing:

This looks much better! Let’s see how C# compiler translated our 2 lines of code by looking at the pseudo-C# IL produced:

Even though we could achieve the same with much less LINQ, C# compiler does a lot of stuff for us, keeping our code simpler to grasp.

Ranges in strings

All these operations apply also to strings:

Ranges in foreach loop

Next interesting usage is in the foreach loop:

Summary

I see these “little” C# 8 indexes and ranges as another helpful feature. It’s less breaking and controversial than nullable reference types, still making us write less C# code in the future.

If you’d like to dig the feature even more, I encourage you to read the discussions about it on GitHub, which I always find very interesting

Developers, what do you think about C# 8 indexes and ranges?

The post C# 8: Slicing with Indexes and Ranges appeared first on CodeJourney.net.

Learn about building C# 8.0

C# 8.0 is going to be the next major release of C# language. It should be released by the same time as .NET Core 3.0.

What I find very interesting is C# language GitHub repository. Among others, you can find there very interesting notes from C# language design meetings If you filter well, you can also list so called “proposals”, which are current candidates for new C#’s version (or sub-versions) features. By clicking here you can list C# 8.0 candidates.

I will publish a separate post regarding each feature which I find particularly interesting

Nullable reference types

First of the C# 8.0 features I’d like to discuss are nullable reference types. It seems it’s going to change the way we write our code, so it’s worth knowing this feature.

A bit of history – NullReferenceException

Consider the following code:

Guess what happens after executing this as a console app? Of course, our favorite NullReferenceException is thrown:

In this case, jsDev.LastName is of course set to null.

NullReferenceException has been a nightmare for thousands of developers (and users) for many years. What if, in the cases like our example, we knew before that in this place our object might be null?

New world – #nullable enable

For that purpose, C# development team is introducing nullable reference types. In essence, it makes reference types (like string) non-nullable! However to not be a tough nut to crack, all issues related to assigning null value to non-nullable reference types are presented as warnings, not errors.

In order to use this feature, you need to enable it. You can do it with a preprocessor directive #nullable enabled. We can do it directly in the source code file. In the future, it should also be possible to enable it on project level.

Notice what happens when we enable it in our example:

We’re still able to compile, but this nice warning is displayed As soon as we initialize LastName property in the constructor, the warning disappears:

If you try to cheat, it still works like a charm :

But hey… what if you actually want null in this particular string? You need to explicitly mark your reference type variable as nullable. You probably recall nullable value types like int?, don’t you (read this post if you don’t remember the difference between value and reference types)? Now you can do the same with non-nullable reference types:

Nonetheless, in such case, the compiler still takes care of us:

The final fix is quite obvious – we need to check for null first:

At this stage all warnings finally disappear. You can find the finished source code here.

Nullable reference types in IL

The only thing I found in the IL code compiled from source code with #nullable enabled directive was NullableAttribute applied to properties:

It seems that nullable reference types are implemented by applying this attribute to variables, which is only used by the compiler (runtime is not affected).

Summary

I’m really excited about nullable reference types in C# 8. I personally feel that it even introduces some clarity in the language – we’ll have explicit declarations for nullable value and reference types. However, in more complex scenarios it may bring some confusion in regards to reference types and using “?” character (like generics or collections – see Jon Skeet’s post for more details). For sure it’s a significant step towards eliminating issues with exceptions caused by null references.

What do you think about nullable reference types? Is it a good direction in which C# language is going? Share your thoughts in the comments!

The post C# 8: Nullable Reference Types appeared first on CodeJourney.net.

By execution model I mean a process of having a .NET Framework application actually executed on the machine (CPU), starting from having its source code written. It contains all steps and actions necessary to happen in order to transform source code (like C#) into machine (assembly) code and execute it.

.NET languages standardization

As we all know, .NET is just an execution platform. There are, however, a lot of programming languages which allow us to write code which can then be executed by the platform-specific (not language-specific!) CLR. In order to give programming languages’ creators a set of rules which their language and its compiler must meet to be .NET-compliant, Microsoft defined a Common Language Infrastructure (CLI, also referred to as MSIL in the old days ) which is also standardized by ISO and ECMA.

The main goal of it is to define how the source code should be compiled to the Common Intermediate Language (CIL). As we already know, the CIL should have a standardized format, because it’s then JIT-compiled to a platform-specific assembly code. Some of the most common implementations of CLI are .NET Framework, .NET Core and Mono. I recommend you to check Matt Warren’s article on .NET Runtimes history as well

Common Language Infrastructure implies also some other aspects of the programming language and its compiler, including Common Type System, language-agnostic metadata and Common Language Specification. You can read more about it for instance here.

This all allows us to use various programming languages, like C#, VB.NET, F# or JScript .NET to develop .NET applications. Anyone can still implement their own own .NET language – it “just” needs to conform to the standards.

There’s also another term introduced somewhat around .NET Core – .NET Standard. Its purpose is to allow code sharing between various .NET implementations. It actually defines a set of APIs that each .NET platform should implement. More details here.

This whole standardization makes us arriving to the .NET applications execution model, which is how every standards-compliant CLR application is executed.

.NET execution model

Independently of the language of choice, .NET application’s execution model can be described as the following 4-steps process:

1. Writing source code in a programming language of choice, including the usage of its compiler
2. Compiling source code to Common Intermediate Language (CIL)
3. Compiling CIL into native code – details of this step in the previous post
4. Execution of the native code.

These steps are well-resented on the schema below and then described in details in the next sections.

Writing source code in a programming language of choice

As mentioned before, .NET applications can be implemented using any language of choice, which is compliant with CLI. Another important part is a compiler – it defines a general syntax of the language, kinds of data types that can be used by a programmer etc. For instance, current open-source compiler for C# and Visual Basic is Roslyn.

In the end, role of the compiler is to transform the source code to its CIL equivalent.

The simplest possible complete application written in C# can look as follows:

Compiling source code to CIL

After the source code is written, it must be compiled to CIL, which is an intermediate language understood by the CLR. On this level, the whole execution process becomes language-agnostic, which means that as soon as the code is compiled to CIL it is executed by the CLR and programming language’s features (e.g. its compiler) don’t play any more role.

CIL code looks more like assembly code, however containing some specific instructions. It’s still far more readable than native code. Apart from the direct transcript of source code, CIL contains metadata about the DLL/EXE assembly file.

The C# fragment presented in the previous section compiled to CIL by Roslyn looks as follows:

A bit more than the original source code, isn’t it?

Firstly there is a metadata instructions section which contains information about the assembly, execution framework’s version, types used in the code and external references. Next there’s the actual CIL code (equivalent to C# code presented above) listed.

Compiling CIL into native code

We’ve already covered this step in details in the previous post about Just-In-Time compilation, which is just a bit more fancy name of this process Now you can see how and where it fits into .NET application’s execution model.

Normal JIT compilation

By default, JIT compilation is done in the place where “Normal JIT Compiler” is drawn on the schema below:

IL instructions of a particular block of code, in the example diagram of a method Foo(), are JIT-compiled when the block is to be used for the first time. Then, this JIT-compiled code is stored as the JIT-ed equivalent of Foo() method’s code in the memory cache. Such native code is then executed and if the same block of code needs to be called again in the future, it will not be re-JIT-compiled, but instead the native code will be taken from the memory cache and executed again.

There’s a possibility to skip adding the native code into memory cache for future executions and it’s referred to as “Econo JIT Compilation”, however it’s not widely used and probably obsolete since few versions of .NET, so we won’t examine it here.

Pre-JIT compilation

The schema presented above changes a bit as soon as we want to use so called Pre-JIT compilation. We mentioned it already and we know that one of the techniques to allow pre-JIT (or ahead-of-time) compilation is using Native Image Generator (ngen.exe) which allows to transform CIL assemblies into native code files and to store them as a file on the disk (in the Native Image Cache). It allows to natively compile the whole assemblies – it doesn’t allow to compile only single methods as JIT compiler does.

Refer to the new version of the schema presented below to see how the process changes with NGen.

Normal and ahead-of-time CIL compilation modes provide different cons and pros, including:

• NGen delivers faster start-up time, especially in large applications used by many users in the same time, but requires more disk space and memory to store both the CIL and pre-compiled images,
• JIT compilation at runtime can deliver faster code, because it targets the current platform of execution; NGen produces the native images with instructions that can be executed on all possible platforms, meaning it must use the oldest from currently used instructions sets to be backwards-compatible,
• JIT is able to dynamically re-compile the code for a better performance depending on the execution conditions (e.g. by detecting a hot path in the executed code).

Choosing one of the modes depends on your use case.

Execution of the native code

Finally, the code blocks are run by the CLR, which asks CPU to execute compiled instructions sets. If you’re interested in how CPU exactly works and executes the instructions, I recommend you watching this YT video.

Summary

We’ve seen today how the .NET applications are executed step-by-step, starting from source code (like C#) up to being actually executed by the processor.

I also encourage you to read – if you haven’t already – all the previous posts in the .NET Internals series. Just click here to see them all

Let me know if you have any doubts or thoughts, I’m always open to your comments and opinions Especially the constructive ones pointing out some parts I omitted or didn’t describe precisely enough.

I hope it helped you learn something new

The post [.NET Internals 10] Application execution model appeared first on CodeJourney.net.

What makes managed applications portable?

Intermediate Language (IL)

Viewing MSIL

Just-In-Time (JIT) compilation

How Toyota brought us JIT?

Let’s start by some historical background In the early 1960s, Japanese engineers at Toyota had to reorganize their warehouse management, because they had very high storage costs – delivery of the parts from suppliers took a lot of time, because they ordered a lot in advance. Every delivery had to be handled by someone, so they needed a lot of employees. Ordered parts were stored in the warehouses for a long time, required a lot or storage space and maintenance.

JIT compiler

Following Toyota’s success , CLR creators introduced a just-in-time (JIT) compiler into the framework. Its role is to compile intermediate language code into assembly language according to hardware and operating system’s characteristics (depending on the machine on which the code is being JIT-compiled). Unlike unmanaged languages, in which the source code is compiled into machine language prior to program’s execution, IL is (by default) compiled to CPU instructions at runtime. That’s how .NET engineers made the machine code delivered on time to the CPU.

JIT compiler is a part of CLR (similarly to garbage collector). To make things simple, it can be said that JIT compiles a particular method’s source code as soon as this method is called for the first time. Of course, it takes into account the hardware, e.g. the CPU type and its instructions set to produce correct assembly code. Such machine code is then saved in the cache and re-used every time the same method is called again within the same application’s execution. It means that the parts of the code which are not used (called) during the program’s execution may not be JIT-compiled at all.

Just-In-Time compilation is actually a combination of two approaches: ahead-of-time compilation and interpretation, mixing advantages and disadvantages of both. However, compiling IL code into assembly one at runtime allows for several interesting features like dynamic typing (ability to retrieve the actual object’s type at program’s execution). That’s why JIT compilation is often treated as a form of dynamic compilation.

Another advantage of having JIT compilation is that the source code we write is “further from the hardware”, so it can be made more readable and human-friendly, whereas the metal-touching stuff is “hidden” in the IL (kudos to MSIL developers!)

Can we still compile ahead-of-time?

Summary

In one of the next posts we’ll dig into application execution model in .NET Framework, where we’ll see when and how exactly the JIT compilation process fits into the whole C#->CPU journey

so if you have your own opinion on JIT or any other remarks, don’t bother and share it in the comments!

The post [.NET Internals 09] Just-In-Time (JIT) compilation appeared first on CodeJourney.net.

(De)allocating objects on LOH

As we know from the second post, during allocating the memory only objects of size greater than 85,000 bytes are placed on LOH. There are also some exceptions, like arrays of double which are put on LOH (in 32-bit architectures only) as soon as they reach 1000 (or more) elements (not something around 10626 elements as could be expected). This is quite important to know to be aware what kind of objects have impact on heap fragmentation (more details below).

So we know when the objects are allocated on LOH, but when are they deallocated?

LOH is collected in the same time as the generation 2 collection occurs. It can be triggered if memory threshold for either gen 2 or LOH is exceeded. Conditions for garbage collection can be found in this post.

That’s why keeping large LOH may affect the GC’s – and the whole application’s – performance.

Garbage collection on LOH

LOH fragmentation

The reason why Large Object Heap is not compacted (by default) is because it’s used to store big objects (>85,000 bytes). Copying such amounts of data would seriously incur the performance of garbage collection process.

Anyway, memory of objects allocated on the LOH is reclaimed so it may eventually become fragmented:

We’ll see below how, but .NET keeps track of “Free space” memory blocks to know which chunks are available for new allocations on LOH. When allocating it looks for a block large enough to store the whole object.

However, imagine that there are 2 free space blocks next to each other. Both were marked free, but they represented different objects (maybe they were next to each other on the heap because one referenced the other). What do you think GC will do? Will it treat them as two separate free memory blocks, making less chances for the next allocated object to fit into one of them?

Fortunately not. GC has an optimization introduced which makes such adjacent free memory chunks “merged” together:

How does GC do it? Let’s see in more details.

Free memory representation on LOH

Instead of compacting Large Object Heap, garbage collector keeps the address ranges of not used large objects in a Free Space Table:

As you can see on the figure above, as soon as gen 2 collection run, address ranges of two unused objects were just added to the Free Space Table.

Now you can see that “merging” two adjacent free memory chunks is just a simple addition operation (or modification of one number in the table).

Allocating memory on LOH

As soon as a new large (>85,000 bytes or applicable array) object is to be allocated on the managed heap, GC looks for a single “Free Space” block to hold it. However, it’s rather unlikely that the particular object’s size will fit into one of the free memory chunks. In that case, a new object will be allocated on the top of the heap (just after ‘Object D’ on the figure above).

It may happen that the memory obtained from the operating system for LOH is already used (read here for more info about memory). Garbage collector then asks the operating system for more memory segments to be acquired for LOH. If it fails, gen 2 collection is triggered hoping that some memory blocks will be freed and then the allocation will be possible.

Let’s now think about it for a while. We said previously that LOH collection triggers gen 2 collection. So trying to clean-up the Large Object Heap every time an allocation on it is made would be a potential performance killer.

How does GC solve this issue? Well, in fact, after a lot of optimizations introduced to LOH management in .NET 4.5, the GC takes the following actions in order to make a new allocation of a large object:

• firstly, the GC tries to allocate new objects into one of the free space “holes” on the LOH (it’s quite simple to calculate whether any chunk is large enough to store the object knowing ranges of free space blocks from the Free Space Table).
• if the above fails – garbage collector prefers to allocate new large objects at the end of the heap. Even though it may involve asking the OS for more memory segments, it’s been found to be easier and less consuming operation than performing full GC hoping to free some memory chunks on the LOH first.
• only if the above fails (LOH cannot be “extended”) – GC triggers gen 2 collection hoping to free some additional space that could be used for a new allocation.

What’s worth noticing is that this actions order seems to be good for performance, but can sometimes be a reason of memory fragmentation.

Manual LOH compaction

As you should already know from the previous article, LOH can be compacted programmatically, by setting the GCSettings.LargeObjectHeapCompactionMode property. The simplest way to force LOH compaction is as the following snippet presents:

There might be some niche cases in which LOH compaction may be useful. More details and discussions can be found for example here.

Best practices working with LOH objects

We can try to simply design our applications to use the less possible number of large objects, but let’s not exaggerate. We are living in a world where 8GBs of 2133MHz RAM costs sometimes less than 100$, so the memory is generally cheap

In principle, the rule seems to be simple: large objects we allocate should be reused (e.g. cached) as much as possible.

We should just keep in mind that allocation of large objects can be costly, because of a need to perform gen 2 collection in some cases before the object is allocated.

An example of potentially problematic large object can be a ViewState used in ASP.NET applications, size of which can easily exceed 85K. There are some good articles explaining how to not stupidly incur ASP.NET app’s performance using it, for instance this one.

There are also a lot of tools which can be used to measure the memory state and performance of our applications (also its internal mechanisms like garbage collection, heaps compaction etc.) which we’ll surely cover in one of the next posts within the series

Summary

Today we examined – previously a bit forgotten – Large Object Heap. We saw how the information about free memory blocks on it is stored by .NET Framework and how new objects are allocated on it.

I think it’s one of the next internal concepts of .NET worth knowing and understanding, even though in the common scenarios and business applications you probably won’t get into troubles with LOH. However, it may be practical and useful to know when working with some more memory-demanding applications like games.

I hope this post clarifies some LOH topics for you.

Let me know if there are any topics you’d be interested in reading about. I’m here to provide some value to you, so I’m open for your criticism and suggestions

Stay tuned!

The post [.NET Internals 08] What about Large Object Heap (LOH)? appeared first on CodeJourney.net.

Unmanaged resources and finalizers

First of all we should all get familiar with what unmanaged resources are. These ones can be files or folders on the disk, database connections, GUI elements or network resources. You can also think of them as some external elements we need to interact with in our applications through some protocol (e.g. database queries or file system manipulation methods). They’re called unmanaged not without a reason – any unmanaged resources accessed from .NET code will not be automatically cleaned by the garbage collector. That’s how they are not(un)managed.

Maybe you’ve heard about destructors in C++ – remember methods called after class’s name (like its constructor) starting with ‘~’ character? This also exists in .NET, but it’s not what you’d expect from C++. In .NET, such methods are referred to as finalizers.

Finalizers and destructors in .NET

Nonetheless, you can define a finalizer in your class in one of the following ways:

 Using “destructor-like” syntax:

Using Finalize() method:

If you examine the IL code of both versions, it turns out that the destructor can be seen as a syntax sugar for the finalizer. In fact, the ~MyTestClass() method is translated into Finalize() method:

The only difference is that – as you can see on the IL above – Finalize() method created by the compiler from the destructor is implicitly calling the Finalize() method on the base class of the object (you can see it in the finally block in the IL generated). So in case of using a destructor-like syntax, we are actually creating a finalizer by overriding Finalize() method, whereas when implementing a Finalize() method directly we do it as we’d use the new keyword (hiding base class’s implementation of Finalize() method).

Will GC call the finalizer for me?

As you may know, finalizers methods are normally used to clean-up some unmanaged resources (e.g. to release a file handle or database connection). You may think that as soon as your object is ready to be collected (nothing references it anymore), garbage collector will firstly call the finalizer (to ensure everything is cleaned up) and then reclaim the memory used by the object.

Unfortunately not. If you think about it one more time – GC cannot know what’s inside your Finalize() method. You can put there the code which downloads the whole Internet just like that:

… and even though we have nowadays a bit faster bandwidths than the one from the gif , you can clearly see it wouldn’t be a good idea to block garbage collection because of some costly process being executed before the memory for the object is reclaimed.

In order to avoid the above-described scenario, which would slow down the GC’s work significantly, .NET contributors decided that the finalizers on the objects are called periodically on a separate thread, completely independently from the GC.

This way the main GC thread is not blocked. However, it adds a bit of complexity to the garbage collection process – continue reading for more details.

Finalization mechanism

As we said in one of the previous posts, one of the various sources of references to objects in our .NET application (GC roots) are “objects finalization references”. Let’s now make this mysterious term clear.

In order to prevent having a finalizable object (containing finalizer method) reclaimed before its Finalize() method is called (which happens independently from the GC as you already know), there’s a separate references list called finalization queue maintained by the GC. As soon as memory for a new finalizable object is allocated, a reference to it is also put on the finalization queue, which becomes a finalization root for this object (it’s not the same kind of root as “normal” GC roots – it’s treated a bit more differently).

The diagram below presents the memory state just after few new objects were allocated on the managed heap.

As you can see, objects ob2, ob3, ob5, ob6 and ob10 are all placed on the finalization queue, which means all of them contain Finalize() method.

Within these objects, ob2 and ob5 don’t have any references (roots) – it’s visible on the left side (heap). It means that these two objects are not used (are not referenced by anything) so they’re ready to be garbage collected. However, each of them contains a finalization root on the finalization queue.

As soon as the next garbage collection occurs, ob2 and ob5 cannot be reclaimed straightaway, as there’s a Finalize() method to be called on them first. Instead, when GC examines these two objects, it sees that they are not referenced from “normal” roots, but they have finalization references on the finalization queue. If that’s the case, GC moves finalization reference to another data structure called fReachable queue (also visible on the diagram above).

After, both queues and the heap look as follows:

That’s what I meant by stating that finalization roots are “special” kind of GC roots. Apart from that, ob2 and ob5 are still seen as “rooted” (as still being referenced) and because of that are not reclaimed by the GC. In that moment, standard generational GC rules apply, so if the collection below was a gen 1 collection, both objects will be promoted to gen 2.

Periodically, the finalization runs on a separate thread exploring the fReachable queue and calling Finalize() method on each object referenced from it. Then it removes the reference to this object from the queue. At this moment, finalizable object becomes rootless and can be reclaimed in the next garbage collection cycle.

Dispose for the rescue

In order to unify usage of finalizers and allow developers to correctly free their resources without unnecessarily prolonging their objects’ lifetime (as you saw above, finalizable objects stay uncollected for at least 1 more GC round than “normal” objects), programmer can (and should) implement a dispose pattern. The purpose of it is to allow the developer to free unmanaged resources “manually” as soon as they are not needed anymore.

The framework also provides a GC.SuppressFinalize method which tells GC that the object has been manually disposed and no more finalization is necessary. As soon as it’s called, the object reference is removed from finalization/fReachable queue. The goal is to prevent finalizing the object twice.

IDisposable

.NET Framework provides a System.IDisposable interface which should be implemented by a class which works with unmanaged resources.

Sample implementation of dispose pattern can look as follows:

Implementing IDisposable interface only forces us to add Dispose() method. Inside, we perform the resources clean-up and call GC.SuppressFinalize(this) to mark our object as finalized. We also implemented a separate method, in the example called CleanUp(bool disposing), which performs the actual releasing of unmanaged resources. But why do we need this bool disposing parameter?

It may be important to differentiate when the resources cleanup is called directly from code (we’ll see how it can be done in the next section) or by the finalization thread. As you already know, finalization is executed periodically on a separate thread. It means that if our UnmanagedClass needs any thread-specific resources cleanup, it’s not a good idea to execute it on totally independent, finalization thread, which is out-of-context at that moment.

That’s why we added a boolean parameter to the CleanUp method, which will be set to “true” as soon as Dispose() method is called directly from code. In that case we know that we can execute thread-specific cleanup code.

Additionally, we implemented the Finalize() method, which also calls CleanUp(), but passing “false” parameter’s value. In that case, as the finalization thread is out-of-context, we shouldn’t execute thread-specific cleanup code. We should treat Finalize() method as a last resort in case Dispose() has not been called because of some reason.

You can also check Microsoft dosc to see what’s their recommendation on implementing IDisposable interface.

Using statement

As mentioned above, the programmer can explicitly call Dispose() method on the class:

but there’s another – and preferred – way: wrapping usage of the class within using statement:

How does using statement ensure that Dispose() is called on our object created within it?

Let’s see the IL:

As you can see, the compiler translates using statement into try-finally block. In the finally part Dispose() method is called on our object’s instance.

This way the compiler ensures that even if there’s any exception thrown in the using block, the object will be disposed. That’s why we should always work with objects that may use unmanaged resources in such a way.

Summary

In this 7th post from .NET Internals series we examined how unmanaged resources are handled within .NET Framework and what’s the GC’s role in that process. We got to know two new GC-internal data structures: finalization and fReachable queues. In the end, we got familiar with dispose pattern and using statement.

I hope it helped you to clarify these topics a bit

See you next week!

The post [.NET Internals 07] Unmanaged resources: finalization, fReachable queue and dispose pattern appeared first on CodeJourney.net.

Heaps generations

As we know from this post, heaps used by .NET process to allocate reference objects are allocated on different kinds of heaps. In the previous article we also got to know that only Small Object Heap (SOH) is the subject of compaction process. Large Object Heap (LOH) is not compacted because of the performance reasons.

This is however not the only optimization introduced by GC contributors into Small Object Heap. SOH in .NET is also divided into 3 generations:

• Generation 0 – containing newly created objects, which did not have any collection performed on them so far,
• Generation 1 – storing objects that survived a single garbage collection (were not reclaimed during it because of still being in use),
• Generation 2 – keeping long-lived objects that survived two or more GC cycles.

LOH is not divided into generations. More details on LOH management in the article linked just before the “Summary” section below.

Thanks to such a division of managed heap, the actual collection is performed not on the whole heap, but only on one of the generations, where:

to perform a collection on generation X == to collect objects on generation X and all its younger generations

That’s why a generation 2 collection is knows as a full garbage collection. During this process objects present on all heaps (on all generations: gen 0, gen 1 and gen 2) are examined and reclaimed. Obviously, this collection is also the most costly one.

Collection survivors

In that place we should introduce a term collection survivor. It can be defined as an object which survives the collection (is not collected during GC collection cycle as it’s still referenced by something).

Knowing that, the following “promotion rules” can be defined:

• object survives generation 0 collection => object is promoted to generation 1,
• object survives generation 1 collection => object is promoted to generation 2,
• object survives generation 2 collection => object stays in generation 2.

Collection thresholds in generations

In the 5th post in the series, it was mentioned that one of the conditions on which the garbage collection can be triggered is when “the memory used by the objects on the managed heap exceeds some defined threshold”. In fact, this is not a single threshold, but several thresholds. More precisely, there’s a separate one per generation. As soon as size of all objects in a particular generation exceeds threshold’s value, the collection is started.

In the beginning, the thresholds’ values for each generation are initialized to the following values:

• gen 0: ~256K,
• gen 1: ~2MBs,
• gen 2: ~10MBs.

Nonetheless, these are only initial values and are adjusted by the GC at runtime. One of the conditions to increase a particular threshold’s value is then the survival rate in a generation is high (more objects from particular generation are either promoted to the next one or stay in generation 2), which makes the GC cycles to run less frequently (the condition of exceeding the threshold is not met that often).

This approach seems reasonable, as the main goal of GC is to collect as many objects as possible and reclaim their memory, not to “waste time” only to get to know that most of the objects are still in use.

In general, CLR tries to find a balance between two factors:

• not letting application’s working memory set to get too big,
• not allowing GC to take too much time.

Structure of generational heap

The diagram below presents how SOH actually looks like being divided into above-described 3 generations:

As can be noticed, the next object pointer (you can read more about it in this post) refers to the place after the last object in generation 0. That’s how it happens that newly created objects are always allocated in generation 0.

Generational GC cycle

The following schema very clearly presents how the full generational collection looks like (click to enlarge to see it clearly):

The first step is generation 0 collection. You can notice that objects H and D are not referenced by anything, so after the first cycle on gen 0 is performed, these objects are reclaimed and not promoted to gen 1. The same happens with object F, which is not being referenced to anymore when the collection on generation 1 is performed – that’s why it’s reclaimed during this step and not promoted to generation 2. In the end of the full GC cycle, only still-referenced objects stay on generation 2.

What’s also important to notice is that after each collection generation 0 becomes empty (contains no objects). Generation 1 may still contain objects promoted from generation 0 and generation 2 may contain long-lived objects.

Why is static data harmful for GC?

Having the knowledge presented in this post, we can deduce that the more objects are stored on a generation the longer collection process lasts. What we should also already be able to see is that generation 2 contains long-lived objects which are mostly static or global objects that survive garbage collection cycles (as there’s always a “live” reference to them in the application).

Of course, there are some reasons why static data can be justified (e.g. server-side apps storing some context), but normally using static data should be avoided as much as possible and reasonable. I think you already know why – it simply incurs the application’s performance by making garbage collector’s work harder and longer.

Cross-generation references

There’s also one very interesting issue in having the managed heap divided into generations. Can you imagine the case, when an object is already promoted to gen 2, but one of its reference-typed properties has just been initialized in gen 0? Consider the following code:

Let’s assume that an instance of GuyFromGen2 has already been created long time ago and it’s already promoted to generation 2. At this moment someone calls CreateYoungObject() method, which initializes _youngObject variable. This makes the “parent” instance of GuyFromGen2 holding a reference to its instance object located in gen 0. 

When there’s a generation 0 collection triggered (which, as we said, examines only objects in generation 0) at this exact moment, it would find _youngObject allocated on the heap having no gen 0 references to it (as it doesn’t check gen 2 references during gen 0 collection) so it would assume the object not being used and reclaim its memory.

Fortunately, .NET has a solution for this issue – card table data structure.

Card table data structure

Card table is used by .NET’s memory management system to store information about objects from older generation (gen 2 in that case) that reference younger objects (from gen 0 or gen 1). The execution engine adds an entry into card table as soon as any reference to gen 0 or gen 1 object is created “inside” gen 2 object.

For performance optimization reasons the card table stores this information per “bit”, which represents 128 bytes (or 256 bytes in 64-bit architectures) of managed memory. That’s how the card table can be imagined:

Then, as soon as gen 0 or gen 1 collection takes place, apart from examining references only from the particular generation’s roots, GC also checks which memory ranges have their bits set to 1, finds out which objects are allocated within these memory ranges and treats them as potential GC roots (analyzing what these objects reference).

What’s more, as Konrad pointed out in his comment, because of write barrier optimization the whole bytes (8-bit chunks, not only single bits as presented on the figure above) are marked as potential roots sources. So in fact, such cross-generations reference can mark up to 2048 bytes (for 64-bit architectures: 256 bytes x 8 = 2048 bytes) memory chunk as a potential GC roots source.

Every referenced object found this way is added to the list of objects still in use (if you still don’t know what are GC roots and still in use objects list, read the previous post now).

but… what about LOH?

If you’re curious about Large Object Heap (LOH) in-depth memory organization, I recommend you reading this article by Maoni Stephens. I can just mention that LOH has its own memory threshold which can be exceeded and it also triggers Gen 2 collection – from GC point-of-view, collection of objects allocated on LOH happens only during Gen 2 collection.

Summary

We went through the concept of having SOH managed heap divided into 3 generations. By understanding how objects are promoted between generations and the implications of being a long-lived object we saw the potential harmful impact of static data on our application’s performance.

Another interesting thing we pointed out was the card table data structure used to solve cross-generations references issue.

Next week we’ll see more on unmanaged resources and objects’ finalization.

Stay tuned!

The post [.NET Internals 06] Generational garbage collection appeared first on CodeJourney.net.

Finding objects to be garbage-collected

As we already know, .NET doesn’t allow developers to directly allocate the objects on the heap. Instead, the only thing the programmer needs to do is to use the new keyword – framework takes care of all the rest (placing the object on the proper heap, initializing it etc.). After that, you can simply forget about your object, because it is released (collected) automatically by the garbage collector as soon as it’s not needed anymore. But how does the GC know which objects it can collect?

In principle, all the garbage collector does is looking for the allocated objects which aren’t referenced by anything, which means that such objects are no longer needed. This makes such objects ready to be collected (their memory can be released – marked as being possible to be used again).

There are several sources of references to objects allocated on the heap:

• stack (as we know from the 2nd post),
• global or static reference variables,
• CPU registers,
• interop references (.NET objects used in COM/API calls),
• objects finalization references.

GC roots tree

All above-listed sources are called GC roots (also referred to as root references). As objects referenced from any of the roots can reference another objects (for example a customer may reference its orders collection), all these nested references are represented as a tree (graph) for every .NET application. Such references tree may look as follows:

As you can see, GC root is present on the stack, pointing to Customer reference object. It contains an ArrayList collection of orders, which is referenced by Customer object. The collection itself also contains references to its elements, so the tree grows up.

Each root contains either a reference (address) to some object or null value.

Unreachable (not used) objects

The GC roots tree described above is used by garbage collector to know which objects are no longer needed. As we see, the graph contains all objects that are still in use (are referenced by something else).

GC uses this information the other way around –  generally the GC classifies all objects that are not reachable from any root (not present in the graph) as garbage. Such objects are then collected (the memory allocated for them is released) by the GC during its collection cycles, which will be described in details in the next posts in the series.

Memory releasing (garbage collection)

As mentioned, garbage collection is a process executed in cycles, which will be covered in the later articles, but the process itself contains two main steps:

1. Marking phase – finding which objects are still in use,
2. Collection – actual collection process of not-anymore-used objects (including SOH compaction).

Let’s see both phases in details.

Marking phase

When garbage collector runs its memory releasing process (referred to as ‘collection’), it needs to know which objects on the heap are still in use by the application – obviously, such objects cannot be collected.

In order to obtain this information, GC goes through the whole root references tree and for each GC root moves along all its reference tree (all objects that are referenced from the particular root) in order to mark each object in this tree as “still in use”. That’s why this integrated part of garbage collection process is called marking phase.

Marking is a recursive operation, because as we said before, roots reference some object, which references another object, which references another object…

so in order to make sure that every object in use is added to the “still in use” list (if it’s not there yet), this recursive marking is done as can be seen below:

Let’s now see how it fits into the actual collection operation.

Collection

The actual collection process, including previously-examined marking phase, can be pictured with the following pseudo-code:

First of all, GC gets a list of all GC roots in the application (the list is maintained by JIT compiler and the runtime). Then it needs to iterate through each root and perform marking operation on it. Knowing that marking is a recursive operation, after this phase is finished, GC can be sure that all objects that have any root reference (either direct one or being pointed to by another objects) are added to the list of objects in use.

Next step is the actual memory releasing process of not used objects, represented as a call to Cleanup() method on the listing above. Memory cleanup itself is a very simple process – it’s just marking a particular memory block(s) as free (not used). There’s no overwriting of memory or anything like that (except ‘side overwriting’ which happens during heaps compaction – described below). Just simple boolean-like terminology is used for memory blocks – used (1) or not used (0). It works that simple for LOH , but collection on SOH (still don’t know what are LOH and SOH? Go and read here quickly!) also involves the compaction process. Let’s see what that is.

Heaps compaction

As we know from the first post, memory on the managed heaps can get fragmented, leaving holes of not used memory between live objects. That’s why the garbage collection for SOH process involves compaction, which is a process of removing the memory holes on the heap. It’s often referred to as a Copy Collection process, because it’s based on finding the dead objects or memory holes (it’s actually the same – holes of the memory that are not used by anything, present in-between live objects) and overwriting these holes with the live objects allocated later (next) on the heap – by copying chunks of live objects ‘down in the heap’.

The result of the compaction process is the contiguous SOH, where objects are allocated on top of each other with no or minimal possible number of memory holes.

Finally, addresses to the objects stored in reference variables are updated in order to point to the correct, new places in the memory occupied by particular object.

The best gain we get from the compaction is that the SOH is not fragmented, so the memory can be used efficiently. On the other hand it requires some CPU time, which may have some impact on our application’s performance.

By default, LOH is not compacted because of the size of objects stored on it (>85 kilobytes). Copying chunks of such objects would take too much time. Instead, used space and memory holes on the LOH are tracked and the allocation algorithm is adjusted to try to reserve the memory for new objects in the optimal way (by finding the best possible free slots on the fragmented heap).

What’s worth noticing is that starting from .NET Framework 4.5.1, LOH compaction can be enabled on demand by using GCSettings.LargeObjectHeapCompactionMode property.

When does the GC run?

It can be stated that GC works in a non-deterministic manner. It runs on a separate thread when one of the following conditions becomes true:

•  the OS sends low memory notification,
• the memory used by the objects on the managed heap exceeds some defined threshold (which is adjusted dynamically at runtime, based on current allocations),
• GC.Collect() method is called in the code.

If you see GC.Collect() directly used in code, in almost every case it means that there’s something wrong with the application’s implementation. Several great Microsoft engineers and open-source contributors have worked on automatic garbage collection in order to not trigger it manually

The exceptions for this rule I’d accept are maybe some gaming implementations, in which you work with huge objects and sometimes you’d like to affect the way when these objects’ memory is cleaned-up or usage in debugging – most of memory profilers (like dotMemory or ANTS Memory Profiler) force the collection before the memory snapshot is done by the user. This is reasonable, because when debugging memory leaks or issues you wanna see which objects remain allocated after GC does its cleanup.

What’s also worth mentioning is that when the garbage collection is run, all threads are stopped by default, except the thread in which the collection was triggered. However, this can be configured by changing the GC’s settings to work in workstation or server mode, as well as making it run concurrently or in the background. There are different use cases in which each settings combination should be used – you can find more details about it here and here.

Summary

Today we’ve covered the main blocks from which the garbage collection process in .NET is built. We went through marking phase, then the actual collection process – also including heaps compaction. We learnt how the GC roots tree is built and how garbage collector knows which objects it can clean-up during its cycle. We also listed cases in which the collection can be triggered.

In the next post we’ll examine a very interesting concept – generational garbage collection.

See ya next week!

The post [.NET Internals 05] Garbage collection: marking, collection and heaps compaction appeared first on CodeJourney.net.

Ready? Let’s start then!

What (or who) is Garbage Collection (Collector)?

As you could read in the previous posts on internal details of .NET,  there are a lot of conditions and assumption of memory organization in the framework. Data is stored on the proper structures, sometimes boxed and unboxed, but what’s the best about it all is that it’s almost completely invisible for us – developers.

In fact, providing this invisibility is a goal of Garbage Collector (GC). It should become our close friend (it’s not that scary as you could have heard – we’ll see that soon ), because this automatic memory manager does a lot of stuff for us, including:

• allowing us to write code without having to release memory (still remember the meme about C++?),
• performing heap allocations in an efficient way,
• reclaiming the objects that are no longer used and clearing their memory,
• ensuring that every new managed object gets clean content to begin with, so there is no work to be done in an object’s constructor to allocate the memory for all its members,
• providing memory safety, so one object cannot use the memory reserved for another object.

The term ‘garbage collection’ was firstly used in LISP programming language in 1959 and since then it represents the concept of automatic memory management in programming languages and frameworks.

Does GC allocate or deallocate the memory?

There’s often this confusion between allocating and deallocating the memory – what is actually responsible for that? In principle, the goal of garbage collector is to deallocate the memory – as its name suggests, it collects (frees, reclaims) the garbage (not-anymore-needed objects).

On the other hand, when there’s any memory allocation request sent by an application, such query is transferred to CLR, which is actually responsible for allocating the memory. However, as we’ll see in the coming posts, the deallocation of memory (which is much more complex process that allocating it) can be triggered on various conditions and it can happen that allocation of the memory requires deallocation to be performed first. It doesn’t mean that this is a sequence of events (i.e. before allocation the deallocation is performed), but that the number of allocations done by the CLR can affect the way and frequency of garbage collections (we’d like to have available memory when an allocation is to be done).

That’s how the CLR ‘asks’ GC for help (or ‘communicates with GC’ by notifying it about allocations) in actually allocating the memory (for instance by making GC running a collection or heap compaction process more frequently). Because of that reason, it’s usually simplified that garbage collector handles both allocation and deallocation of memory.

As we’ll see below, allocation of memory is as simple as 1 + 2 = 3 – continue reading to get to know why!

Memory allocation in .NET

As we read in the first post of the series, each process has its own virtual address space. Garbage collector is responsible for allocating the memory on the managed heaps (more about heaps here) created in this process’s space. It means that garbage collector only takes care of reference type objects.

Value types are stored on the stack in LIFO order and their memory is reclaimed as soon as the method in which these value types are defined exits. It would make no sense if GC had to manage these theoretically simple and (in a perfect world) locally declared variables.

Next object pointer

To make it simple, as soon as managed heap is created, it contains a pointer to the memory address where the next object will be allocated. This pointer is simply a number and is referred to as next object pointer. Initially, its value is set to managed heap’s base address:

As soon as the first-ever reference object is to be allocated on this particular heap, the memory is allocated in the place where the next object pointer currently points to. State of the heap after first allocation would be something like that:

As presented above, the next object pointer is moved to the address just after the allocated object. It ensures that there are no unnecessary memory gaps on the heap.

Complexity of memory allocation

In reality, the next object pointer’s value is a number in hexadecimal format (e.g. 0xF7279). When a new allocation is to be made, the only operation to perform is an addition operation. It is known how much space is needed for the new object (more details in the next section), so in order to reserve the proper memory chunk, number of bytes necessary for a new object is added to the next object pointer’s current value.

This is how the reservation of managed memory works. Because of that it can be said that allocating the memory for reference types is almost as fast as allocating variables on the stack.

If you’d like to see how you can check the memory address of an object, I recommend checking this article.

How is an object’s size calculated?

In the comments under the previous post, Jelena asked for explanation on how the size of an object is calculated. Here it comes

As we all know from the second post, object is sometimes allocated on the Small Object Heap (SOH) or Large Object Heap (LOH) depending on its size. For that reason before the allocation is made, the object’s size must be determined. How is this size calculated then?

It would make things easy to assume that size of an object is everything it contains (also other objects’ sizes). In reality, it’s not counted this way. Objects contained in another objects are allocated separately on the heap, so their actual sizes are not included in the parent object’s size.

Very good example is given in Chris Farrell’s and Nick Harrison’s book – Under the Hood of .NET Memory Management. If we consider the following code:

That’s how the heaps’ state looks like as soon as as instance of MyClass is created:

The size of MyClass object instance will include:

• general class stuff (e.g. some metadata – you can check it yourself in IL code with ILSpy as we did with boxing and unboxing),
• memory required to store the pointers (memory addresses) to the string and bytes array.

string Test will be allocated on the SOH, and byte[] data will be allocated on the LOH (it’s more than 85K).

In general, the size of an object is calculated by adding sizes of:

• general class data,
• pointers (for reference types class members) containing only addresses in memory (hexadecimal numbers),
• value types (for value types class members) – more details here.

Summary

I hope this post makes the role of garbage collector a bit more clear for you. We’ve also seen how easy the memory allocation is and what’s the general way of calculating an object’s size.

Even we don’t perform the actual allocations ourselves manually and may think like this guy:

We’ll see if that’s true next week, when we’ll dig into more complex topic, the real purpose of the GC – releasing the memory!

Stay tuned!

The post [.NET Internals 04] What is Garbage Collection? Memory allocation in .NET appeared first on CodeJourney.net.

What is boxing and unboxing?

In the previous post we learnt what are value and reference types and got to know that the former are stored on the stack, whereas the latter are stored on the managed heap. So why should we care? Isn’t it .NET runtime which correctly manages these data structures and what’s stored on them so we don’t need to worry about that?

In fact, no. What’s crucial to know and understand are the implications of moving data from the stack to the heap and otherwise.

Remember:

• when any value type variable is assigned to a reference type variable, data is moved from the stack to the heap and this is called boxing,
• when any reference type variable is assigned to a value type variable, data is moved from the heap to the stack and this is called unboxing.

Microsoft Docs examples illustrate these actions very well.

Consider the following example of boxing:

and the memory state as it executes:

In order to store “123” value in an object, the “box” is created on the heap and the value is copied inside it.

On the other hand, when unboxing happens:

that’s how stack’s and heap’s content changes:

Value “123” is taken out of the “box” and placed back on the stack.

Notice that when i value type is boxed into object o, there’s a reference stored on the stack and the actual memory is allocated on the heap (as for all reference types). As soon as the unboxing happens, the real data being on the heap must be copied to the stack (variable j). In both cases our goal is to deal with the same value (123).

As you can imagine, these operations produce some additional cost and affect the performance, which we’ll discuss in a moment.

Let’s see some IL

When analyzing such performance or memory management aspects in our code (C# in that case), it’s often worth to see how the Intermediate Language (IL) looks like.

We haven’t covered this concept yet, but as you probably know when C# code is compiled into DLL or EXE, these output files actually contain the IL code, which is later JIT-compiled and executed by the virtual machine (more details here). The .NET runtime must somehow know whether is should box or unbox the particular variable, as it requires some special memory allocation actions to be taken.

Let’s create some simple .NET Console Application with the following code in its Main method:

Let’s compile the application so we can find BoxingUnboxingTest.exe file in the output directory. Now we will use ILSpy to see the IL code inside the executable.

As soon as the EXE file is opened in ILSpy, we can go directly to see Main(string[]) : void method’s compiled content, choosing “IL with C#” view to make it simpler for us:

Notice the box statement just after the assignment of value type to reference type happens (object obj = num). Similarly for unbox.any statement just after assignment of reference type to value type (int num2 = (int)obj).

That’s how boxing and unboxing is represented in the IL.

When do we box and unbox?

The sample code above may seem naive and you may think “hey, but I never do such things”. That’s true in most cases, but values in our code are often boxed/unboxed even if we are not aware of that.

Non-generic collections

For instance, there still exists the old-school ArrayList collection:

which as you can see above has the Add method taking an object parameter. It means that when we’d like to add an integer to ArrayList:

boxing takes place:

Such issues were eliminated by generics and generic collections.

String concatenation

Another interesting example is the concatenation of strings with value types using “+” operator:

Such operation involves String.Concat method’s version taking two object parameters, so it implies boxing the integer first:

In order to avoid it, it’s enough to slightly modify the code by using ToString() method on the integer variable (for now ignoring ReSharper telling you it’s redundant ;)):

and there’s no boxing anymore, as now String.Concat method’s version taking two string parameters is used:

There could be many more cases to present, but the goal is that you get a strong feeling what boxing and unboxing is and when it occurs.

Performance implications of boxing and unboxing

As we already know, boxing and unboxing imply some cost. In case of simple stuff like concatenating the string once with some integer, the performance gain by introducing ToString on integer first is unnoticeable. That’s why, as I wrote before, even ReSharper tells you to not do it:

In that case it’s better to keep the clarity of the code without ToString method.

The perspective changes when you need to perform such operations in a loop for hundreds or thousands times. In this case the execution of code using boxing can be even 150% longer than its equivalent with no boxing (you can create a simple application and measure the execution time of code with and without boxing or check this article).

Boxed values also take more space in the memory than value types stored on the stack. Copying the value to/from the stack is also a cost. According to MSDN, boxing can generally take even 20 times longer that simple reference assignments, whereas unboxing can be 4 times slower than assignments.

So… why to use boxing and unboxing?

Despite all performance implications boxing and unboxing have, these concepts were introduced into .NET because of some reasons:

• there’s unified types system in .NET, which allows to “represent” both value and reference types in a similar way – thanks to boxing,
• collections could be used for value types before generics were introduced into .NET,
• it simplifies our code, like we saw for string concatenation and in most cases this clarity gives us much more than the performance we’d gain trying to avoid boxing.

Boxing and unboxing are so common that we can’t avoid them. We should be aware how it works in order to be able to minimize their usage, but it should always be considered reasonably. Don’t spend time on optimizing your code, checking IL all the time to get to the point with the smallest possible number of box statements used. Keep in mind that the clarity of your code, its self-explanatory structure and comfort of reading is sometimes much more valuable than some small or barely noticeable performance gain.

Summary

In today’s post we saw what is boxing and unboxing, how it’s represented in the IL code and what implications on performance it may have. I hope it clarifies these commonly mentioned concepts for you, at least a bit

If you’re interested in digging into nitty-gritty, low-level details of boxing and unboxing in .NET, I invite you to read this great post by Matt Warren about even more internal internals of boxing and unboxing

In the next posts in the series we’re going to start exploring the garbage collection. If you have any suggestions or ideas for the coming articles, let me know in the comments (some of you already did on reddit – new posts scheduled ).

See you (or better read you ) next week!

The post [.NET Internals 03] Boxing and unboxing appeared first on CodeJourney.net.

If you’re here for the first time, I encourage you to first read the .NET Internals series introductory post explaining the basics of memory structure.

Memory division

By default, when the .NET application is started and virtual address space is allocated for the process (as we saw in the previous post), the following data structures, represented as heaps, are created:

• Code Heap – storing JIT-compiled native code,
• Small Object Heap (SOH) – storing objects of size less than 85 kilobytes,
• Large Object Heap (LOH) – storing objects of size greater than 85 kilobytes*,
• Process Heap.

*side note: there’s an exception for arrays of double,  which are allocated on the LOH much before reaching 85K (double[] is considered “large” when it exceeds 1000 elements). This is for the sake of 32-bit code optimization. More details here and here.

Division of storing small and large objects on different heaps has its performance reasons, which will be covered in the next posts about garbage collection.

Elements placed on the heap have their addresses, which are a kind of pointers to the place in memory where the particular object is located.

Heap is however not the only data structure used by .NET process. There’s also a stack, which helps in tracking code’s execution and storing some special types of data. We’ll now see in details how stack and heap are used and organized.

Stack

Stack is a memory structure organized as LIFO (last-in-first-out) collection. If you think about it, it’s perfect for storing anything that may soon need to be used (easily popped out from the top of the stack). This nature of stack leads to its two main purposes: keeping track of execution order and storing value types variables.

Keeping track of execution order – call stack

Most of the code we write is wrapped into classes and methods, which can call another methods, which call another methods etc. .NET Framework must all the time keep track of this execution’s order. Moreover, it also has to remember the state of variables and parameters of the method calling another method, while this another method executes (to be able to restore the calling method’s state after the called method returns).

As soon as any method is called, .NET initializes a stack frame (kind of container) which stores all data necessary to execute the method: parameters, local variables and address of the line of code to go back to when the method finishes.  Stack frames are created on the stack on top of each other. Each method has its own stack frame. All that behavior is well illustrated on the figure below.

Stack used for storing the execution order is often referred to as call stack, execution stack or program stack.

Take a look at the following code:

In order to call Method2, the framework must save an execution return address which will be the next line of code to execute after Method2 finishes (line 4 in the example above). This address altogether with parameters and local variables of the calling and called methods are stored on the call stack, as the schema below presents.

You can also notice what happens when Method3 returns (its stack frame is popped out from the stack – it disappears).

Storing value types

Stack is also used to store variables of any of .NET value types, including bool, decimal, int etc. Full list of .NET value types can be found here.

Value types are basically the types which keep the data and memory in the same location. What’s also interesting to know is that all value-typed local variables of the method allocated on the stack are cleaned-up after the method’s execution finishes. It happens because the method’s stack frame becomes unreachable – stack has some pointer to the beginning of the top-level stack frame (current stack frame pointer), which is simply moved to the second stack frame from the top as soon as the method’s execution is finished (the data is still physically there, but it’s not reachable by default .NET mechanisms).

Heap

Heap is similar to stack, but if the stack can be imagined as a series of boxes stacked on top of each other, where we always push or pop the top one, heap contains boxes that can be accessed at any time. The boxes in the heap are placed in different places, not necessarily on the top of each other. To access one of the boxes we don’t need to pop out the top ones first.

Storing reference types

All other variables, which are not value types i.e. are either string or object (or deriving from one of these, so all classes, interfaces and delegates) are referred to as reference types. All reference types are allocated on the managed heap (SOH or LOH, depending on their size). However, even though an object is allocated on the heap, a reference to it (address on the heap) is stored on the stack.

Consider the following code:

The figure below presents how stack and heap would look like here in terms of allocated data:

The “OBJREF” stored on the stack is actually a pointer (reference) to the MyClass object on the heap.

NOTE:  MyClass myObj statement doesn’t allocate any space on the heap for myObj variable. It only creates “OBJREF” variable on the stack, initializing it with NULL value. By the time new statement is used, the actual memory allocation on the heap takes place and the reference’s value is set.

Value types vs reference types (stack vs heap)

The crucial difference between value and reference types is that when a value type variable is passed to another method as parameter or just assigned to another value type variable, its value is copied into the new variable. That’s why when passing a value type to another method, which modifies this variable (parameter), when the method returns the original variable’s value is not changed (it was its copy which was used inside the called method). Figure below shows this behavior.

It’s different in case of reference types. When reference type variable is assigned to another reference type variable (or method’s parameter), it’s not its value which is copied, but only the reference to it (address in the memory where the data is actually stored on the heap). In effect, the new variable (or method’s parameter) still points to the same place in memory, so modifying such reference type variable inside the called method will also be in effect outside of the method. This case is illustrated below.

Of course, storing some data on the stack and some on the heap (moreover: different types of heaps) has its precise purpose, which we’re going to explore in the coming posts

Summary

In today’s post we went through stack and heap, two data structures used for storing value and reference types and tracking execution order of .NET application. In the next posts in the series we will see what is boxing and unboxing and how it can affect application’s performance. We will also examine the garbage collection process in details.

Stay tuned!

The post [.NET Internals 02] Stack and heap – .NET data structures appeared first on CodeJourney.net.

Ever been surprised that there’s no need to worry about memory allocation and deallocation using high-level programming languages such as Java or C# after leaving the university* ?

Still remember (old) C++ times with delete statement?

By this post, I’d like to introduce a new “.NET Internals” series on the blog. I will be publishing a new post on .NET internal concepts every Wednesday. No end date for the moment

Posts within the series will be mostly about memory management and performance aspects of .NET applications. All discussed concepts are applicable to most of the modern programming platforms, but the examples will be based on .NET Framework.

If you’re interested in such topics, I encourage you to check this blog every Wednesday starting today

*there is 1 assumption and 1 fact here:

• assumption: you used to program in C/C++ (before C++11) on the university,
• fact: that’s not true you don’t need to worry about memory management using high-level programming languages; wait for the next posts to get to know why.

CLR

Each application targeting .NET is managed by CLR (Common Language Runtime), which is a part of .NET Framework, which must be installed on the computer on which the .NET application is launched – the exception are .NET Core self-contained apps, which we are not discussing in this series. By default, the CLR is loaded by each .NET process as a hosted process (so the CLR runs as a part of .NET application).

This CLR “layer” is responsible for various things while our application is running:

• executing CIL (Common Intermediate Language),
• ensuring type safety,
• handling exceptions,
• managing threads,
• garbage collection.

Memory structure

Already running application is represented as a process in the operating system. As software developers we know that such process saves some data into the memory, which is determined by the amount of RAM available on the computer. However, process never operates on this “open memory” – only OS has direct access to it. When the CLR is launched within .NET process, it requests some amount of RAM from the operating system. This memory chunk is called virtual address space. Each process has its own virtual address space, but all processes running on the same machine share the same physical memory.

32-bit OS

Do you remember 32-bit editions of Windows XP, which could only use 4GBs of RAM (with even less visible to the user)? The reason for that is that in 32-bit operating systems total virtual address space is 4GBs, usually divided into two equal chunks – first one owned by the OS and the second one prepared for user processes. The diagram below presents.

64-bit OS

In 64-bit Windows theoretical amount of virtual address space is 16 exabytes (2⁶⁴ bytes). In reality, only a part of it is used: 8-terabyte chunk is used for user space and portions of 248-terabyte are used for the OS:

Sharing memory between processes

What seems to be critical to understand is how this whole virtual memory is shared between processes. As we said before, multiple processes share the same memory, but accessing different (separate) chunks of it. Let us illustrate once more:

As can be seen, each process reserves different pages of memory when it’s needed. Even though the default 2GBs of virtual memory reserved for a single process may seem to be a lot, every developer needs to be aware that when an allocation (reservation of memory chunk) is made (manually or by memory management system) the virtual memory manager needs to find a single, continuous virtual memory block large enough to store the requested amount of data. This rule leads to the presence of “holes” in the memory. This virtual memory fragmentation is well-presented on the diagram below:

However, this requirement of finding a continuous block to be allocated refers only to the virtual memory. Physical memory is divided into pages, which can be stored in totally different places – as you can see on the diagram above with Notepad.exe application using continuous blocks of its virtual memory, but committed to noncontiguous pages scattered in various physical locations.

If the process requests for too much memory, which cannot be continuously allocated in the virtual address space (your process is out of virtual memory) or no more virtual memory can be committed (assigned) to the physical storage, it can result in OutOfMemoryException thrown by .NET code.

Summary

This was the introductory post of my new .NET Internals series. We saw what are the basic concepts of the memory structure, learnt what is virtual address space, how physical memory is shared between processes and why Windows XP was able to handle only 4GBs of RAM

See you next Wednesday!

The post [.NET Internals 01] Basics of memory structure appeared first on CodeJourney.net.

In today’s post I’m going to sum up the conference and the sessions I attended

Tickets and organization

I attended the same conference 2 years ago and that time the conference was free. This year (I don’t know how it was organized last year) the tickets were paid. Normal ticket costed 150 PLN (~45 USD). The organizers also offered tickets for students for 50 PLN (~ 15 USD). This shows that the entrance was rather cheap as for such a full-day event, also compared to some similar conferences held in Poland.

In my opinion, making the conference paid improved the event in general. I felt the talks were generally more concrete, given by some well-known speakers (also abroad ones). The whole event seemed to be more “cosy” than 2 years ago. There were less people, at least I felt so (not knowing the real numbers).

This year, the conference was held at Marek Edelman Dialogue Center in Łódź:

The place was OK in general, with two rooms in which the presentations were done – one downstairs, the other upstairs. The building is quite spatial and comfortable and in a good location.

There was just one downside of this venue – there turned out to be not enough places to eat. The stand-up tables were OK during the coffee breaks, but it turned out to be insufficient during lunch. All attendees started to take their plates and eat, but there were no places to sit down. It was quite difficult to eat in such conditions, so there’s a place to improve here. But come on, that’s not food what we came there for, right?

Sessions

Below I’m listing and describing my subjective feelings on the sessions I chose to attend from two simultaneous tracks available. The sessions included both Polish and English ones (in the schedule there was always at least one English session to choose at a time).

Mark Rendle – Cloud Native .NET

First presentation of the day, the opening keynote, was held by Mark Rendle. The title of his talk was “Cloud Native .NET”. Mark explained us what the “Cloud Native” actually means, mentioning and digging into concepts of microservices and Kubernetes. During the talk Mark was using the presentation system he implemented, which is slidable.io. What’s interesting, the system allowed to ask questions directly from mobile devices and few folks made use of this functionality

All examples and concepts Mark presented during his talk are available in the source code of Slidable, which can be treated as a reference for building web application in accordance with “Cloud Native” and its principles.

Tomasz Onyszko – application + cloud + security => what could go wrong!

I was a bit disappointed when I entered the room in which Tomasz‘s presentation took place, as the presenter made the disclaimer in the very beginning that we will see a lot of Azure during the talk. For the last 2-3 years I’ve seen too much of Azure on various conferences and meetups so I’m a bit fed up with it. Maybe it’s only my personal feeling, but I think that Azure is very fancy and extremely helpful for developers, but it’s not a thing that you can start playing with directly after the conference. Most of the services are paid and for sure worth their prices in an enterprise, but not easy to start working with on a development playground after work. I really like to see some sneak peaks of trendy technology or interesting aspects that I could focus on in software development, but I didn’t feel getting it after this talk.

The presentation showed few mostly authentication and security-related functionalities in Azure, which are for sure useful for administrators working on machines’ configuration. I didn’t really feel like I learnt something new and useful during the talk, however the presenter was very good and with a nice sense of humor, so for people interested in such topics and working with Azure on a daily basis I’d definitely recommend this session.

Rafał Legiędź – developing for mixed reality with Hololens

For me, Rafał‘s talk was the best one on the whole conference. He presented us with the basics of Hololens, including basic technical and construction details of the device and the most common usage cases of it. First of all, I was surprised that in contrary to popular belief devices like Hololens and not used mostly for having fun or playing games. It’s used for things like elevator maintenance or cars design and production process.

The presenter defined the two popular terms: VR (virtual reality) and AR (augmented reality) emphasising the difference between them. By the end of the talk, Rafał showed us how easy it is to start developing for Hololens with Unity and Visual Studio.

The topic of AR devices is very inspiring and seems to be developing rapidly. On the other hand, there are already a lot of real, deployed solutions that help manufacturing or banking corporations to build their products more effectively, which I was not aware of. I think this topic is really a future and will be an important piece of software development in the upcoming years, even though it may seem a niche for the moment. I really recommend everyone to get into the topic, even it may not be affordable for everyone – the cheapest, development Hololens set with no guarantee costs 3000$. However, there are some cheaper AR glasses slowly appearing on the market (can be bought for even 900-1000$), so it should become more available in the coming years. Nevertheless the topic is worth attention, as this technological stack is a completely different world, offering totally different possibilities than “casual” software.

Matthias Koch – become a productive .NET developer with JetBrains Rider

First presentation after the lunch break I decided to attend was done by Matthias Koch – a developer working for JetBrains on a cross-platform Rider IDE for .NET. Although the talk was very lively done, it was a step-by-step presentation of functionalities coming from ReSharper introduced into the Rider UI. I use ReSharper on daily basis so I knew most of the functionalities, however I learnt some new and useful ones

Apart from that, the Rider IDE didn’t seem anything special itself. Apart from being cross-platform I didn’t notice any big difference between using Rider or Visual Studio with ReSharper (assuming you work on Windows). Matthias mentioned that as Rider is based on JVM, the garbage collection is more effective than in VS, as it doesn’t need to stop all of the working threads during the collection cycle. Maybe it improves the performance of the IDE in some extent.

Michał Bartyzel – please, motivate me!

The talk by Michał Bartyzel was the one non-technical with a hope to get some new ideas of self-development or productivity. It was generally well-done, with the good sense of humor from the presenter included. Michał presented some set of non-obvious recommendations to get ourselves self-motivated including sleeping well, not being hungry when making important decisions or creation of non-TODO-lists. Some of the techniques he mentioned are familiar to me as you could read in my post about productivity.

Basically, Michał gave us some easy tips than can be extremely helpful in self-organization. Such recommendations may sound obvious and not worth attention, but I really recommend at least trying to introduce some of them – it really helps. Michał published the slides from his presentation (Polish only) here.

Ian Cooper – 12-Factor apps

The closing talk was done by Ian Cooper, software development geek coming from London and being in the programming industry for quite a few years The talk was concerning the concept of “12-Factor apps”, which is basically a set of principles that every modern, “cloud native” application should follow. Ian focused on few most important of these rules and showed us some code presenting these concepts. More details about 12 factors can be read on https://12factor.net/

The whole idea is definitely worth knowing, especially in the modern world when the web applications are required to be build in software-as-a-service approach. I think it was a very good, meaty summary of the day, encapsulating concepts that are currently very trendy and asked by the business. The Twelve-Factor App concept can be seen as a guide to build reliable, extensible and loosely-coupled software with low entry threshold for new developers. With the power of cloud and computation centers like Azure, which is now mentioned during almost every presentation :), I see it as a crucial set of principles to follow to be able to use the power of hardware we have (powered by robust software), to produce high-quality and scalable IT systems and applications.

Summary

To sum up, the conference was very good. It felt to be small, family-atmosphere conference with not thousands of attendees. The talks were generally high-quality. Even if some I decided to attend were not especially interesting for me, I’m sure these ones provided great value for people being really into the covered topics.

I definitely recommend the conference to all .NET (and not only!) developers and will surely attend again

The post GET.NET 2018 – Łódź edition appeared first on CodeJourney.net.

Day 1 – workshops

Breaking Apps Apart Intentionally – Visual Studio + Docker + Sprinkles of Azure = Modern Microservices (Alex Mang)

Here I need to mention one organizational drawback – we didn’t have any desks in the workshop’s room, which was quite uncomfortable – it’s always better to be able to have your laptop on the desk in front of you in order to write down some stuff or test things covered by the speaker. It was surprising as in the other rooms where the rest of workshop sessions were held the desks were present.

 

That is however the only dark side of this workshop. Alex hosted a very good session, explaining the containers concept using Docker, DockerHub, Visual Studio and Microsoft Azure. Maybe in the beginning the session was too “slidy” with a lot of architectural diagrams, but fortunately around the middle of workshop Alex switched to VS and PowerShell showing us several fancy features of Docker:

Day 2 – conference 1st day

5 unit testing facts I wish I know 10 years ago (Dror Helper)

The session’s difficulty level was defined as 200, which means it’s an “easy” one, however I really enjoyed it. Dror showed some basic, but very important concepts needed to start writing unit tests. Maybe it’s even more accurate to say he eliminated all of the most common excuses a lot of us (programmers) have to not write unit tests. Very important knowledge so to say

Slides were very good quality and the speaker himself is quite a funny guy

How Far Can ‘Serverless’ Actually Go? (Alex Mang)

Alex is a kind of guy I like – a bit american-styled speaker with appropriate sense of humor. During the session he explained and showed how to build an application using Azure Functions. Even though I don’t use Azure at all personally (just tried hosting few simple web apps) the topic is interesting showing how easy it can be to build serverless apps or services without setting up any infrastructure.

PS: The photo I’ve taken was even chosen as the photo of the day by Alex He is active in social media and I definitely recommend to follow this guy.

Adding history to CRUD (Dino Esposito)

Launching patterns for containers – it’s more than just scheduling (Michele Leroux Bustamante)

There were some issues with the demos Michele showed, but this happens to the best even

Building for the Future without Abandoning the Past (Jeff Burtoft)

Jeff is Microsoft’s employee and you can definitely feel that. Presentation was generally simple and lightweight, but well-told and properly organized. Jeff presented us with some techniques on how to start making our legacy apps/systems better without breaking the previous interfaces and backwards compatibility.

Q&A panel, DevTalk Live and a conference party!

After all 2nd day’s sessions were finished, we officially started a conference party. During this time we’ve had some delicious dinner, drunk few craft beers. In the meantime we took a pint and went for a Q&A discussion panel with all the speakers:

Few very interesting topics were covered during the discussion panel. I think that the most active speakers during the panel were Alex and Michele. Dino also added quite a few nice insights

After the discussion panel there was a 2nd edition of DevTalk Live hosted by Maciej Aniserowicz:

Day 3 – conference 2nd day

Past, Present & Future of C# Debugging (Alon Fliess)

Creating a real-world Xamarin application architecture using MVVM (Gill Cleeren)

Organization

Registration

I was privileged by attending the pre-cons workshops on Wednesday, which were held in EXPO XXI – the same place as the main conference the next day – and the registration process was held there. It was pretty well organized and took us no more than 10 minutes. The other participants who didn’t attend the pre-cons told me there were also no issues with the registration directly at the conference on Thursday. If this it true then congrats to organizers for eliminating last year’s huge queues

It was a very good idea – practical and useful gadget.

Food

Food was the same as last year – very good, huge choice, efficient service. Craft beers served during the conference party on Thursday were also really nice

Party

The conference party which I already mentioned was well-organized. We had a lot of places to sit – so we made a use of it and had a lot of veeeery interesting and inspiring conversations with other participants That’s a very important part of such events – networking with the others, getting to know their problems in their companies, businesses or life. I think this is one of the aspects in our programming community we are missing the most – meeting others, discussing and exchanging ideas and insights – not matter which company, country or programming religion you come from.

Summary

However, it was great to gain some invaluable knowledge, meet many new people and have a lot of great conversations. Thank you all for organizing and attending the conference. See you next time!

The post .NET Developer Days 2017 appeared first on CodeJourney.net.

Why to keep and restore Activity’s state?

As I already described in my post about Android Activities, the OS may react to some “constant” state changes by calling lifecycle methods during Activity’s life, which may be overridden by the programmer in order to take some additional actions. However, there are some behaviors in Android apps that may change its configuration, after which the state of the Activity (for instance: arrangement or some UI elements or checkboxes’ selections) may be lost if not handled properly.

To give you a real example: in MoneyBack application there is a main screen with two tabs. I noticed that when the second tab is selected:

and the device is rotated:

first tab becomes selected. User is losing his input (selection of the 2nd tab).

Screen orientation change is one of the reasons why we should care about saving and restoring Activities’ states.

Keeping state changes – OnSaveInstanceState()

Each Activity allows us to override OnSaveInstanceState(Bundle outState) method, which is called when the current Activity is to be killed (e.g. when device’s screen orientation is changing or the Activity needs to be killed by Android in order to release some resources).

Please don’t confuse it with Activity Lifecycle Methods (which I already described here), like OnPause() or OnStop() which are always called by the OS when particular action occurs. OnSaveInstanceState() method is generally called after OnPause() and before OnStop(), but it’s not always the case. For instance, it won’t be called at all when user navigates back from ActivityB to ActivityA, as in this case ActivityB will never be killed. More details can be found in official Android documentation.

OnSaveInstanceState() is called with Bundle parameter provided. It represents a simple key-value dictionary, which is additionally serialized, so it should be used for storing simple values like strings, integers etc. There are other structures and techniques for storing more complex data on state changes events (see the link in “Summary” section of this post).

In order to save currently selected tab, we can implement the method as follows:

Now, each screen orientation change will make this method called and the index of selected tab will be saved into Bundle outState dictionary under “selectedTabPosition” key.

Restoring state changes – OnRestoreInstanceState()

As soon as the Activity considered backs to life (is being resumed), OnRestoreInstanceState(Bundle savedInstanceState) method is called by the OS. It will only be called if there is a saved instance kept by calling OnSaveInstanceState() method before. As you can see, this method also comes with a Bundle parameter, which is the same key-value serialized dictionary we used for saving the Activity’s state.

In case of tabs selection restoring, we can implement this method as the following code presents:

previouslySelectedTabPosition variable will be initialized with integer value from Bundle dictionary saved under “selectedTabPosition” key, or default value (0) if nothing saved for the key is found.

NOTE: Bundle parameter provided in OnRestoreInstanceState() is exactly the same, as the one available in OnCreate(Bundle bundle) method called on Activity’s creation. We could actually also retrieve our saved value there, but in many cases restoring the state (like selecting previously selected tab) requires some UI elements to be already initialized, which should be handled within OnCreate() method before. Because of that, I’d rather suggest to restore Activity’s state using OnRestoreInstanceState() as a general rule (of course there may be some exceptional cases ). OnRestoreInstanceState() will always be called after OnCreate().

Moreover, if we wanted to use Bundle parameter in OnCreate() method, we’d need to check each time if bundle is not NULL (as we don’t know whether OnSaveInstanceState() had been called before).

As a result, the same tab as previously is selected even after rotating the device on MoneyBack‘s MainActivity:

“Automatic” state changes handled by Android OS

Maybe you noticed in your Android application, that not every UI element needs to be handled “manually” by implementing the above-mentioned methods in order to save its state. Android performs some kind of “automatic” saving and restoration of basic UI elements’ states.

The rule here is that as long as UI element (like TextView or Button) has its android:id set in .axml layout file, OS will automatically manage those elements’ states (e.g. text entered into EditText).

For example, for EditText declared like that:

its properties (especially its Text property) will be automatically saved and restored on screen orientation changes.

Summary

We’ve seen how to handle Android Activities’ state changes caused by configuration modification (e.g. screen orientation change, Activity being killed by the OS in order to free some resources etc.) by implementing OnSaveInstanceState() and OnRestoreInstanceState(), where we added and retrieved key-value data to/from Bundle serialized dictionary. We just need to remember that those methods are not always called (e.g. when navigating between Activities using “back” button), so these should be used only for certain kind of state changes.

We’ve also seen that UI controls with their android:id defined in layout files have their states resistant for configuration changes (handled automatically by Android OS).

Bundle dictionary is serialized for better performance and memory utilization, so only simple-typed values should be stored in it (like strings or integers). In order to store more complex data, Android provides different possibilities of managing it (e.g. using OnRetainNonConfigurationInstance).

The post Managing Activity state changes using Bundle appeared first on CodeJourney.net.

The dialog looks as follows:

It may be opened e.g. when clicking on a button in the app, as I used it in MoneyBack.

Creating DatePickerFragment

First of all, we will implement the dialog to be displayed within DialogFragment. Doing that our dialog will be able to be displayed as independent piece of UI on the top of any Activity.

The second requirement is to implement IOnDateSetListener interface (coming from Java/Android), which provides a callback on date selection action done by the user.

The DatePickerFragment meeting those two requirements may look as follows:

There are few crucial parts in above-listed class:

• Line 1: as said before, the class derives from DialogFragment and implements IOnDateSetListener interface
• Line 3: here we have public static readonly string TAG variable defined – it is used as a unique identifier of the Fragment; in some cases, Fragments can be displayed without any UI – in such case the only possibility to identify and get the Fragment is by using findFragmentByTag() method
• Lines 5-10: define _dateSelectedHandler delegate which may be provided to the constructor of our DatePickerFragment with the method to be called-back when user selects a date
• Lines 12-20: inheriting from DialogFragment allows us to override its lifecycle methods – one of them is Dialog OnCreateDialog(Bundle savedInstanceState), in which we actually create the Dialog-inheriting class – in our case – DatePickerDialog – with initially selected date (set to current date in the provided example)
• Lines 22-26: implementing IOnDateSetListener interface requires to implement OnDateSet method, which is called when the user selects the date and clicks “OK” button;
  NOTE: one of this method’s arguments is monthOfYear integer value, which contains month number expressed as values 0-11 (NOT 1-12) for compatibility with java.util.Calendar.MONTH

Using DatePickerFragment

In MoneyBack application I’ve used DatePickerFragment on creating/modifying an event, when user needs to select its date:

As soon as the button is clicked, the following code executes:

As you can see, to DatePickerFragment‘s constructor I’m providing the delegate with a method to be called when the date is selected by the user (setting _selectedDate property with selected time and putting the date in short format as button’s Text).

In order to display a dialog, Show method is called, taking current FragmentManager and already mentioned per-Fragment-unique tag string, defined statically in DatePickerFragment class.

Summary

Today’s short post presented how to easily create a DialogFragment showing DatePickerDialog for date selection, which allows to provide any method to be called-back when the date is selected by the user.

I hope you’ll find it useful

The post Date selection using DatePickerDialog in Xamarin.Android appeared first on CodeJourney.net.

One-to-many, many-to-one

One-to-many relationships are used in general for parent-children or whole-elements relations. Classic examples are: bus and passengers, document and elements etc.

One-to-many relationship means that the one-end entity knows about its children and many-end entity has a reference (foreign key) to its parent (but not necessarily knows about it).

On the other hand, the opposite relationship to one-to-many is many-to-one. In that case, many-end entity has a reference to its parent and knows about it, but the one-end entity doesn’t necessarily know about its children (at least not directly).

I used a verb to know several times – so it’s time to explain it 🙂 By “knowing” about the other end of a relationship I understand having a reference to it. It means that, for instance, in many-to-one relationship, the one-end of it doesn’t have a reference to its children.

However in most cases what we’d like to have is a hybrid of one-to-many and many-to-one relationships. I will call it one-to-many with inversion. We want both parent to know about its children, and each children to know about its parent.

In this post we’ll cover one-to-many with no reversion and one-to-many with reversion, as it also includes many-to-one relationship so you can have a comprehensive overview. We are going to see it on the example of Employee and Duty* entities. Each employee has a list of his duties, whilst each single duty is assigned to only one employee.

* “Task” would probably be a better name, but I didn’t want to make it confused with .NET Tasks 🙂

One-to-many with no inversion

Firstly, let’s model this kind of relationship:

Now we can transform it into C# classes:

In the Employee class (parent, one-end of the relationship) we define the collection of children, decorated with OneToManyAttribute. Collections types supported as for the time of writing this post by SQLite-Net Extensions are List and Array and can be used as you prefer.

Let’s now see how the child entity (Duty) looks like:

In the Duty class (child, many-end of the relationship) we need to have a foreign key to parent entity defined. For that purpose, we create a property representing it (EmployeeId), decorating it with ForeignKeyAttribute, additionally specifying the type of parent referenced (Employee).

That’s it. We can already use it in our code:

No rocket-science here. What’s interesting for us it how employeeStored looks in the end:

As you can see, GetWithChildren method returned object of type Employee with its Duties collection properly retrieved (containing two duties assigned to the employee before). Moreover, each children has its foreign key (EmployeeId) automatically retrieved from the DB – there is no overhead here, this is simply foreign key field stored in the same SQLite database table (Duties).

One-to-many with inversion (one-to-many + many-to-one)

As previously, let’s first see how the class diagram changes after adding inversion:

What changed is that now each Duty has a property of type Employee.

To realize the above class diagram and make each children (each Duty in our case) knowing about its parent (having a reference to responsible Employee), the only thing we need to do is to add the following property to Duty model class:

[ManyToOne]
public Employee Employee { get; set; }
so the model class finally looks as follows:








  





  [Table("Duties")]



  public class Duty



  {



      [PrimaryKey, AutoIncrement]



      public int Id { get; set; }








      public string Description { get; set; }








      public DateTime Deadline { get; set; }








      [ForeignKey(typeof(Employee))]



      public int EmployeeId { get; set; }








      [ManyToOne]



      public Employee Employee { get; set; }



  }






        view raw

        

          Employee_withInv.cs

        

        hosted with ❤ by GitHub
      


As you can see, we have just created many-to-one relationship by using ManyToOneAttribute. So currently we have the hybrid of both types of relationships within out two models.
The usage in our code doesn’t need to be changed at all. Now, employeeStored entity after being initialized by the same GetWithChildren method as previously, for each Duty in addition to EmployeeId field also contains Employee property properly retrieved by SQLite-Net Extensions:

Again, there is no db overhead here, because when retrieving Employee entity from the database we already have it, so the operation of initializing the Employee entity in each Duty contained within Duties collection doesn’t require any more database querying.
Summary
Today we’ve seen how to model and use one-to-many (with and without inversion) relationships in SQLite database with the usage of SQLite-Net Extensions. The automatic initialization of one-end or many-end entities by the ORM is extremely helpful when dealing with such kind of objects in our app. The amount of code to be written is also minimal.
We’ve covered one-to-many and a hybrid of one-to-many + many-to-one relationships, but of course you can also define just many-to-one relationship – the models would then look as in this Gist. For me, it makes very limited sense, but maybe can be useful in some specific situations.
I invite you to read my other posts about SQLite-Net Extensions ORM, where we also covered the other types of relationships that can be modeled using this library.
As this is already the third post from this series, I’d like to ask you: what do you think about SQLite-Net Extensions ORM? Do you find it useful? Do you use it in your apps? Can you can recommend some other libraries/ORMs to use with SQLite database that allow to model different kind of relationships?
The post SQLite-Net Extensions – one-to-many relationships appeared first on CodeJourney.net.
]]>
					
					https://www.codejourney.net/sqlite-net-extensions-one-to-many-relationships/feed/
			2
		
		
		2251	
		
		
		https://www.codejourney.net/moneyback-3-weeks-before-the-end-of-dsp/
					https://www.codejourney.net/moneyback-3-weeks-before-the-end-of-dsp/#respond
		
		
		Tue, 09 May 2017 22:09:45 +0000
				
		
		
		
		
		
		http://www.dsibinski.pl/?p=2192

					In today’s post I’d like to present what’s the current status of works on MoneyBack application, 3 weeks before the end of DSP competition. In general, the project hasn’t been going as well as I expected and is currently not as advanced as I’d expect it to be at this stage. There are many reasons behind that,…

The post MoneyBack – 3 weeks before the end of DSP appeared first on CodeJourney.net.
]]>
										In today’s post I’d like to present what’s the current status of works on MoneyBack application, 3 weeks before the end of DSP competition.


In general, the project hasn’t been going as well as I expected and is currently not as advanced as I’d expect it to be at this stage. There are many reasons behind that, including lack of time recently and many issues met during development, however, as I already wrote in my DSP introducing post, the most important is to survive with the project to the end of the competition and meet all requirements. The end of DSP doesn’t equal the end of MoneyBack 
That’s why in this post I’d like to show you how the project looks currently and what’s still to be done. Moreover, as I promised before, I’m publishing my Trello board with tasks still to be done within the project.
UI changes
For the last days working on the project I’ve been trying to make the UI of MoneyBack some kind of “final” version, so I can easily extend it adding new functionalities and focus on development instead of spending time on interface design, which is not important for me at the moment. I met many issues with that, which I think came from using some deprecated UI elements I used (e.g. ListView and Adapters) that are not fully-supported in the newest versions of Android. Tutorials are also not really up-to-date for those components. Maybe that’s also because when I wanted to implement displaying a list of entities I looked in Google for something like “xamarin android list of elements” which lead me to this ListView and Adapters tutorial. Instead of firstly reading the whole User Interface tutorial, I wanted to implement displaying the list quickly, so I used those deprecated elements instead of implementing for instance newer and recommended RecyclerView from the beginning, which would be easier extensible now. That definitely teaches me that I should get to know a particular topic/area better and wider before deciding to use one of the possible solutions to solve a problem. Sometimes we want to implement something particular quickly, but later it produces more issues than benefits.
Finally I added tabs (only one tab for People for now, but it will be easy to add new ones now) and ActionBar menu on the main Activity of the app:

On the screenshot above, in the right-upper corner there is “Add” button visible when clicking on ActionBar menu. It opens a new person’s adding screen, on which I made name and email fields mandatory:

People added are then displayed on the main screen of the app in “People” tab:

Code – interesting parts
Using tabs layout in an Activity
It turned out that using tabs-like layout in the Activity is pretty easy. In the AXML layout file for MainActivity I just added a FrameLayout container, within which the tabs and their contents are to be displayed:








  





<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"



    android:orientation="vertical"



    android:layout_width="match_parent"



    android:layout_height="match_parent"



    android:minWidth="25px"



    android:minHeight="25px">



    <FrameLayout



        android:minWidth="25px"



        android:minHeight="25px"



        android:layout_width="match_parent"



        android:layout_height="match_parent"



        android:id="@+id/tabFragmentsContainer" />



</LinearLayout>






        view raw

        

          Main.axml

        

        hosted with ❤ by GitHub
      


I called it tabFragmentsContainer, because within each tab we will display a Fragment. Fragment represents a part of UI and/or behavior, which is displayed within an Activity. You can think of Fragments as smaller pieces, that can be placed onto (inside) Activities.
Basically, I changed PeopleListActivity to PeopleListFragment, which in fact only required changing inheritance from being a child of Activity to inheriting from ListFragment (which is a special kind of Fragment used for displaying lists of elements). The complete code of PeopleListFragment can be found in this public Gist.
In the end, making MainActivity displaying tabs requires setting its ActionBar’s navigation mode to ActionBarNavigationMode.Tabs:
this.ActionBar.NavigationMode = ActionBarNavigationMode.Tabs;
and adding as many tabs as you wish, using for instance code similar to the following one:








  





void AddTab(string tabText, Fragment view)



{



    var tab = this.ActionBar.NewTab();



    tab.SetText(tabText);








    tab.TabSelected += delegate (object sender, ActionBar.TabEventArgs e)



    {



        var fragment = this.FragmentManager.FindFragmentById(Resource.Id.tabFragmentsContainer);



        if (fragment != null)



            e.FragmentTransaction.Remove(fragment);



        e.FragmentTransaction.Add(Resource.Id.tabFragmentsContainer, view);



    };



    tab.TabUnselected += delegate (object sender, ActionBar.TabEventArgs e) {



        e.FragmentTransaction.Remove(view);



    };








    this.ActionBar.AddTab(tab);



}






        view raw

        

          MainActivity_AddTab.cs

        

        hosted with ❤ by GitHub
      


The arguments are name of the tab and an instance of a Fragment we want to have displayed in this tab (PeopleListFragment‘s instance in our case).
Lines 6-12 present what happens when the tab is selected by the user – Fragment‘s instance is added to previously defined tabFragmentsContainer.
Lines 13-15 define callback method on tab deselection – Fragment is removed from the container.
Adding ActionBar with menu to the Activity
Addition of ActionBar with menu button in right-upper corner of the Activity requires creating an XML view in Resources/menu folder first (with one button in our case):








  





<menu xmlns:android="http://schemas.android.com/apk/res/android">



  <item android:id="@+id/menuAdd"



        android:title="@string/menuAddTitle"/>



</menu>






        view raw

        

          ActionBarMenu.xml

        

        hosted with ❤ by GitHub
      


and then in the Activity considered overriding OnCreateOptionsMenu method inflating ActionBar’s menu using created view:








  





public override bool OnCreateOptionsMenu(IMenu menu)



{



    MenuInflater.Inflate(Resource.Menu.ActionBarMenu, menu);



    return base.OnPrepareOptionsMenu(menu);



}






        view raw

        

          MainActivity_OnCreateOptionsMenu.cs

        

        hosted with ❤ by GitHub
      


We also need to define what happens when a menu option is selected. For that purpose, override OnOptionsItemSelected method:








  





public override bool OnOptionsItemSelected(IMenuItem item)



{



    if (item.ItemId == Resource.Id.menuAdd)



    {



        OpenAddingNewPerson();



        return true;



    }








    return base.OnOptionsItemSelected(item);



}






        view raw

        

          MainActivity_OnOptionsItemSelected.cs

        

        hosted with ❤ by GitHub
      


Next coding tasks
From now on you can watch my public Trello board, where I have all tasks to be done in the nearest future in MoneyBack defined. I will try to define deadlines for those tasks as well.
At the moment of writing this post, I’ve started adding a possibility to create Events. I will need to keep many-to-many relationship between Person and Event entities and I will probably use SQLite-Net Extensions library for that. You can expect a separate post about defining relationships using this library soon 
Other tasks left to be done
Apart from coding and developing the project, I still need to fulfill the DSP competition’s requirements. This is my 14th post within DajSiePoznac2017, so 6 posts still to be written. I have the ideas in my mind already and few next posts planned. There are only 3 weeks left to the end of the competition, so those days will need to be more intensive that it recently was, but that’s good, it means that you should be receiving more content here in the coming days and I will be spending more time on developing MoneyBack. Only positive changes, I hope 
Summary
We saw the status of MoneyBack project today. There are still many things to be done in the app to at least fulfill the minimal requirements I assumed in the beginning of the competition. but as I mentioned, the end of DSP is not the end of MoneyBack  On the other hand, DSP is a very good motivator, so I’ll try to make coming days and weeks more intensive. The most important goal for now is to fulfill the competition’s requirements for having 20 posts written until the end of May, which shouldn’t be problematic at all 
I’ve also published my Trello dashboard. You can find a link to it here.
Using this opportunity, I’d like to wish all DSP’17 participants good luck in the last weeks of the competition. I’m constantly reading the others’ blogs and I see a lot of very nice and decent quality stuff appearing in Polish programming community on your blogs and repositories – great job!
Keep your fingers crossed! 
The post MoneyBack – 3 weeks before the end of DSP appeared first on CodeJourney.net.
]]>
					
					https://www.codejourney.net/moneyback-3-weeks-before-the-end-of-dsp/feed/
			0
		
		
		2192	
		
		
		https://www.codejourney.net/solving-performance-issues-with-dottrace/
					https://www.codejourney.net/solving-performance-issues-with-dottrace/#comments
		
		
		Sun, 12 Mar 2017 14:40:24 +0000
				
		
		
		
		
		http://www.dsibinski.pl/?p=728

					If you’ve ever had any performance issues with your .NET applications, you may find this writing interesting. In this post I’d like to present to you a tool produced by JetBrains – dotTrace, showing how using only one of its functionalities can help in finding causes of performance issues in .NET applications. Performance issue on production Months…

The post Solving performance issues with dotTrace appeared first on CodeJourney.net.
]]>
										If you’ve ever had any performance issues with your .NET applications, you may find this writing interesting.

In this post I’d like to present to you a tool produced by JetBrains – dotTrace, showing how using only one of its functionalities can help in finding causes of performance issues in .NET applications.


Performance issue on production
Months ago some of your colleagues implemented an application and delivered it to the Customer. The app is “Countries and Cities Manager”, which is used by some administrator working in local government unit. That’s how it looks:

That’s cool, isn’t it ? This “administration panel” allows to add a new city to the country. User fills name of the city and provides ISO code of the country considered. You may think “who would ever use it and for what?”. True. But first of all, this is just an example  , and secondly, imagine that Russia annexes some big European country. What happens then ? User administrating local government system must use your manager application to insert hundreds or thousands of new cities in Russia. Gotcha!
The next day you receive a ticket in your company’s issues tracking system. Taking into consideration that an average number of cities in east-European countries is ~600, the ticket’s priority is set to “Blocking”. The description of the issue is:

User fills the “City” and “Country ISO Code”, clicks “Add city”
Waits for 10 seconds…
Waits for 20 seconds…
Waits for 30 seconds…
Error message received:



User corrects “RU” to “RUS”
 

Waits for 10 seconds…
Waits for 20 seconds…
Waits for 30 seconds…
 City added!
Scenario repeats for each new city to be added. Performance is unacceptable.

Initially you have no idea what could be the reason. Such simple operation and such terrible performance? Let’s see…
Looking for issue with dotTrace profiling
In such cases, especially if I don’t know the code of the application considered, the very first thing I do it to use dotTrace in order to profile the application when executing bad-performance actions. It provides many features, including:

detecting performance bottlenecks in .NET applications
getting information about calls execution times
analyzing calls order on a timeline
profilling SQL queries and HTTP requests
profiling unit tests
… and many more.

We will use only the simplest profiling mode which is called Sampling.
After you downloaded and installed dotTrace, launch the application you want to profile first. Then, launch dotTrace and in Attach to Process section find and select process of your app and select Sampling mode:

Then, click “Run” button just below Sampling mode selection – you’ll see the profiling session has started:

Next, go to your application and execute bad-performance action. In our case, we click on “Add city” button:

App is not responding – cool! Now we wait those 10, 20, 30 seconds until we get the error message described by the user. To be sure what’s happening, we can click the button few more times (maybe the next executions are better?). I clicked it 3 times are the times were pretty the same.
That’s the end of our bad-performance action, so in dotTrace session’s window click “Get Snapshot and Wait”:

After few seconds, JetBrains dotTrace Performance Viewer opens. Open “Threads Tree” on “All Calls” tab:

In this section we normally see all the threads running within our application during profiling time. We had only one thread, so let’s expand the details to see the time and percentage duration of particular calls from the calls stack:

At this moment, without even opening the code, we already see that method called GetAllEuropeanCountries takes 83.63% of our thread’s execution time. What’s more, we can see that this method is defined in CountriesCitiesDbService which apperently retrieves list of European countries from the database or some external source.
The most percipient of you probably noticed there is Thread.Sleep(Int32) method called one level down – yes, that’s for example’s purposes  , but in real world this would be a database query or a web service call here. The most important is that we know what takes the longest during our operation. Let’s now fix it!
Fixing the issue
Let’s open CountriesCitiesManager solution and use the knowledge we already have from the profiling. We go to the AddNewCity(String, String) method, in which the method for getting countries was called (this is deductible from the calls tree in performance viewer). Here’s the code:








  





private bool AddNewCity(string cityName, string countryIso)



{



    var city = CountriesCitiesDbService.GetNewCity();



    var europeanCountries = CountriesCitiesDbService.GetAllEuropeanCountries();



    var country = europeanCountries.FirstOrDefault(ec => ec.IsoCode.Equals(countryIso));








    if (country == null)



        throw new ArgumentException($"Country with ISO Code {countryIso} does not exist!");








    city.Name = cityName;



    city.Country = country;








    CountriesCitiesDbService.SaveCity(city);



    return true;








}






        view raw

        

          AddNewCity.cs

        

        hosted with ❤ by GitHub
      


AddNewCity method is called every time the “Add city” button is pressed. In line number 4. we are getting the list of all 46 European countries:

How probable it is that the number of countries in Europe changes during our application’s runtime? Very close to 0, I’d say. So there is totally no sense in retrieving the list of EU countries (from the DB, web service or whatever) every time the button is clicked!


We need to cache this list somehow. Let’s simply extract the variable for storing EU countries as read-only class property initialized only when used for the first time:








  





private List<Country> _europeanCountries;








public List<Country> EuropeanCountries => 



    _europeanCountries ?? (_europeanCountries = CountriesCitiesDbService.GetAllEuropeanCountries());






        view raw

        

          EuropeanCitiesProperty.cs

        

        hosted with ❤ by GitHub
      


and use it in AddNewCity method:








  





private bool AddNewCity(string cityName, string countryIso)



{



    var city = CountriesCitiesDbService.GetNewCity();








    var country = EuropeanCountries.FirstOrDefault(ec => ec.IsoCode.Equals(countryIso));



    // the rest of method's code…



}






        view raw

        

          AddNewCity_Corrected.cs

        

        hosted with ❤ by GitHub
      


When the application is now launched, sampling started and button clicked 3 times, we already see the improvement:

AddNewCity takes 65.96% now. This is average value from all the calls we made, so let’s now perform sampling starting from the 2nd click:

The whole AddNewCity method is now taking only 0.41% of the calls during the operation.
Now, if we want to make Customer’s life beautiful, we’d extract initialization of EU countries list to another thread, so it can happen when the application launches and UI is not frozen. We will however stop here. You get the idea, don’t you ? 
Summary
Today we’ve seen how easily – by performing just the simplest profiling session that can be done in dotTrace – it may be to detect what causes real performance issues in .NET application. Of course the example was pretty naive and simple, but I didn’t make it up – many times I’ve already worked with performance issues in production code which were caused by retrieving so called “dictionary” or static data using some API or database connection every time in a loop or repetitive process, which was totally unnecessary. By only implementing some kind of cache (maybe a bit more reasonable than the one we did, which in our case was actually enough), I managed to optimize processes executing time from 3 or 4 hours to several seconds.
Obviously if you’re an experienced developer, you’d probably identify such places in which caching or similar mechanism should be used when writing code. Profiling may however be useful for less experienced programmers or when working with legacy code we didn’t write (or we wrote months or years ago  ). It doesn’t cost much, except the cost of dotTrace itself, which is a part of ReSharper Ultimate, but this tool is just an example, you are not limited to it. For sure there are many other tools that offer the same (or maybe even more?) functionalities.
What are the tools you use for debugging performance issues ? Maybe you also use dotTrace, but can share some experience in using its other functionalities?
The post Solving performance issues with dotTrace appeared first on CodeJourney.net.
]]>
					
					https://www.codejourney.net/solving-performance-issues-with-dottrace/feed/
			3
		
		
		728	
		
		
		https://www.codejourney.net/windows-services-using-topshelf/
					https://www.codejourney.net/windows-services-using-topshelf/#respond
		
		
		Sat, 14 May 2016 18:19:52 +0000
				
		
		
		
		http://www.dsibinski.pl/?p=26

					Hello again 🙂 Visibly this is going to be my first technical post here. I hope you’ll find it useful. Windows Services Recently I came up with a need to create an application, which executes some tasks in the background based on file system’s events. “That’s easy”, one would say. “Build Windows Service app”, he…

The post Building Windows Services using Topshelf appeared first on CodeJourney.net.
]]>
										Hello again 🙂

Visibly this is going to be my first technical post here. I hope you’ll find it useful.


Windows Services
Recently I came up with a need to create an application, which executes some tasks in the background based on file system’s events. “That’s easy”, one would say. “Build Windows Service app”, he would add. Sure, that was also my first idea.
I think that’s very common that we want to create an application without any GUI, which purpose is just to do something in the background. Moreover we’d like it to be independent from the user running it – we want it to be running all the time, without having to manually launch it – especially when the application is running on customer’s environment, where we can’t be logged in all the time running the application.
Here comes the service – long-running executable with no user interface, which runs in its own Windows session. Such application can be:

executed by any user created in the system (e.g. with specific security policies assigned),
started automatically when the system launches,
automatically restarted in case of errors,
at any time started, paused, stopped or restarted.

Even if you’ve never developed any service applications, you are using many of them – check your services.msc to see the list of services currently running in your system.
Topshelf
Visual Studio 2015 provides a template to create Windows Service application and probably I could finish my post here. However, I would like to share with you an even easier way of building services: to do it with Topshelf – a very lightweight open-source library, which purpose it to make services development, debugging and deployment painless. Topshelf allows the developer to focus on service’s functionality, instead of wasting time interacting with .NET Framework’s build-in services support. There is also no more need to install your service using InstallUtil or learn how to attach your Visual Studio debugger to running service to detect issues.
Using Topshelf it’s enough to create console application, add one service class with public Start and Stop methods, configure service’s startup parameters (its name, display name, description etc.) and you’re ready to go. Below we’ll see how to do it.
Creating Windows Service application
After creating console application in Visual Studio, add Topshelf to the project by installing it from Nuget:

Adding service class
Then, create service class, which is responsible for executing your service’s logic:
class FileWatcherService
{
   public bool Start()
   {
      // Initialize your service application here
      return true;
   }


   public bool Stop()
   {
      // Executed when the service is stopped
      // Dispose your objects here etc.
      return true;
   }
}

Both of those methods return boolean to indicate that starting/stopping of the service completed successfully.
Adding service configuration
Now we only need to configure the service’s parameters. Topshelf provides HostFactory class which contains Run method, to which we can pass fluently-created set of settings:

service class instance,
methods to be executed while the service is started/stopped/paused/continued,
service’s name, display name, description,
start/stop timeouts,
startup type (automatic, manual, automatic delayed),
… and many more.

The set-up code should be put in the Main method of Program.cs. In our case, the simplest configuration could look as follows:
static void Main(string[] args)
{
    HostFactory.Run(serviceConfig =&amp;amp;amp;amp;gt;
    {
        serviceConfig.Service&amp;amp;amp;amp;lt;FileWatcherService&amp;amp;amp;amp;gt;(serviceInstance =&amp;amp;amp;amp;gt;
        {
            serviceInstance.ConstructUsing(() =&amp;amp;amp;amp;gt; new FileWatcherService());
            serviceInstance.WhenStarted(execute =&amp;amp;amp;amp;gt; execute.Start());
            serviceInstance.WhenStopped(execute =&amp;amp;amp;amp;gt; execute.Stop());
        });

        serviceConfig.SetServiceName("FileWatcherService");
        serviceConfig.SetDisplayName("File Watcher Service");
        serviceConfig.SetDescription("Service monitoring local folders for file system events");
        serviceConfig.StartAutomatically();

    });
}

Debugging the service
Ease of implementation and configuration of the service application isn’t the only benefit of using Topshelf. Our application has been created as a console app, so it can be simply run and debugged by using F5 button. No more need to attach debugger to the service itself 🙂
Installing the service
Topshelf also adds some “shell layer” to our application. To install executable file (which is the output of our console application) into Windows Service Control Manager (SCM) to be running as a “real” Windows Service, it’s enough to open the command line, navigate to the folder where the .exe file is present and install it passing install as .exe’s parameter (e.g.: FileWatcherService.exe install). After executing it, we can see our service created by looking into services.msc:


It’s similarly easy to uninstall the service.
Not only Windows
The official Topshelf’s project’s page says:

Topshelf works with Mono, making it possible to deploy services to Linux. The service installation features are currently Windows only, but others are working on creating native host environment support so that installation and management features are available as well.

So it’s the next great benefit of Topshelf to use applications built with it on Linux.
More
If you want to check what’s more that Topshelf can do, you can visit project’s official page, check it’s GitHub repository or watch this very good Pluralsight course.
Summary
Summing up, I found Topshelf very handy in developing service applications. One of the major pros of using it is ease of installation – it may become very important in the production environment, especially when dealing with any kind of distributed environment. Moreover, debugging the service as a console application makes fixing bugs quick and comfortable. I recommend everyone to use it in case of a need to write any service application.
The post Building Windows Services using Topshelf appeared first on CodeJourney.net.
]]>
					
					https://www.codejourney.net/windows-services-using-topshelf/feed/
			0
		
		
		26