Implementation of caching in C# .

NET
In software development, caching is a commonly used pattern. It is a simple concept, but one that is very effective.
In this case, the results of the operation are reused. After we perform a heavy operation, the result will be saved to
our cache container. Instead of performing the heavy operation again the next time we require that result, we will
pull it from the cache container.

You may need to consult the database in order to obtain a person's avatar, for example. This will prevent you from
having to perform that trip again and again, as we will save the Avatar in the cache, and pull it from memory as
needed.

When data changes infrequently, caching is an excellent solution. The best course of action is to never change. A
cache should not be used for data that is constantly changing, such as the current computer's time.

Distributed, persistent, and in-process caches

Cached items include:

 A single-process cache is implemented in part using in-memory cache. Processes die along with their caches. Each
server will have its own cache if the same process is run across multiple servers.
 Cache that persists outside of the process memory is persistent in-process cache. Files and databases can contain
it. You can still maintain the cache by restarting the process. Whenever you get an expansive cached item, as well
as your process tends to restart a lot.
 Several machines can share a cache with Distributed Cache. Many servers are usually involved. An external service
stores it in a distributed cache. A cache item can also be used by other servers if it is saved on one server. It's great
to use services like Redis.
Our focus will be on in-process caching.

Poor Implementation

C# implementation of a simple cache:

public class NaiveCache<TItem>

{
Dictionary<object, TItem> _cache = new Dictionary<object, TItem>();

public TItem GetOrCreate(object key, Func<TItem> createItem)

{
if (!_cache.ContainsKey(key))
{
_cache[key] = createItem();
}
return _cache[key];
}
 }

Usage:

var _avatarCache = new NaiveCache<byte[]>();

// ...
var myAvatar = _avatarCache.GetOrCreate(userId, () => _database.GetAvatar(userId));

Solving a crucial problem with this code. User avatars can be obtained only with the first request. It then saves
avatar data (byte[]) in process memory. We will use the avatar from memory for subsequent requests, saving time
and resources.

No programming task is as simple as it seems. There are several problems with the above solution. Thread-safety is
a concern with this implementation. Using multiple threads may result in exceptions. A cached item will also
remain in memory forever, which is bad.

Taking items out of cache has these benefits:

1. It is possible that cache can consume a lot of memory, eventually leading to crashes.
2. Memory pressure (GC pressure) is caused by high memory use. This affects performance due to increased garbage
collection work.
3. Changing data may require refreshing the cache. This should be supported by our caching infrastructure.
A cache framework's Eviction policy (aka Removal policy) tackles these problems. According to some logic, this is
how items get removed from cache. A typical eviction policy is:

 After a defined period of time, an item will be removed from cache by Absolute Expiration.
 Cache items are automatically removed if they haven't been accessed for a set amount of time. As long as I use
the item every 30 seconds, the item will stay in cache for 1 minute. A minute after I stop using it, it is evicted.
 This policy limits the cache memory size.
Having identified our needs, now let's move to better solutions.

Good Solutions

To my chagrin as a blogger, Microsoft's great cache already exists. Compared to building a similar implementation
I, writing this blog post was easier.

Here's how you can leverage Microsoft's solution, and how to improve it in some situations.

System.Runtime.Caching/MemoryCache vs Microsoft.Extensions.Caching.Memory

Microsoft provides two different caching solutions via NuGet packages. They are both excellent. Following
Microsoft's recommendations, we recommend using Microsoft.Extensions.Caching.Memory in order to enhance its
integration with Asp. NET Core. The dependency injection mechanism of ASP .NET Core can easily accommodate
this.
The following is an example of using Microsoft.Extensions.Caching.Memory:

public class SimpleMemoryCache<TItem>

{
private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions());

public TItem GetOrCreate(object key, Func<TItem> createItem)

{
TItem cacheEntry;
if (!_cache.TryGetValue(key, out cacheEntry))// Look for cache key.
{
// Key not in cache, so get data.
cacheEntry = createItem();

// Save data in cache.

_cache.Set(key, cacheEntry);
}
return cacheEntry;
}
}

Usage:

var _avatarCache = new SimpleMemoryCache<byte[]>();

// ...
var myAvatar = _avatarCache.GetOrCreate(userId, () => _database.GetAvatar(userId));

My own NaiveCache is very similar to this, so what has changed? In any case, this implementation is thread-
safe. This method can be safely invoked from multiple threads simultaneously.

A second feature of MemoryCache is that it supports all the eviction policies that we discussed earlier. The
following is an example:

IMemoryCache with eviction policies:

public class MemoryCacheWithPolicy<TItem>

{
private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions()
{
SizeLimit = 1024
});

public TItem GetOrCreate(object key, Func<TItem> createItem)

 {
TItem cacheEntry;
if (!_cache.TryGetValue(key, out cacheEntry))// Look for cache key.
{
// Key not in cache, so get data.
cacheEntry = createItem();

var cacheEntryOptions = new MemoryCacheEntryOptions()

.SetSize(1)//Size amount
//Priority on removing when reaching size limit (memory pressure)
.SetPriority(CacheItemPriority.High)
// Keep in cache for this time, reset time if accessed.
.SetSlidingExpiration(TimeSpan.FromSeconds(2))
// Remove from cache after this time, regardless of sliding expiration
.SetAbsoluteExpiration(TimeSpan.FromSeconds(10));

// Save data in cache.

_cache.Set(key, cacheEntry, cacheEntryOptions);
}
return cacheEntry;
}
}

Taking a closer look at the new features:

1. MemoryCacheOptions now include a SizeLimit option. This adds a policy based on the size of the cache container.
There is no unit of measure for size. Rather than setting the size amount per cache entry, we should set the
amount per cache entry. With SetSize(1), we are setting the amount to 1 each time. This means that there is a limit
of 1024 items in the cache.
2. Which cache item should be removed when the size limit is reached? .SetPriority(CacheItemPriority.High) can be
used to set priority. Low, Normal, High, and NeverRemove are the different levels.
3. Setting sliding expiration to 2 seconds has been added with SetSlidingExpiration(TimeSpan.FromSeconds(2)). In
other words, if an item is not accessed within two seconds, it will be removed from the system.
4. In order to set absolute expiration to 10 seconds, the method SetAbsoluteExpiration(TimeSpan.FromSeconds(10))
was added. Consequently, if it is not already removed, this item will be removed within 10 seconds.

The RegisterPostEvictionCallback delegate can be set in addition to the options in the example, and will be invoked
when an item is evicted.

That is an impressive list of features. The question arises whether there is anything left to add. Several things need
to be addressed.
 Problems and features that are missing

Some key components of this implementation are missing.

1. There is no real way to monitor GC pressure even though the size limit can be changed. Also, caching does not
actually monitor GC pressure. In the event that we did monitor it, we would be able to tighten policy when there is
a high level of pressure, and loosen policy when the pressure is low.
2. Added new threads of the same item at the same time may not wait for the first thread to complete before
requesting the same item again. It is likely that items will be created more than once. Suppose that through
caching, we may be able to get an avatar from the database within 10 seconds if we are caching it. It is possible to
get a cached avatar 2 seconds after the first request, but it will still run another query in the database to figure out
whether the avatar has been cached (it has not yet).

As for the first problem of GC pressure, there are several techniques and heuristics that can be used to monitor
GC pressure. If you are interested in reading an article that discusses some helpful methods for finding, fixing, and
avoiding memory leaks in C#.NET, please read Find, Fix, and Avoid Memory Leaks in C#.NET: 8 Best Practices.
There is an easier solution to the second problem. It is factually possible to implement MemoryCache in such a
way as to solve it completely:

public class WaitToFinishMemoryCache<TItem>

{
private MemoryCache _cache = new MemoryCache(new MemoryCacheOptions());
private ConcurrentDictionary<object, SemaphoreSlim> _locks = new ConcurrentDictionary<object,
SemaphoreSlim>();

public async Task<TItem> GetOrCreate(object key, Func<Task<TItem>> createItem)

{
TItem cacheEntry;

if (!_cache.TryGetValue(key, out cacheEntry))// Look for cache key.

{
SemaphoreSlim mylock = _locks.GetOrAdd(key, k => new SemaphoreSlim(1, 1));

await mylock.WaitAsync();
try
{
if (!_cache.TryGetValue(key, out cacheEntry))
{
// Key not in cache, so get data.
CacheEntry = await createItem();
_cache.Set (key, cacheEntry);
}
}
Finally
{
 mylock.Release ();
}
}
Return cacheEntry;
}
}

Usage:

Var _avatarCache = new WaitToFinishMemoryCache<byte []> ();

// ...
Var myAvatar =
Await _avatarCache.GetOrCreate (userId, async () => await _database.GetAvatar (userId));

If another thread is currently working on the same object, you will have to wait for it to finish before you can get it.
The other thread will cache the result for you.

How the code works

A new item can only be created this way. The key goes with the lock. On another thread, we can get John's or
Sarah's cached values if we're waiting for Alex's avatar.

Locks are stored in the dictionary _locks. SemaphoreSlim, which works with async/wait, replaces regular locks.

It's checked twice (!_cache.TryGetValue (key, out cacheEntry)). A single creation is ensured by the one inside the
lock. Optimization is the one outside the lock.

If WaitToFinishMemoryCache is used

There is some overhead in this implementation. It's not always necessary to do so.

WatchToFinishMemoryCache when:

 When there is a cost associated with the creation of an item.

 If an item takes a long time to create.
 Create items once per key when possible.

When using WaitToFinishMemoryCache:

 Multithreading isn't a problem here.

 Item creation isn't a problem for you. It wouldn't matter much if we did one more run of the database.
Taking a Moment to Summarize

A cache pattern is one of the most powerful patterns in programming. As well as being dangerous, it also comes
with its own set of complications. In the event your cache more than is necessary, you can put excessive pressure
on the GC. It is possible to cause performance problems when you cache too little data. Additionally, there is the
possibility of utilizing distributed caching systems, which is quite exciting. It's always something new to learn when
it comes to software development, but I guess that's what it's like.