Model Caching Approach

Caching Approach using CachedManager and CachedModelMixin

The core module (available at https://github.com/openimis/openimis-be-core_py ) provides a caching mechanism for Django models to reduce database queries and improve performance for frequently accessed data. This is achieved through the CachedManager and CachedModelMixin classes defined in core/utils.py.

Purpose

CachedManager: A custom Django manager that handles querying from cache when possible. It overrides standard query methods to check the cache first, falling back to the database if needed, and stores results in cache for future requests.
CachedModelMixin: A mixin to be inherited by models, which adds cache invalidation logic. It ensures that cache entries are cleared or updated on model save, delete, or other mutations to maintain data consistency.

This approach is particularly useful for read-heavy models where data doesn't change frequently.

How to Use

if you use one of the base class, this part is already covered VersionedModel , HistoryModel, HistoryBusinessModel

To enable caching on a model:

Import the classes from core.utils:
from core.utils import CachedManager, CachedModelMixin
Inherit from CachedModelMixin in your model and assign CachedManager as the objects manager:
class MyModel(CachedModelMixin, models.Model): # Your fields here... objects = CachedManager()

Key Methods and Behavior

CachedManager:
- Overrides get_queryset() to integrate caching logic.
- Uses Django's cache framework (e.g., Memcached or Redis backend configured in settings).
- Key methods include:
  - get_from_cache(pk): Retrieves an instance by primary key from cache or DB.
  - get(**kwarg) if the kwargs have only one element which is part of the UNIQUE_FIELDS
  - filter(**kwarg) if the kwargs have only one element which is part of the UNIQUE_FIELDS
  - Cache keys are typically generated as by an utils.
    - def get_cache_key(model, id): return f"cs_{model.__name__}_{id}"
CachedModelMixin:
- Overrides save() and delete() to invalidate related cache entries.
- Use CACHE_TIMEOUT constant set to 1 day: 3600 * 24

Configuration

Ensure your Django settings have a cache backend configured, e.g.:

CACHES = {
    'default': {
        'BACKEND': os.environ.get('CACHE_BACKEND'),
        'OPTION': os.environ.get("CACHE_OPTIONS", None)
        'LOCATION':  os.environ.get('CACHE_URL'),
    }
}

The default and what we use for test and development is django.core.cache.backends.locmem.LocMemCache

In production system we highly recommend django.core.cache.backends.redis.RedisCache and to set the proper CACHE_URL environment variable

An other 2 cache is using the same backend but does not use the class above:

'coverage' that is dedicated to coverage caching
'location' that save the location tree and user district

Example

For a model in another package (e.g., https://github.com/openimis/openimis-be-%7Bsomepackage%7D_py):

from django.db import models
from core.utils import CachedManager, CachedModelMixin

class ExampleModel(CachedModelMixin, models.Model):
    name = models.CharField(max_length=100)
    
    objects = CachedManager()

Queries like ExampleModel.objects.get(pk=1) will now use caching.

Customization Options

In the model's Meta class, you can define additional attributes to customize the caching behavior:

USE_CACHE = True: A boolean flag to enable (default) or disable caching for this model. Set to False to bypass caching entirely.
UNIQUE_FIELDS = []: A list of field names that are unique (e.g., unique constraints in the model). These allow caching and lookups by those fields in addition to the primary key, improving query efficiency for unique identifiers.
CACHED_FK = []: A list of foreign key field names whose related objects should be pre-cached or included in the caching strategy, reducing subsequent database hits for relations.

Example with Customizations