Simple Django Logging

Post by Saul Shanabrook

I have been having a lot of trouble getting logging in Django to work
properly.

Django logging with a dictionary. Refer to their example
configuration

for an idea of what each of the options does.

I use Sentry to track exceptions. They
also have an example
configuration

for how to use Sentry with Django through the dictionary configuration.

Lots of documentation and examples, sounds great right? Well not really,
I am still trying to understand how to do two things.

​1) Log everything above DEBUG to the console. 2) Log everything above WARNING to Sentry.

I am going to start with just the first problem.

I was hoping that using the root logger along with settings
disable_existing_loggers would allow me to send all the logging through the console.

LOGGING = {  
    'version': 1,
    'disable_existing_loggers': True,
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
        }
    },
    'root': {
        'handlers': ['console', ],
        'level': 'INFO'
    },
}

However, the Django logging did not go through the console with this
configuration. Using
logging-tree to show the layout of my logs with the configuration, I saw that although
the root logger was sending to the console, the django logger was
overriding that to send it to mail.

<--""
   Level INFO
   Handler Stream <open file '<stderr>', mode 'w' at 0x10c3551e0>
   |
   o<--"django"
   |   Handler Stream <open file '<stderr>', mode 'w' at 0x10c3551e0>
   |     Filter <django.utils.log.RequireDebugTrue object at 0x10d058d10>
   |   |
   |   o<--[django.db]
   |   |   |
   |   |   o<--"django.db.backends"
   |   |
   |   o   "django.request"
   |       Level ERROR
   |       Propagate OFF
   |       Handler <django.utils.log.AdminEmailHandler object at 0x10d058e10>
   |
   ...

Then I tried settings the django logger to explicitly propagate to the
root logger:

LOGGING = {  
    'version': 1,
    'disable_existing_loggers': True,
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
        }
    },
    'loggers': {
        'django': {
            'propogate': True
        }
    },
    'root': {
        'handlers': ['console', ],
        'level': 'INFO'
    },
}

And now the when I hit an exception when loading a Django page, it did
print to the console.

<--""
   Level INFO
   Handler Stream <open file '<stderr>', mode 'w' at 0x10ae021e0>
   |
   o<--"django"
   |   |
   |   o<--[django.db]
   |   |   |
   |   |   o<--"django.db.backends"
   |   |
   |   o<--"django.request"
   |
   ...

You can see that now the django logger (and its sub loggers) do not
define and handlers, so that they all propagate to root logger, which
defines the console handler. This works fine, but what about other
libraries? I want to override every libraries logging, not just
django’s, to use the root handlers behavior. After looking at the python
source for configuring logging with a dictionary, I realized that even
with disable_existing_loggers it will not actually reset those
loggers
.
It will only do that if you redefine those loggers, or their parents.

So then my next idea was to get all of the currently configured loggers
in the settings and then explicitly redefine those, so that they would
all propagate

LOGGING = {  
    'version': 1,
    'disable_existing_loggers': True,
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
        }
    },
    'loggers': {},
    'root': {
        'handlers': ['console', ],
        'level': 'INFO'
    },
}

root = logging.root  
existing = root.manager.loggerDict.keys()

for logger in existing:  
    LOGGING['loggers'][logger] = {}

However, I realized that when the the django settings file is called,
the django logger is not set up yet, so that this will not be redefined.

So I finally gave in, and defined the django logger explicitly and
tried to catch all the others by getting them using the existing
loggers.

LOGGING = {  
    'version': 1,
    'disable_existing_loggers': True,
    'handlers': {
        'console': {
            'level': 'INFO',
            'class': 'logging.StreamHandler',
        }
    },
    'loggers': {
        'django': {},
    },
    'root': {
        'handlers': ['console', ],
        'level': 'INFO'
    },
}


# Get all the existing loggers
root = logging.root  
existing = root.manager.loggerDict.keys()

# Set them explicitly to a blank value so that they are overidden
# and propogate to the root logger
for logger in existing:  
    LOGGING['loggers'][logger] = {}

Then if you want to use Sentry, just add that as a handler after
defining the original LOGGING and add it to the root’s handlers.

INSTALLED_APPS += (  
    'raven.contrib.django.raven_compat',
)
LOGGING['handlers']['sentry'] = {  
    'level': 'ERROR',
    'class': 'raven.contrib.django.raven_compat.handlers.SentryHandler',
}
LOGGING['root']['handlers'].append('sentry')  

Or you could combine it and define it in one logging dict, I choose to
add sentry afterword, so that way I can turn it off when developing
locally so that logs are only sent to my console.