Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": No module named svn). Look in the Trac log for more information.

Changes between Version 2 and Version 3 of logging


Ignore:
Timestamp:
05/16/06 17:09:59 (13 years ago)
Author:
jorge.vargas
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • logging

    v2 v3  
     1NOTE: this is NOT finish 
     2 
     3= Undestanding the Logger = 
     4 
     5As the original post<link to post> said this is a normal python logger so we go to http://docs.python.org/lib/module-logging.html to learn how to use it.  
     6the're 2 places where logging is stored both mark by the tag [logging] your dev/prod.cfg and the config/log.cfg the idea is that log.cfg will have things that don't change based on the location. 
     7 
     8Please note <your package> is use across this doc, on your local copy it should say the name of your project's package 
     9 
     10= Using the Logger = 
     11Basically what you need is a quick read of http://docs.python.org/lib/module-logging.html, the first page is sufficient for now, 
     12 
     13after that two lines in your controllers.py is all you need to get going 
     14 
     15{{{ 
     16log = logging.getLogger("<your package>.controllers") 
     17}}} 
     18 
     19That is the prefered way to create a logger the param string is the name which could be anything, although it's good practice to follow a convention, here TG by default suggest to name the logger after the package/module since this is a unique name. 
     20 
     21the other line you should care about is 
     22 
     23{{{ 
     24log.debug("Happy TurboGears Controller Responding For Duty") 
     25}}} 
     26 
     27Which as you read in http://docs.python.org/lib/module-logging.html u did right? is a log with level 10 you also should know that  
     28 
     29== Why not just use logging.debug(msg)? == 
     30 
     31because we want to be able to do this [http://docs.python.org/lib/multiple-destinations.html Multiple Destinations] so you can later find out if the error is on your side or on the framework just by looking at the logs. You can also send diferent logs to diferent places, in general is a better practice to do it this way, and you will take full advantage of the python logging module. 
     32 
     33= TurboGears .cfg files = 
     34if you are used to the "normal" way of creating loggers, either by logging.basicConfig or [http://docs.python.org/lib/logging-config-fileformat.html config files] you may get confused by this format. 
     35 
     36So lets start with the format this is a ConfigObj and its format is at [http://www.voidspace.org.uk/python/configobj.html#the-config-file-format ConfigObj File Format] for you the framework user the only diference from [http://docs.python.org/lib/module-ConfigParser.html ConfigParser] is the extra nested tags and by now you should know that those generate a dict inside another. On the other hand if you want to explore the goodies of ConfigObj Here are the [http://www.voidspace.org.uk/python/configobj.html docs] 
     37 
     38by the way if you ever want to store python type in a config file [http://www.voidspace.org.uk/python/configobj.html#unrepr-mode unrepr] is what you need which is exactly what TG is doing and yes those are format strings and python lists on the default log.cfg 
     39 
     40== What is this *()  == 
     41think of it as %() 
     42=== Why change it? === 
     43http://trac.turbogears.org/turbogears/browser/tags/0.9a5/turbogears/config.py#L117 and if you still dont believe me  
     44http://trac.turbogears.org/turbogears/browser/tags/0.9a5/turbogears/config.py#L23 now can we go on? 
     45 
     46== logging format == 
     47So the file log.cfg file always starts with  
     48{{{ 
     49[logging] 
     50}}} 
     51 
     52and there is a section in dev/prod.cfg with the same name, they are all load up into the same dict so in practice defining things in either is ok. 
     53 
     54This is the head if the logging cfg, if we ever more this to other file the core will still work thanks to configObj 
     55 
     56then you have 3 sections  
     57{{{ 
     58[[formatters]] 
     59[[handlers]] 
     60[[loggers]] 
     61}}} 
     62 
     63where all are optional although you should at least have "handlers" and "loggers" 
     64after this all lvl 3 are just names which as all dict keys and therefore must be unique 
     65 
     66== Formatters == 
     67=== format key === 
     68each lvl 3 aggregate should have only one key call format which becomes the fmt param of a [http://docs.python.org/lib/node357.html Formatter Object], there you can see all the possible values this can have, as well as what happens went ommited 
     69 
     70== handlers == 
     71=== class key === 
     72each handler must have at least a class key which has to be one of [http://docs.python.org/lib/node345.html Handler Objects] and yes you can subclass it but I think you will have enough with the defaults, also TG will error out since your class is not part of logging.__dict__ 
     73=== args key === 
     74the "optional" args key are the parameters pass to each subclass of Handler 
     75=== level key === 
     76the level key indicates the loggin level of this handler this can be any of the default levels DEBUG, INFO, WARNING, ERROR and CRITICAL or a string with the number, i got a patch to fix this TODO summit patch 
     77=== formatter key === 
     78formatter any lvl 3 name form formatters  
     79if level is ommited level=NOTSET if formatter is omitted the default is used '%(message)s' 
     80 
     81{{{ 
     82... 
     83[[[full_content]]] 
     84format='*(asctime)s *(name)s *(levelname)s *(message)s' 
     85... 
     86[[[debug_out]]] 
     87class='StreamHandler' 
     88level='DEBUG' 
     89args='(sys.stdout,)' 
     90formatter='full_content' 
     91}}} 
     92So now this makes sence We are creating a StreamHandler to sys.stdout with lvl debug using the formatted defined above  
     93 
     94== loggers == 
     95This is a another way to create [http://docs.python.org/lib/node341.html Logger Objects]  You can either define them here or do the calls in your code, it depends on your situation/taste there for I'll make a reference to the method call and you will know how to do it both ways and I'll type it ones 
     96=== qualname === 
     97name of this logger,  
     98log=logging.getLogger(qualname) 
     99if omitted this is the root logger 
     100log=logging.getLogger() 
     101=== level ===  
     102log.setLevel(level) 
     103=== handlers ===  
     104handlers is a list so  
     105for handler in handlers: 
     106 log.addHandlers(handler) 
     107=== propagate ===  
     108 log.propagate=propagate 
     109 
     110== Putting it all together== 
     111 
     112=== in log.cfg === 
     113 
     114{{{ 
     115[logging] 
     116[[formatters]] 
     117[[[message_only]]] 
     118format='*(message)s' 
     119 
     120[[[full_content]]] 
     121format='*(asctime)s *(name)s *(levelname)s *(message)s' 
     122 
     123[[handlers]] 
     124[[[debug_out]]] 
     125class='StreamHandler' 
     126level='DEBUG' 
     127args='(sys.stdout,)' 
     128formatter='full_content' 
     129 
     130[[[access_out]]] 
     131class='StreamHandler' 
     132level='INFO' 
     133args='(sys.stdout,)' 
     134formatter='message_only' 
     135 
     136[[[error_out]]] 
     137class='StreamHandler' 
     138level='ERROR' 
     139args='(sys.stdout,)' 
     140}}} 
     141 
     142We define 2 formatters named message_only and full_content, then we define 3 handlers which all Stream to stdout with diferent levels and using the two formaters we define, and we have no loggers because we are handling that from controllers.py 
     143 
     144=== In dev.cfg === 
     145{{{ 
     146[logging] 
     147 
     148[[loggers]] 
     149[[[your_project]]] 
     150level='DEBUG' 
     151qualname='<your package>' 
     152handlers=['debug_out'] 
     153 
     154[[[allinfo]]] 
     155level='INFO' 
     156handlers=['debug_out'] 
     157 
     158[[[access]]] 
     159level='INFO' 
     160qualname='turbogears.access' 
     161handlers=['access_out'] 
     162propagate=0 
     163}}} 
     164 
     165We define 3 loggers [\[[your_project]]] which is the one we use at the start of this tutorial that prints DEBUG messages to stdout, [\[[allinfo]]] which is the main logger (notice missing qualname) and [\[[access]]] that prints INFO messages to stdout. 
     166 
     167So now that you finally undestand how it works it's time to turn it around and do lots of stuff with it check out (LoggingConfiguration move this to LoggingConfigurationExamples?, and add an explanation) for some nice examples. 
     168 
     169= Custom level Logging = 
     170TODO add patch to add custom levels 
     171 
     172That is the prefered way to create a logger the param string is the name which could be anything, although it's good practice to follow a convention, here TG by default suggest to name the logger after the package/module since this is a unique name. 
     173 
     174the other line you should care about is 
     175{{{ 
     176log.debug("Happy TurboGears Controller Responding For Duty") 
     177}{{ 
     178 
     179Which as you read in http://docs.python.org/lib/module-logging.html u did right? is a log with level 10 you also should know that :) 
     180 
     181== Why not just use logging.debug(msg)? == 
     182 
     183because we want to be able to do this [http://docs.python.org/lib/multiple-destinations.html Multiple Destinations] so you can later find out if the error is on your side or on the framework just by looking at the log. You can also send diferent logs to diferent places, in general is a better practice to do it this way, and you will take full advantage of the python logging module. 
     184 
     185= TurboGears log.cfg = 
     186if you are used to the "normal" way of creating loggers, either by logging.basicConfig or [http://docs.python.org/lib/logging-config-fileformat.html files] you may get confused by this format. 
     187 
     188So lets start with the format this is a ConfigObj and its format is at [http://www.voidspace.org.uk/python/configobj.html#the-config-file-format ConfigObj File Format] for you the framework user the only diference from [http://docs.python.org/lib/module-ConfigParser.html ConfigParser] is the extra nested tags (\[\]) and by now you should know that those generate a dict inside another. On the other hand if you want to explore the goodies of ConfigObj Here are the [http://www.voidspace.org.uk/python/configobj.html docs] 
     189 
     190by the way if you ever want to store python type in a config file [http://www.voidspace.org.uk/python/configobj.html#unrepr-mode unrepr] is what you need which is exactly what TG is doing and yes those are format strings and python lists on the default log.cfg 
     191 
     192== What is this *()  == 
     193think of it as %() 
     194=== Why? === 
     195http://trac.turbogears.org/turbogears/browser/tags/0.9a5/turbogears/config.py#L117 and if you still dont believe me  
     196http://trac.turbogears.org/turbogears/browser/tags/0.9a5/turbogears/config.py#L23 now can we go on? 
     197 
     198== log.cfg format== 
     199So the file always starts with  
     200{{{ 
     201[logging] 
     202}}} 
     203This is the head if the logging cfg, if we ever more this to other file the core will still work thanks to configObj 
     204 
     205then you have 3 sections  
     206{{{ 
     207[[formatters]] 
     208[[handlers]] 
     209[[loggers]] 
     210}}} 
     211 
     212where all are optional although you should at least have "handlers" and "loggers" 
     213after this all lvl 3 are just names which as all dict keys must be unique 
     214 
     215== Formatters == 
     216=== format key === 
     217each lvl 3 aggregate should have only one key call format which becomes the fmt param of a [http://docs.python.org/lib/node357.html Formatter Object], there you can see all the possible values this can have, as well as what happens when ommited 
     218 
     219== handlers == 
     220=== class key === 
     221each handler must have at least a class key which has to be one of [http://docs.python.org/lib/node345.html Handler Objects] and yes you can subclass it but I think you will have enough with the defaults, also TG will error out since your class is not part of the logging module, if you want to make a TG extension and patch this go ahead :) 
     222=== args key === 
     223the "optional" args key are the parameters pass to each subclass of Handler 
     224=== level key === 
     225the level key indicates the loggin level of this handler this can be any of the default levels DEBUG, INFO, WARNING, ERROR and CRITICAL or a string with the number 
     226 
     227=== formatter key === 
     228formatter any lvl 3 name form formatters  
     229if level is ommited level=NOTSET if formatter is omitted the default is used '%(message)s' 
     230 
     231{{{ 
     232... 
     233[[[full_content]]] 
     234format='*(asctime)s *(name)s *(levelname)s *(message)s' 
     235... 
     236[[[debug_out]]] 
     237class='StreamHandler' 
     238level='DEBUG' 
     239args='(sys.stdout,)' 
     240formatter='full_content' 
     241}}} 
     242So now this makes sence We are creating a StreamHandler to sys.stdout with lvl debug using the formatted defined above  
     243 
     244== loggers == 
     245This is a another way to create [http://docs.python.org/lib/node341.html Logger Objects]  You can either define them here or do the calls in your code, it depends on your situation/taste therefore I'll make a reference to the method call and you will know how to do it both ways and I'll type it ones :) 
     246 
     247=== qualname === 
     248name of this logger. 
     249 
     250{{{ 
     251if qualname: 
     252log=logging.getLogger(qualname) 
     253else 
     254log=logging.getLogger() #root logger 
     255}}} 
     256 
     257=== level ===  
     258{{{ 
     259log.setLevel(level) 
     260}}} 
     261=== handlers ===  
     262handlers is a list so  
     263{{{ 
     264for handler in handlers: 
     265 log.addHandlers(handler) 
     266}}} 
     267=== propagate ===  
     268{{{ 
     269log.propagate=propagate 
     270}}} 
     271 
     272== Putting it all together == 
     273 
     274=== in log.cfg === 
     275 
     276{{{ 
     277[logging] 
     278[[formatters]] 
     279[[[message_only]]] 
     280format='*(message)s' 
     281 
     282[[[full_content]]] 
     283format='*(asctime)s *(name)s *(levelname)s *(message)s' 
     284 
     285[[handlers]] 
     286[[[debug_out]]] 
     287class='StreamHandler' 
     288level='DEBUG' 
     289args='(sys.stdout,)' 
     290formatter='full_content' 
     291 
     292[[[access_out]]] 
     293class='StreamHandler' 
     294level='INFO' 
     295args='(sys.stdout,)' 
     296formatter='message_only' 
     297 
     298[[[error_out]]] 
     299class='StreamHandler' 
     300level='ERROR' 
     301args='(sys.stdout,)' 
     302}}} 
     303 
     304We define 2 formatters named message_only and full_content, then we define 3 handlers which all Stream to stdout with diferent levels and using the two formaters we define, and we have no loggers because we are handling that from controllers.py 
     305 
     306=== In dev.cfg === 
     307{{{ 
     308[logging] 
     309 
     310[[loggers]] 
     311[[[your_project]]] 
     312level='DEBUG' 
     313qualname='<your package>' 
     314handlers=['debug_out'] 
     315 
     316[[[allinfo]]] 
     317level='INFO' 
     318handlers=['debug_out'] 
     319 
     320[[[access]]] 
     321level='INFO' 
     322qualname='turbogears.access' 
     323handlers=['access_out'] 
     324propagate=0 
     325}}} 
     326 
     327We define 3 loggers "your_project" which is the one we use at the start of this tutorial that prints DEBUG messages to stdout, "allinfo" which is the main logger (notice missing qualname) and "access" that prints INFO messages to stdout. 
     328 
     329NOTE: your_project was a bug in the code generator this has been fix in r1367. and is part of 0.9a6 
     330 
     331So now that you finally undestand how it works it's time to turn it around and do lots of stuff with it check out (LoggingConfiguration move this to LoggingConfigurationExamples?, and add an explanation) for some nice examples. 
     332 
     333= Custom level Logging = 
     334TODO add patch to add custom levels