fullstackpython.com/logging.html at pre-jan-content-reorder · BasmaRG/fullstackpython.com · GitHub

History

291 lines (290 loc) · 16.6 KB

Raw

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

<!DOCTYPE html>

<title>Logging - Full Stack Python</title>

<!--[if lt IE 9]>

<![endif]-->

<style>

.toc-indent {padding-left: 25px;}

.toc-more-indent {padding-left: 37px;}

</style>

var _gaq = _gaq || [];

_gaq.push(['_setAccount', 'UA-19910497-7']);

_gaq.push(['_trackPageview']);

(function() {

var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;

ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';

var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);

})();

</script> </head>

<body>

<span class="logo-title"><a href="/">Full Stack Python</a></span>

</div>

</div> <div class="row">

<h1>Logging</h1>

<p>Logging saves output such as errors, warnings and event information to

files for debugging purposes. </p>

<h2>Why is logging important?</h2>

<p>Runtime exceptions that prevent code from running are important to log to

investigate and fix the source of the problems. Informational and debugging

logging also helps to understand how the application is performing even if

code is working as intended.</p>

<h2>Logging levels</h2>

<p>Logging is often grouped into several categories:</p>

<ol>

<li>Information</li>

<li>Debug</li>

<li>Warning</li>

<li>Error</li>

</ol>

<p>Logging errors that occur while a web framework is running is crucial to

understanding how your application is performing. </p>

<h2>Logging aggregators</h2>

<p>When you are running your application on several servers, it is helpful

to have a monitoring tool called a "logging aggregator". You can configure

your application to forward your system and application logs to one location

that provides tools for viewing, searching, and monitoring logging events

across your cluster. </p>

<p>Another advantage of log aggregatortion tools is they allow you to set up

custom alerts and alarms so you can get notified when error rates breach a

certain threshold.</p>

<h3>Open source log aggregators</h3>

<ul>

<li>

<p><a href="https://github.com/getsentry/sentry">Sentry</a> started as a Django-only

exception handling service but now has separate logging clients to cover

almost all major languages and frameworks. It still works really well for

Python-powered web applications and is often used in conjunction with other

monitoring tools. <a href="http://raven.readthedocs.org/en/latest/">Raven</a> is open

source Python client for Sentry.</p>

</li>

<li>

<p><a href="http://graylog2.org/">Graylog2</a> provides a central server for log

aggregation as well as a GUI for browsing and searching through log events.

There are libraries for most major languages, including python. Saves data

in Elasticache.</p>

</li>

<li>

<p><a href="http://logstash.net/">Logstash</a> Similar to Graylog2, logstash offers

features to programatically configure log data workflows.</p>

</li>

<li>

<p><a href="https://github.com/facebook/scribe">Scribe</a> A project written by Facebook

to aggregate logs. It's designed to run on multiple servers and scale with

the rest of your cluster. Uses the Thrift messaging format so it can be

used with any language. </p>

</li>

</ul>

<h3>Hosted logging services</h3>

<ul>

<li>

<p><a href="https://www.loggly.com/">Loggly</a> Loggly is a third party cloud based

application that aggregates logs. They have instructions for every major

language, including python. It includes email alerting on custom searches. </p>

</li>

<li>

<p><a href="https://papertrailapp.com/">Papertrail</a> Paper trail is similar to both

loggly and splunk and provides integration with S3 for long term storage.</p>

</li>

<li>

<p><a href="http://www.splunk.com/">Splunk</a> Splunk offers third party cloud and self

hosted solutions for event aggregation. It excells at searching and data

mining any text based data. </p>

</li>

<li>

<p><a href="http://raygun.io/">Raygun</a> logs errors and provides immediate notification

when issues arise.</p>

</li>

<li>

<p><a href="https://www.scalyr.com/">Scalyr</a> provides log aggregation, dashboards,

alerts and search in a user interface on top of standard logs.</p>

</li>

<li>

<p>There is a <a href="https://www.getsentry.com/welcome/">hosted version of Sentry</a>

in case you do not have the time to set up the open source project yourself.</p>

</li>

</ul>

<h2>Logging resources</h2>

<ul>

<li>

<p>This

<a href="http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/">intro to logging</a>

presents the Python logging module and how to use it.</p>

</li>

<li>

<p><a href="http://www.hybridcluster.com/blog/logging-storytelling/">Logging as Storytelling</a>

is a multi-part series working the analogy that logs should read like

a story so you can better understand what's taking place in your web

application.

<a href="http://www.hybridcluster.com/blog/logging-storytelling-lets-add-action/">Part 2 describes actions</a>

and

<a href="http://www.hybridcluster.com/blog/logging-storytelling-3-types/">part 3 talks about types</a>.</p>

</li>

<li>

<p><a href="https://lukasa.co.uk/2014/05/A_Brief_Digression_About_Logging/">A Brief Digression About Logging</a>

is a short post that gets Python logging up and running quickly.</p>

</li>

<li>

<p><a href="https://hynek.me/articles/taking-some-pain-out-of-python-logging/">Taking the pain out of Python logging</a>

shows a logging set up with uWSGI.</p>

</li>

<li>

<p>Django's 1.3 release brought unified logging into project configurations.

This <a href="http://www.djm.org.uk/how-to-log-file-django-13-and-above/">post shows how to set up logging</a>

in a project's settings.py file. Caktus Group also has a nice tutorial on

<a href="http://www.caktusgroup.com/blog/2013/09/18/central-logging-django-graylog2-and-graypy/">central logging with graypy and Graylog2</a>.</p>

</li>

</ul>

<h2>Logging learning checklist</h2>

Read how to integrate logging into your web application framework. </p>

Ensure errors and anomalous results are logged. While these logs can be stored

in <a href="/monitoring.html">monitoring</a> solutions, it's best to have your own log

storage location to debug issues as they arise to complement other monitoring

systems.</p>

Integrate logging for system events you may need to use for debugging purposes

later. For example, you may want to know the return values on functions when

they are above a certain threshold. </p>

<h3>Logging isn't enough. How do I analyze more data about the app?</h3>

How can I monitor my live app with tools other than just logs?

</p>

</div>

</a>

I want to learn more about the users of my application.

</p>

</div>

Something in the logs looks strange. How do I learn about security?

</p>

</div>

</div> <div id="mc_embed_signup">

<h4>Interested in major updates to this site or an alert when a complete Full Stack Python book with detailed example code is released? Sign up here and you'll an occasional email only when there's big news to report.</h4>

</div>

</div>

</form>

</div>

<h3 class="panel-head"><a href="/table-of-contents.html" style="color: #fff;">Table of Contents</a></h3>

</div>

<a href="/introduction.html" class="list-group-item smaller-item ">Introduction</a>

<a href="/web-frameworks.html" class="list-group-item smaller-item ">Web Frameworks</a>

<a href="/django.html" class="list-group-item smaller-item ">Django</a>

<a href="/flask.html" class="list-group-item smaller-item ">Flask</a>

<a href="/bottle.html" class="list-group-item smaller-item ">Bottle</a>

<a href="/morepath.html" class="list-group-item smaller-item ">Morepath</a>

<a href="/other-web-frameworks.html" class="list-group-item smaller-item ">Other Web Frameworks</a>

<a href="/deployment.html" class="list-group-item smaller-item ">Deployment</a>

<a href="/servers.html" class="list-group-item smaller-item ">Servers</a>

<a href="/platform-as-a-service.html" class="list-group-item smaller-item ">Platform-as-a-service</a>

<a href="/operating-systems.html" class="list-group-item smaller-item ">Operating Systems</a>

<a href="/web-servers.html" class="list-group-item smaller-item ">Web Servers</a>

<a href="/wsgi-servers.html" class="list-group-item smaller-item ">WSGI Servers</a>

<a href="/source-control.html" class="list-group-item smaller-item ">Source Control</a>

<a href="/application-dependencies.html" class="list-group-item smaller-item ">Application Dependencies</a>

<a href="/databases.html" class="list-group-item smaller-item ">Databases</a>

<a href="/no-sql-datastore.html" class="list-group-item smaller-item ">NoSQL Data Stores</a>

<a href="/web-design.html" class="list-group-item smaller-item ">Web Design</a>

<a href="/cascading-style-sheets.html" class="list-group-item smaller-item ">Cascading Style Sheets</a>

<a href="/javascript.html" class="list-group-item smaller-item ">JavaScript</a>

<a href="/continuous-integration.html" class="list-group-item smaller-item ">Continuous Integration</a>

<a href="/code-metrics.html" class="list-group-item smaller-item ">Code Metrics</a>

<a href="/configuration-management.html" class="list-group-item smaller-item ">Configuration Management</a>

<a href="/static-content.html" class="list-group-item smaller-item ">Static Content</a>

<a href="/caching.html" class="list-group-item smaller-item ">Caching</a>

<a href="/task-queues.html" class="list-group-item smaller-item ">Task Queues</a>

<a href="/application-programming-interfaces.html" class="list-group-item smaller-item ">Application Programming Interfaces</a>

<a href="/api-integration.html" class="list-group-item smaller-item ">API Integration</a>

<a href="/api-creation.html" class="list-group-item smaller-item ">API Creation</a>

<a href="/logging.html" class="list-group-item smaller-item active">Logging</a>

<a href="/monitoring.html" class="list-group-item smaller-item ">Monitoring</a>

<a href="/web-analytics.html" class="list-group-item smaller-item ">Web Analytics</a>

<a href="/web-application-security.html" class="list-group-item smaller-item ">Web Application Security</a>

<a href="/best-python-resources.html" class="list-group-item smaller-item ">Best Python Resources</a>

<a href="/development-environments.html" class="list-group-item smaller-item ">Development Environments</a>

<a href="/about-author.html" class="list-group-item smaller-item ">About the Author</a>

<a href="/change-log.html" class="list-group-item smaller-item ">Change Log</a>

<a href="/future-directions.html" class="list-group-item smaller-item ">Future Directions</a>

<a href="/what-full-stack-means.html" class="list-group-item smaller-item ">What "Full Stack" Means</a>

<a href="/why-use-python.html" class="list-group-item smaller-item ">Why Use Python?</a>

<a href="/websockets.html" class="list-group-item smaller-item ">WebSockets</a>

<a href="/docker.html" class="list-group-item smaller-item ">Docker</a>

</div>

<div class="panel-heading"><h3 class="panel-head">Logging</h3></div>

<a href="/">Full Stack Python</a> is an open book that explains

each Python web application stack layer and provides the

best web resources for those topics.

<hr/>

There's a work-in-progress

<a href="/full-stack-python-map.pdf">subjects map (.pdf)</a>

that visually lays out each chapter in addition to the table of

contents found below.

<hr/>

Need more detailed tutorials and walkthroughs than you see here?

<a href="/email.html">Sign up for an email alert when that content is created.</a>

<hr/>

<a href="http://twitter.com/mattmakai">Matt Makai</a> built

this site with greatly appreciated assistance from community

<a href="https://github.com/makaimc/fullstackpython.github.com/pulls">pull requests</a>.

</div>

</div></div>

<hr/>

<a href="http://www.mattmakai.com/" class="underline">Matt Makai</a>

2015

</div>

</body>

</html>

fullstackpython.com/logging.html at pre-jan-content-reorder · BasmaRG/fullstackpython.com · GitHub

Expand file tree

Breadcrumbs

logging.html

Latest commit

History

Breadcrumbs

logging.html

File metadata and controls