fullstackpython.com/data.html at gh-pages · macfire/fullstackpython.com · GitHub

History

223 lines (223 loc) · 26.4 KB

Raw

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

<!DOCTYPE html>

<head>

<title>Data - Full Stack Python</title>

</head>

<body>

<a href="https://www.gumroad.com/l/python-deployments" style="color: #fff">Learning web development? Check out the newly-updated Guide to Deployments book</a>!

</p>

</div>

</div> <div class="container">

<span class="logo-title"><a href="https://www.fullstackpython.com/">Full Stack Python</a></span>

</div>

<!--<a href="/blog.html" class="submenu-item-first">Blog</a> |

<a href="/books.html" class="submenu-item">Books</a> | -->

<a href="/table-of-contents.html">All topics</a>

<a href="/email.html">Newsletter</a>

<a href="https://twitter.com/fullstackpython">@fullstackpython</a>

<a href="https://www.facebook.com/fullstackpython">Facebook</a>

<a href="https://github.com/mattmakai/fullstackpython.com">Source</a>

</div> </div>

</div><div class="row">

<p>Data is an incredibly broad topic but it can be broken down into many

subsections, including (in no particular order):</p>

<ul>

<li>data processing / wrangling</li>

<li>machine learning</li>

<li>data analysis</li>

<li>visualization</li>

<li>geospatial mapping</li>

<li>persistence via <a href="/databases.html">relational databases</a> and

<a href="/no-sql-datastore.html">NoSQL data stores</a></li>

<li><a href="/object-relational-mappers-orms.html">object-relational mappers</a></li>

<li>natural language processing (NLP)</li>

<li>indexing, search and retrieval</li>

</ul>

<p>The Python community has built and continues to create open source libraries

and tutorials for all of the above topics.</p>

<h2>Why is Python a great language choice for data tasks?</h2>

<p>Python has a wide array of open source code libraries available and a

diverse community of people with different backgrounds who contribute to

make those libraries better each day.</p>

<p>In addition, Python data manipulation code can be combined with

<a href="/web-frameworks.html">web frameworks</a> and

<a href="/application-programming-interfaces.html">web APIs</a> to build software

that would be difficult to create with a single other language. For example,

Ruby is a fantastic language for building web applications but its data

analysis and visualization libraries are very limited compared to what

is currently available in the Python ecosystem.</p>

<h2>How did Python become so widely used for working with data?</h2>

<p>Python is a general purpose programming language and can be applied to

many problem areas. Over the past couple of decades, Python has become

increasingly popular in the scientific and financial communities. Projects

such as <a href="http://pandas.pydata.org/">pandas</a> grew out of a hedge-fund while

<a href="http://www.numpy.org/">NumPy</a> and <a href="http://www.scipy.org/">SciPy</a> were

created in academic environments then improved by the broader open source

community.</p>

<p>The question is: why Python was used to created these projects? The answer

is a mix of luck, the growth of the open source community as Python was

maturing and wide adoption by people not formally trained as computer

scientists. The pragmatic syntax and explicit style helped very intelligent

people without programming backgrounds to pick up the language and get their

work done with less fuss than other programming languages. Over time the

code used in the financial world and scientific community was shared at the

same time global open source communities were developing, further spreading

their usage among a broader base of software developers.</p>

<p>There's no doubt some of the momentum behind Python's wide adoption for all

types of data manipulation was that it happened to be the right language in

the right place at the right time. Nevertheless, it was ultimately the hard

work of a massive number of engineers and scientists around the world who

created the incredible mix of data code libraries available today.</p>

<h3>General Python data resources</h3>

<ul>

<li>

<p><a href="http://pydata.org/">PyData</a> is a community for developer and users of

Python data tools. They put on fantastic conferences around the world and fund

the continued development of open source data-related libraries.</p>

</li>

<li>

<p><a href="http://continuum.io/">Continuum Analytics</a> is one of the leading Python

companies that pours a tremendous amount of time and funding into

the data community.</p>

</li>

<li>

<p><a href="http://nbviewer.ipython.org/gist/rpmuller/5920182">A crash course in Python for scientists</a>

provides an overview of the Python language with iPython Notebook for those

in scientific fields.</p>

</li>

<li>

<p>The videos of Travis Oliphant on

<a href="https://www.youtube.com/watch?v=oXRvpBJ-Dkc">Python's Role in Big Data Analytics: Past, Present, and Future</a>

and

<a href="https://www.youtube.com/watch?v=d9Qm3PPoYNQ">Building the PyData Community</a>

give historical perspective on how the Python data tools

have evolved over the past 20ish years based on his first-hand experience

as a leader and member in that community.</p>

</li>

</ul>

<h3>What else would you like to learn about Python and data?</h3>

<p class="under-btn">Tell me about standard relational databases.</p> </div>

</div>

<p class="under-btn">What're these NoSQL data stores hipster developers keep talking about?</p> </div>

</div>

<p class="under-btn">Why is Python a good programming language to use?</p> </div>

</div>

</div><div id="mc_embed_signup">

<h4>Sign up here to receive a monthly email with major updates to this site, tutorials and discount codes for Python books.</h4>

</div>

</div>

</form>

</div>

<p style="font-size: .8em; margin-top: 10px;">Searching for a complete, step-by-step deployment walkthrough? Learn more about <a href="http://www.deploypython.com/">The Full Stack Python Guide to Deployments book</a>.

</p>

</div>

</div><div class="panel">

<h3 class="panel-head">Email Updates</h3>

</div>

<h5>Sign up to get a monthly email with Python tutorials and major updates to this site.</h5>

</div>

</form>

</div>

</div><div class="panel" id="full-toc">

<h3 class="panel-head"><a href="/table-of-contents.html" style="color: #fff;">Table of Contents</a></h3>

</div>

<a href="/introduction.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>1. Introduction</a><a href="/learning-programming.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Learning Programming</a><a href="/why-use-python.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Why Use Python?</a><a href="/python-2-or-3.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Python 2 or 3?</a><a href="/enterprise-python.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Enterprise Python</a><a href="/python-community.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Python Community</a><a href="/best-python-resources.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Best Python Resources</a><a href="/best-python-videos.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Best Python Videos</a><a href="/development-environments.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>2. Development Environments</a><a href="/vim.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Vim</a><a href="/emacs.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Emacs</a><a href="/python-programming-language.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>3. Core Language</a><a href="/generators.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Generators</a><a href="/comprehensions.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Comprehensions</a><a href="/web-development.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>4. Web Development</a><a href="/web-frameworks.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Web Frameworks</a><a href="/django.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Django</a><a href="/flask.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Flask</a><a href="/bottle.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Bottle</a><a href="/pyramid.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Pyramid</a><a href="/morepath.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Morepath</a><a href="/other-web-frameworks.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Other Web Frameworks</a><a href="/web-design.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Web Design</a><a href="/cascading-style-sheets.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Cascading Style Sheets (CSS)</a><a href="/javascript.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>JavaScript</a><a href="/websockets.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>WebSockets</a><a href="/template-engines.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Template Engines</a><a href="/web-application-security.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Web Application Security</a><a href="/static-site-generator.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Static Site Generators</a><a href="/jinja2.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Jinja2</a><a href="/pelican.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Pelican</a><a href="/data.html" class="list-group-item smaller-item active" style='font-family: "Helvetica Neue",sans-serif;'>5. Data</a><a href="/databases.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Relational Databases</a><a href="/no-sql-datastore.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>NoSQL Data Stores</a><a href="/object-relational-mappers-orms.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Object-relational Mappers</a><a href="/postgresql.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>PostgreSQL</a><a href="/mysql.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>MySQL</a><a href="/sqlite.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>SQLite</a><a href="/sqlalchemy.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>SQLAlchemy</a><a href="/peewee.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Peewee</a><a href="/application-programming-interfaces.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>6. Web APIs</a><a href="/api-integration.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>API Integration</a><a href="/api-creation.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>API Creation</a><a href="/twilio.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Twilio</a><a href="/deployment.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>7. Deployment</a><a href="/servers.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Servers</a><a href="/platform-as-a-service.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Platform-as-a-Service</a><a href="/operating-systems.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Operating Systems</a><a href="/web-servers.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Web Servers</a><a href="/wsgi-servers.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>WSGI Servers</a><a href="/source-control.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Source Control</a><a href="/application-dependencies.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Application Dependencies</a><a href="/static-content.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Static Content</a><a href="/task-queues.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Task Queues</a><a href="/configuration-management.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Configuration Management</a><a href="/continuous-integration.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Continuous Integration</a><a href="/logging.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Logging</a><a href="/monitoring.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Monitoring</a><a href="/web-analytics.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Web Analytics</a><a href="/docker.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Docker</a><a href="/caching.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Caching</a><a href="/microservices.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Microservices</a><a href="/devops.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>DevOps</a><a href="/nginx.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Nginx</a><a href="/apache-http-server.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Apache HTTP Server</a><a href="/caddy.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Caddy</a><a href="/green-unicorn-gunicorn.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Green Unicorn (Gunicorn)</a><a href="/ubuntu.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Ubuntu</a><a href="/testing.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>8. Testing</a><a href="/unit-testing.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Unit Testing</a><a href="/integration-testing.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Integration Testing</a><a href="/code-metrics.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Code Metrics</a><a href="/debugging.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Debugging</a><a href="/bots.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Bots</a><a href="/what-full-stack-means.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif;'>9. Meta</a><a href="/change-log.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Change Log</a><a href="/future-directions.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>Future Directions</a><a href="/about-author.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",sans-serif; padding-left: 27px;'>About the Author</a> <a href="/table-of-contents.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",san-serif;background-color:#22B24C; color: #fff;'>...or <span style="border-bottom: 1px dotted;">view the full table of contents</span>.</a>

</div>

</div> <div class="panel">

Major updates are tweeted via

<a href="https://twitter.com/fullstackpython">@fullstackpython</a>.

<hr/>

Need more detailed tutorials than you see here?

<a href="http://www.deploypython.com/">Learn more about The Full Stack Python Guide to Deployments book.</a>

</div>

<h3 class="panel-head"><a href="/table-of-contents.html" style="color: #fff;">Chapters</a></h3>

</div>

<a href="/introduction.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>1. Introduction</a><a href="/development-environments.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>2. Development Environments</a><a href="/python-programming-language.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>3. Core Language</a><a href="/web-development.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>4. Web Development</a> <a href="/data.html" class="list-group-item smaller-item active" style='font-family: "Helvetica Neue",sans-serif;'>» 5. Data</a>

<a href="/application-programming-interfaces.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>6. Web APIs</a><a href="/deployment.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>7. Deployment</a><a href="/testing.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>8. Testing</a><a href="/what-full-stack-means.html" class="list-group-item smaller-item " style='font-family: "Helvetica Neue",sans-serif;'>9. Meta</a> <a href="/table-of-contents.html" class="list-group-item smaller-item" style='font-family: "Helvetica Neue",san-serif;background-color:#22B24C; color: #fff;'>...or <span style="border-bottom: 1px dotted;">view the full table of contents</span>.</a>

</div>

</div></div></div>

<hr/>

</div>

<a href="http://www.mattmakai.com/" class="underline">Matt Makai</a> 2016

</div>

(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){

(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),

m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)

})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-19910497-7', 'auto');

ga('send', 'pageview');

</script><script type='text/javascript'>

var trackOutboundLink = function(url) { ga('send', 'event', 'outbound', 'click', url, {'hitCallback': function () { document.location = url; } }); }

</script>

</body>

</html>

fullstackpython.com/data.html at gh-pages · macfire/fullstackpython.com · GitHub

Expand file tree

Breadcrumbs

data.html

Latest commit

History

Breadcrumbs

data.html

File metadata and controls