Data science, I'm sorry to say, often involves cleaning up input data into a usable and uniform format. Command line tools like grep, awk and sed provide an arcane power to manipulate text in files of arbitrary size. Mastering these tools can separate data science novices from data scientists with …
There are comments.
Download IPython notebook for this post.
It's easy enough to make a plot using matplotlib.
import matplotlib.pyplot as plt
import numpy as np
time_point_array = np.arange(0, 5, .1)
y_value_array = np.exp(time_point_array)
plt.plot(time_point_array, y_value_array)
This plot is not great data science. In fact it's poor data …
...read moreThere are comments.
pandas has a lot of lifesaving features for dealing with dates. Here's an example timeseries data file, which happens contains a missing date:
date,temp
11-1-2014,56
11-2-2014,56
11-3-2014,59
11-5-2014,60
11-6-2014,55
Loading and plotting the data in pandas gives this result:
In [0]: %matplotlib inline
In …There are comments.
A common need in data science is to test if a some group of data contains a given value. One specific example would be to test if a word is a stop word.
If the elements of the group exist in a list named group in python …
There are comments.
Often when working with text documents it is useful to filter out words that occur frequently in all documents (e.g. 'the', 'is', ...). These words, called stop words, don't give any special hint about the document's content. The nltk (Natural Language Toolkit) library for python includes a list of stop …
...read moreThere are comments.
The nltk library for python contains a lot of useful data in addition to it's functions. One convient data set is a list of all english words, accessible like so:
from nltk.corpus import words
word_list = words.words()
# prints 236736
print len(word_list)
You will probably first have to download …
...read moreThere are comments.
A friend of mine recently asked me to share some of my experiences in making the transition from a biophysics Ph.D. student to data scientist. I realized there are probably a lot of people interested in making a similar transition who could benefit from my experience.
A year …
...read moreThere are comments.
I put together a list of data science and python videos that I've found to be especaily useful and entertaining. The list is here, also linked from the nav bar up top.
...read moreThere are comments.
I put together a list of data science books I recommended, including a few for preparing for data science interviews. The list is availiable here.
...read moreThere are comments.
« Page 2 / 2