Week 03: Python Libraries

Page

Content:

This week we will introduce some of the cool libraries for processing data with Python. The libraries we will be looking at are:

    • Numpy/Scipy for handling matrices and for scientific computing
    • Scikit-learn for machine learning
    • Cython for writing C extensions for Python code
    • Pandas for managing and working with data structures and data analysis workflows

Using some of these libraries makes Python a really powerful tool for data analysis, but each library has tons of functionality. During this week we will focus on outlining the capabilities of each library, so that you will pick the right subset the next time you are facing a problem.

Learning objectives:

After this week, you are supposed to know:

  • How to install and load each of the libraries listed above.
  • What the capabilities are of each of the above Python libraries.
  • How to use basic features of each library listed above.

Resources:

Exercises:

Exercise 3.1 (numpy):

Write a script which reads a matrix from a file like this one and solves the linear matrix equation Ax=b where b is the last column of the input-matrix and A is the other columns. It is okay to use the solve()-function from numpy.linalg. Does the result make sense?

Exercise 3.2 (scipy):

Write a script that reads in this list of points (x,y), fits/interpolates them with a polynomial of degree 3. Solve for the (real) roots of the polynomial numerically using Scipy’s optimization functions (not the root function in Numpy). Does the result make sense (plot something to check).

Exercise 3.3 (pandas):

Do the first two exercises (Todo’s) at the bottom of http://byumcl.bitbucket.org/bootcamp2013/labs/pandas.html

Exercise 3.4 (scikit-learn):

Last week you read in a dataset for this Kaggle competition and created a bag-of-words representation on the review strings. Train a logistic regression classifier for the competition using your bag-of-words features (and possibly some of the others) to predict the variable “requester_received_pizza”. For this exercise, you might want to work a little bit more on your code from last week. Use 90% of the data as training data and 10% as test data.

If you don’t know anything about machine learning, try to Google a bit and figure out what training and test data is, and how you train a classifier.

How good is your classifier? Discuss the performance of the classifier.

Exercise 3.5 (cython):

Write a simple Python function for computing the sum \frac{1}{1^2} + \frac{1}{2^2} + \frac{1}{3^2} + \ldots with 10,000 terms (this should be around 1.644), 500 times in a row (to make the execution time measurable). Now compile the code with Cython and see how much speedup you can achieve by this. Remember to declare your variable types.

One thought on “Week 03: Python Libraries

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s