Fall 2019

Day 4

5 minutes

Report 1 on US baby names

Due 8:00am Saturday.

True or False?: You can turn in your report late for reduced credit.

False! There is no such thing as a late report.

Quiz

I know someone whose first name is "Kira".

Quiz Name: Kira

Question: Do you think this person is female (F) or male (M)?

Exercise

Examine gender specificity of the last latter of names.

Example here

Web scraping

requests module

requests.get()

Simple exercise: Is [word of your choice] mentioned on the "front" page of today's NY Times?

Example

Grab a price from Amazon

split-and-select for extracting parts of strings

Low-tech method often easier to use than alternatives such as "regular expressions" or beautifulsoup module.

Spoofing user agent

Sometimes you will find Amazon refuses to serve the page to a script (robot). In that case we will need to fake our User Agent.

s = requests.get(url, headers={'User-Agent':ua})


where ua is something like:

ua = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Safari/537.36'


Exercise

What is the total number of seats available in each MTH course this semester, and how many are empty?