Report 2 pointers

Contents

Pandas for concatenation of data columns

concatdemo.png

Do NOT use choropleths for "extensive" quantities

For example, not appropriate for tons of CO2 emitted or for total sales, etc.

choropleth_extensive.png

Use only for "intensive" properties, like densities, percentages, etc.: people per square km, tonnes of CO2 emitted per 1000 persons or per sq km

Key question to ask yourself: If two (neighboring) counties decided to merge (or if one county split into two) would the picture change in a sensible way?

For future reference, if you do want to represent an extensive quantity, possible options (for another report) are:

disks (or other objects) whose are is proportional to the represented quantity:

hisp_pies.gif

or a "cartogram" in which areas are distorted so as to be proportional to the represented quantity:

cartogram_PaullHennig2016WorldMap.OAha.CC-BY-4.0.jpg

(source)

But for this report, you will be making a choropleth map of an intensive quantity.

Do not use black as a choropleth background color

unless black is your page color. It is hard on the reader's eyes to have a picture with a black background embedded in a white page.

2D color maps

Let's explore this.

2017_03_02/colormaps1.jpg 2017_03_02/colormaps2.jpg 2017_03_02/colormaps3.jpg

Make the most revealing combinations/comparisons of data

To show correlation (or not) instead of a 2D color map with redness for poverty and blueness for birthrate, consider using color as a function of the (log?) ratio.

Avoid code repetition

Repeating makes your code (i) difficult to maintain reliably, (ii) difficult and time-consuming for others to read and understand, and (ii) bigger than it has to be.

Instead of this:

      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Ancestry:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Ancestry[item.attrib['id']][0]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('reportedanc.svg', 'w')
      o.write(news)
      o.close()

      # American Roots
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Ancestry:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Ancestry[item.attrib['id']][1]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('American.svg', 'w')
      o.write(news)
      o.close()

      # Arab Roots
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Ancestry:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Ancestry[item.attrib['id']][2]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('Arab.svg', 'w')
      o.write(news)
      o.close()

      # West Europe Roots
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Ancestry:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Ancestry[item.attrib['id']][3]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('WesternEurope.svg', 'w')
      o.write(news)
      o.close()

      #Eastern Europe Roots
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Ancestry:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Ancestry[item.attrib['id']][4]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('Eastern Europe.svg', 'w')
      o.write(news)
      o.close()

      # French Canadian Roots
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Ancestry:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Ancestry[item.attrib['id']][5]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('FrenchCanadian.svg', 'w')
      o.write(news)
      o.close()

      # High School or Higher
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Education:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Education[item.attrib['id']][0]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
      news= etree.tostring(doc)
      o=open('HS.svg', 'w')
      o.write(news)
      o.close()

      # Bachelors or Higher
      for item in doc:
          if item.attrib['id'].isdigit():
              if item.attrib['id'] in Education:
                      s = 'font-size:12px;fill:'
                      s += myf(float(Education[item.attrib['id']][1]))
                      s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                      item.attrib['style']= s
news= etree.tostring(doc)
      o=open('College.svg', 'w')
      o.write(news)
      o.close()

write this:

def foo(thing, i, filename):
      for item in doc:
               if item.attrib['id'].isdigit():
                   if item.attrib['id'] in thing:
                           s = 'font-size:12px;fill:'
                           s += myf(float(thing[item.attrib['id']][i]))
                           s += ';fill-rule:nonzero;stroke:#000000;stroke-opacity:1;stroke-width:0.1;stroke-miterlimit:4;stroke-dasharray:none;stroke-linecap:butt;marker-start:none;stroke-linejoin:bevel'
                           item.attrib['style']= s
      news= etree.tostring(doc)
      o=open(filename, 'w')
      o.write(news)
      o.close()

foo(Ancestry ,0,   'reportedanc.svg')
foo(Ancestry ,1,      'American.svg')
foo(Ancestry ,2,          'Arab.svg')
foo(Ancestry ,3, 'WesternEurope.svg')
foo(Ancestry ,4,'Eastern Europe.svg')
foo(Ancestry ,5,'FrenchCanadian.svg')

foo(Education,0,'HS.svg')
foo(Education,1,'College.svg')

Likewise, never have two instances of a number that are always supposed to be the same: give it a name and use the name in the multiple places:

DO NOT DO THIS:

a = zeros((99,99))
# ...
b = zeros(99)

DO THIS:

n = 99
a = zeros((n,n))
# ...
b = zeros(n)

Color bars

Need these for maps to be quantitative.

Ways: one example given in class the previous week.

Once in a while, point-and-click is the way to go!

Other way: edit SVG blank map with Inkscape to insert a (blank) color bar for subsequent modification by our Python code:

colorbar_inkscape.png

Here is some of the new SVG representing the color bar we just added (viewd in meld):

colorbar_meld.png

Subtle point: even though the rectangles are perfectly aligned, the background will bleed through along the junction (due to anti-aliasing). To prevent that, you can set

shape-rendering="crispEdges"

on the shapes you want it to apply to. Compare default (left) and crisp (right) below:

adjacent_rectangles.svg adjacent_rectangles_crisp.svg

Perhaps even better (if your color map is a piecewise linear gradient), you could use SVG's linear gradient feature:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg
   xmlns:dc="http://purl.org/dc/elements/1.1/"
   xmlns:cc="http://creativecommons.org/ns#"
   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
   xmlns:svg="http://www.w3.org/2000/svg"
   xmlns="http://www.w3.org/2000/svg"
   xmlns:xlink="http://www.w3.org/1999/xlink"
   version="1.1"
   id="svg2"
   viewBox="0 0 744.09448819 1052.3622047"
   height="297mm"
   width="210mm">
  <defs
     id="defs4">
    <linearGradient
       id="linearGradient4156">
      <stop
         id="stop4158"
         offset="0"
         style="stop-color:#8888ff;stop-opacity:1;" />
      <stop
         id="stop4160"
         offset="1"
         style="stop-color:#ffff88;stop-opacity:1;" />
    </linearGradient>
    <linearGradient
       gradientUnits="userSpaceOnUse"
       y2="505.21933"
       x2="665.86426"
       y1="505.21933"
       x1="82.70714"
       id="linearGradient4162"
       xlink:href="#linearGradient4156" />
  </defs>
  <metadata
     id="metadata7">
    <rdf:RDF>
      <cc:Work
         rdf:about="">
        <dc:format>image/svg+xml</dc:format>
        <dc:type
           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
        <dc:title></dc:title>
      </cc:Work>
    </rdf:RDF>
  </metadata>
  <g
     id="layer1">
    <rect
       y="478.07648"
       x="82.85714"
       height="54.285713"
       width="582.85712"
       id="rect4154"
       style="color:#000000;display:inline;overflow:visible;visibility:visible;opacity:1;fill:url(#linearGradient4162);fill-opacity:1;fill-rule:nonzero;stroke:#000000;stroke-width:0.30000001;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1;marker:none;enable-background:accumulate" />
  </g>
</svg>
lineargradient.png