index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Kate Lyons</title>
    <link>/index.xml</link>
    <description>Recent content on Kate Lyons</description>
    <generator>Hugo -- gohugo.io</generator>
    <language>en-us</language>
    <copyright>&amp;copy;2017 Kate Lyons</copyright>
    <lastBuildDate>Sun, 12 Mar 2017 21:13:14 -0500</lastBuildDate>
    <atom:link href="/index.xml" rel="self" type="application/rss+xml" />
    
    <item>
      <title>Emoji Dictionary</title>
      <link>/portfolio/2017-10-04-emoji-dictionary/</link>
      <pubDate>Sun, 12 Mar 2017 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-10-04-emoji-dictionary/</guid>
      <description>&lt;p&gt;If you are working with social media data, it is very likely you’ll run into emojis. Because of their encoding, however, they can be tricky to deal with. Fortunately, &lt;a href=&#34;http://opiateforthemass.es/articles/emoticons-in-R/&#34;&gt;Jessica Peterka-Bonetta’s work&lt;/a&gt; introduced the idea of an emoji dictionary which has the prose name of an emoji matched up to its R encoding and unicode codepoint. This list, however, does not include the newest batch of emojis, Unicode Version 9.0, nor the different skin color options for human-based emojis. Good news though – I made &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;my own emoji dictionary&lt;/a&gt; that has all 2,204 of them! I also have included the number of each emoji as listed in the &lt;a href=&#34;http://unicode.org/emoji/charts/emoji-list.html&#34;&gt;Unicode Emoji List v. 5.0&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;If you use this emoji dictionary for your own research, please make sure to acknowledge both myself and Jessica.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This dictionary is available as a CSV file on my &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;github page&lt;/a&gt;. The prose emoji name in the CSV file conveniently has spaces on each side of the emoji name (e.g. &amp;quot; FACEWITHTEARSOFJOY “) so if emojis are right next to other words they won’t be smushed together. Emoji names themselves have no spaces if the name of the emoji is longer than one word. I did this to make text analyses such as sentiment analysis and topic modeling possible without endangering the integrity of the emoji classification. (As we don’t want stop words that are part of emoji names to be deleted!)&lt;/p&gt;
&lt;p&gt;Here is how to use this dictionary for emoji identification in R. There are a few formatting steps and a tricky find-and-replace producedure that requires another R package, but once you have the dictionary loaded and the text in the right format you will be ready to go!&lt;/p&gt;
&lt;div id=&#34;processing-the-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Processing the data&lt;/h2&gt;
&lt;p&gt;Load in the CSV file. You want to make sure it is located in the correct working directory so R can find it when you tell it to read it in.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets=read.csv(&amp;quot;Col_Sep_INSTACORPUS.csv&amp;quot;, header=T)
emoticons &amp;lt;- read.csv(&amp;quot;Decoded Emojis Col Sep.csv&amp;quot;, header = T)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To transform the emojis, you first need to transform your tweet data into ASCII:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets$text &amp;lt;- iconv(tweets$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, 
                    sub = &amp;quot;byte&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;processing-the-data-1&#34; class=&#34;section level2&#34;&gt;
&lt;p&gt;To ‘count’ the emojis you do a find and replace using the CSV file of ‘Decoded Emojis’ as a reference. Here I am using the &lt;a href=&#34;http://www.inside-r.org/packages/cran/DataCombine/docs/FindReplace&#34;&gt;DataCombine package&lt;/a&gt;. What this does is identifies emojis in posts and then replaces them with a prose version. I used whatever description pops up when hovering one’s cursor over an emoji on an Apple emoji keyboard. If not completely the same as other platforms, it provides enough information to find the emoji in question if you are not sure which one was used in the post. You can also cross-check the name listed on the dictionary and the number of the emoji entry in the &lt;a href=&#34;http://unicode.org/emoji/charts/full-emoji-list.html#1f918&#34;&gt;Unicode Emoji List&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(DataCombine)
tweets &amp;lt;- FindReplace(data = tweets, Var = &amp;quot;text&amp;quot;, 
                      replaceData = emoticons,
                      from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                      exact = FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You now have a data frame with emojis in prose form. You can do fun things like &lt;a href=&#34;https://lyons7.github.io/portfolio/2017-03-10-emoji-maps/&#34;&gt;make maps with emojis&lt;/a&gt; (if you have geotag information) or note which are the most frequent emojis and &lt;a href=&#34;https://github.com/dill/emoGG&#34;&gt;plot them&lt;/a&gt; etc. etc. – there are possibilities galore! Have fun 😄&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Collating Spatial Data </title>
      <link>/portfolio/2017-03-14-collate/</link>
      <pubDate>Sun, 12 Mar 2017 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-14-collate/</guid>
      <description>&lt;!-- BLOGDOWN-HEAD --&gt;
&lt;!-- /BLOGDOWN-HEAD --&gt;

&lt;!-- BLOGDOWN-BODY-BEFORE --&gt;
&lt;!-- /BLOGDOWN-BODY-BEFORE --&gt;
&lt;div id=&#34;the-problem&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;The Problem&lt;/h2&gt;
&lt;p&gt;I have one data set of around 1,000 observations of public signs with latitude and longitude coordinates and another data set of around 15,000 tweets that have gone through topic modeling and have topics assigned to them. What I would like to do is link up tweets that have happened close by to the signs I’ve recorded and see the most common topic that is present ‘around’ that sign. This has two issues: 1) the coordinates of public signs and tweets are &lt;em&gt;not&lt;/em&gt; going to be the exact same and 2) there are way more tweets than public signs in the area I’m looking at, so I can’t just merge these two together. I have to figure out a way to look at all the tweets that have occurred near a sign and &lt;em&gt;then&lt;/em&gt; find the topic that has most frequently occured in those tweets associated with that location.&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;the-solution&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;The Solution&lt;/h2&gt;
&lt;p&gt;Luckily, there is a way to do this with packages like &lt;a href=&#34;https://cran.r-project.org/web/packages/fuzzyjoin/fuzzyjoin.pdf&#34;&gt;fuzzyjoin&lt;/a&gt; and &lt;a href=&#34;https://cran.r-project.org/web/packages/dplyr/dplyr.pdf&#34;&gt;dplyr&lt;/a&gt;!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;packs = c(&amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;,&amp;quot;devtools&amp;quot;,&amp;quot;DataCombine&amp;quot;,&amp;quot;ggmap&amp;quot;,
          &amp;quot;topicmodels&amp;quot;,&amp;quot;slam&amp;quot;,&amp;quot;Rmpfr&amp;quot;,&amp;quot;tm&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,
          &amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;,&amp;quot;xlsx&amp;quot;)
lapply(packs, library, character.only=T)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;matching-tweets-with-physical-sign-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Matching tweets with physical sign data&lt;/h2&gt;
&lt;p&gt;What we are trying to do is to match up locations recorded in the physical realm with the digital. Because we do not have &lt;em&gt;exact&lt;/em&gt; matches, we will use the awesome &lt;a href=&#34;https://cran.r-project.org/web/packages/fuzzyjoin/fuzzyjoin.pdf&#34;&gt;fuzzyjoin package&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(fuzzyjoin)
library(dplyr)
pairsdf &amp;lt;- ll %&amp;gt;%
  geo_inner_join(tweets, unit=&amp;#39;km&amp;#39;,distance_col=&amp;quot;distance&amp;quot;) %&amp;gt;%
  filter(distance &amp;lt;= 0.018288)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Joining by: c(&amp;quot;latitude&amp;quot;, &amp;quot;longitude&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# I have to use filter here because &amp;#39;max_distance&amp;#39; is not geared to be less than 1 km or 1 mi
# If you are a weirdo like me looking at things much smaller than a mile or kilometer, you have
# to filter afterwards...&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Voila! I have a data frame with a row of each time a post has occurred in a 60 foot vicinity of an LL object. This might be a little big, but this ensures we get more tweets associated with signs. If you’d like to look at smaller radii, just put in whatever fraction of a kilometer in the ‘distance’ parameter that you are interested in.&lt;/p&gt;
&lt;p&gt;Now what I would like to do is figure out the most common topic that is associated with a particular sign. We’ll use the idea of ‘mode’ here with our topics and the &lt;strong&gt;group_by()&lt;/strong&gt; function from dplyr as suggested &lt;a href=&#34;http://stackoverflow.com/questions/25198442/how-to-calculate-mean-median-per-group-in-a-dataframe-in-r&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;As R does not have a built in function for mode, we build one. Code for this available &lt;a href=&#34;https://www.tutorialspoint.com/r/r_mean_median_mode.htm&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# To get the mode
getmode &amp;lt;- function(v) {
   uniqv &amp;lt;- unique(v)
   uniqv[which.max(tabulate(match(v, uniqv)))]
}

# Tell R your topic categories are a number so it can deal with them
pairsdf$V1&amp;lt;- as.numeric(pairsdf$V1)

# Now calculate things about the topics per sign
topicmode &amp;lt;- pairsdf%&amp;gt;%
group_by(SIGN_ID)%&amp;gt;% 
summarise(Mode = getmode(V1))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s now combine this with our other data, but just include those instances that have a topic assigned (not all signs got a corresponding tweet)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;topicsigns &amp;lt;- inner_join(ll, topicmode, by = &amp;quot;SIGN_ID&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There you have it! You now have a data frame which includes a record of public signs that have tweets that have occurred in their vicinity and the most common topic associated with those tweets.&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Maps! Maps! Maps!</title>
      <link>/portfolio/2017-03-14-maps/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-14-maps/</guid>
      <description>&lt;p&gt;The &lt;a href=&#34;https://cran.r-project.org/web/packages/ggmap/ggmap.pdf&#34;&gt;ggmap&lt;/a&gt; package is awesome. It enables you to get a map from Google maps (in various forms too! Just check out the package documentation) and &lt;em&gt;then&lt;/em&gt; you can plot stuff on top of the maps, which is really useful particularly if you are dealing with spatial data.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Load in the package
library(ggmap)

# Get a map!
# For fun, I&amp;#39;ll do my home town. ggmap does surprisingly well with 
# limited search terms, so you do not have to worry about being super # specific or explicit. 
map &amp;lt;- get_map(location = &amp;#39;Soda Bay,
               California&amp;#39;, zoom = 11)

# The &amp;#39;base&amp;#39; of this is ggmap(map). This will just &amp;#39;print&amp;#39; your map. 
ggmap(map)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-14-maps_files/figure-html/unnamed-chunk-1-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Now you can use &lt;a href=&#34;https://cran.r-project.org/web/packages/ggplot2/ggplot2.pdf&#34;&gt;ggplot2&lt;/a&gt; to plot stuff &lt;em&gt;on top&lt;/em&gt; of the map you’ve just generated. This is useful for all kinds of spatial data, like geotagged social media posts or census data. Here’s an example with census data. (If you want to see how I got this data with R, check out this post &lt;a href=&#34;https://lyons7.github.io/portfolio/2017-03-12-census-data/&#34;&gt;here&lt;/a&gt; or this tutorial &lt;a href=&#34;http://mazamascience.com/WorkingWithData/?p=1494&#34;&gt;here&lt;/a&gt; or &lt;a href=&#34;http://zevross.com/blog/2015/10/14/manipulating-and-mapping-us-census-data-in-r-using-the-acs-tigris-and-leaflet-packages-3/#census-data-the-easyer-way&#34;&gt;here&lt;/a&gt; or &lt;a href=&#34;http://dlab.berkeley.edu/blog/season-sharing-data-working-newly-released-census-2010-2014-acs-5-year-data-r&#34;&gt;here&lt;/a&gt;!)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;ggmap(map) +
  geom_polygon(data = homevalue_points, aes(x = long, y = lat, group = group, fill = Median_Value),   alpha=0.75) +
  scale_fill_distiller(palette = &amp;quot;Blues&amp;quot;) +
  guides(fill = guide_legend(reverse = TRUE)) +
  theme_nothing(legend=TRUE) +
  coord_map() +
  labs(title = &amp;quot;2013 Median Value for Owner-Occupied Housing Units&amp;quot;,
  fill = &amp;quot;Value (Dollars)&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-14-maps_files/figure-html/unnamed-chunk-3-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;In essence, think of the ggmap as a base which you can build on. As long as you have coordinates, you are able to plot things on top of a ggmap, even &lt;a href=&#34;https://lyons7.github.io/portfolio/2017-03-10-emoji-maps/&#34;&gt;series of data&lt;/a&gt;!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Having Fun with Tidy Text!</title>
      <link>/portfolio/2017-03-09-fun-w-tidy-text/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-09-fun-w-tidy-text/</guid>
      <description>&lt;p&gt;Julia Silge and David Robinson have a wonderful new book called “Text Mining with R” which has a &lt;a href=&#34;http://tidytextmining.com/&#34;&gt;companion website&lt;/a&gt; with great explanations and examples. Here are some additional applications of those examples on a corpus of geotagged Instagram posts from the Mission District neighborhood in San Francisco.&lt;/p&gt;


&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Make sure you have the right packages!
packs = c(&amp;quot;twitteR&amp;quot;,&amp;quot;RCurl&amp;quot;,&amp;quot;RJSONIO&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;,&amp;quot;devtools&amp;quot;,&amp;quot;DataCombine&amp;quot;,&amp;quot;ggmap&amp;quot;,
          &amp;quot;topicmodels&amp;quot;,&amp;quot;slam&amp;quot;,&amp;quot;Rmpfr&amp;quot;,&amp;quot;tm&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,
          &amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;,&amp;quot;xlsx&amp;quot;,&amp;quot;scales&amp;quot;,&amp;quot;ggrepel&amp;quot;,&amp;quot;lubridate&amp;quot;,&amp;quot;purrr&amp;quot;,&amp;quot;broom&amp;quot;)
lapply(packs, library, character.only=T)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The data need to be processed a bit more in order to analyze them. Let’s try from the start with &lt;a href=&#34;http://tidytextmining.com/&#34;&gt;Silge and Robinson&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Get rid of stuff particular to the data (here encodings of links and such)
# Most of these are characters I don&amp;#39;t have encodings for (other scripts, etc.)
tweets$text = gsub(&amp;quot;Just posted a photo&amp;quot;,&amp;quot;&amp;quot;, tweets$text)
tweets$text = gsub( &amp;quot;&amp;lt;.*?&amp;gt;&amp;quot;, &amp;quot;&amp;quot;, tweets$text)

# Get rid of super frequent spam posters
tweets &amp;lt;- tweets[! tweets$screenName %in% c(&amp;quot;4AMSOUNDS&amp;quot;,
      &amp;quot;BruciusTattoo&amp;quot;,&amp;quot;LionsHeartSF&amp;quot;,&amp;quot;hermesalchemist&amp;quot;,&amp;quot;Mrsourmash&amp;quot;),]


# Now for Silge and Robinson&amp;#39;s code. What this is doing is getting rid of URLs, re-tweets (RT) and ampersands. This also gets rid of stop words without having to get rid of hashtags and @ signs by using str_detect and filter! 
reg &amp;lt;- &amp;quot;([^A-Za-z_\\d#@&amp;#39;]|&amp;#39;(?![A-Za-z_\\d#@]))&amp;quot;
tidy_tweets &amp;lt;- tweets %&amp;gt;% 
  filter(!str_detect(text, &amp;quot;^RT&amp;quot;)) %&amp;gt;%
  mutate(text = str_replace_all(text, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, text, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% stop_words$word,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Awesome! Now our posts are cleaned with the hashtags and @ mentions still intact. What we can try now is to plot the frequency of some of these terms according to WHERE they occur. Silge and Robinson have an example with persons, let’s try with coordinates.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq &amp;lt;- tidy_tweets %&amp;gt;% 
  group_by(latitude,longitude) %&amp;gt;% 
  count(word, sort = TRUE) %&amp;gt;% 
  left_join(tidy_tweets %&amp;gt;% 
              group_by(latitude,longitude) %&amp;gt;% 
              summarise(total = n())) %&amp;gt;%
  mutate(freq = n/total)

freq&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Source: local data frame [61,008 x 6]
## Groups: latitude, longitude [1,411]
## 
##    latitude longitude          word     n total       freq
##       &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;         &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;      &amp;lt;dbl&amp;gt;
## 1  37.76000 -122.4200       mission  1844 21372 0.08628112
## 2  37.76000 -122.4200           san  1592 21372 0.07448999
## 3  37.76000 -122.4200      district  1576 21372 0.07374134
## 4  37.76000 -122.4200     francisco  1464 21372 0.06850084
## 5  37.75833 -122.4275          park   293  2745 0.10673953
## 6  37.75833 -122.4275       mission   285  2745 0.10382514
## 7  37.75833 -122.4275       dolores   278  2745 0.10127505
## 8  37.76000 -122.4200 #sanfrancisco   275 21372 0.01286730
## 9  37.76300 -122.4209         alley   245  2273 0.10778707
## 10 37.76300 -122.4209       clarion   242  2273 0.10646722
## # ... with 60,998 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The n here is the total number of times this term has shown up, and the total is how many terms there are present in a particular coordinate.&lt;/p&gt;
&lt;p&gt;Cool! Now we have a representation of terms, their frequency and their position. Now I might want to plot this somehow… one way would be to try to plot the most frequent terms (n &amp;gt; 50) (Some help on how to do this was taken from &lt;a href=&#34;http://blog.revolutionanalytics.com/2016/01/avoid-overlapping-labels-in-ggplot2-charts.html&#34;&gt;here&lt;/a&gt; and &lt;a href=&#34;http://stackoverflow.com/questions/14288001/geom-text-not-working-when-ggmap-and-geom-point-used&#34;&gt;here&lt;/a&gt;).&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq2 &amp;lt;- subset(freq, n &amp;gt; 50) 

map &amp;lt;- get_map(location = &amp;#39;Valencia St. and 20th, San Francisco,
               California&amp;#39;, zoom = 15)

freq2$longitude&amp;lt;-as.numeric(freq2$longitude)
freq2$latitude&amp;lt;-as.numeric(freq2$latitude)
lon &amp;lt;- freq2$longitude
lat &amp;lt;- freq2$latitude

mapPoints &amp;lt;- ggmap(map) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freq2, aes(x = lon, y = lat, label = word),size = 2) &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-09-fun-w-tidy-text_files/figure-html/unnamed-chunk-6-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let’s zoom into that main central area to see what’s going on!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;map2 &amp;lt;- get_map(location = &amp;#39;Lexington St. and 19th, San Francisco,
               California&amp;#39;, zoom = 16)
mapPoints2 &amp;lt;- ggmap(map2) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freq2, aes(x = lon, y = lat, label = word),size = 2) &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-09-fun-w-tidy-text_files/figure-html/unnamed-chunk-8-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;This can be manipulated in many different ways – either by playing with what frequency of terms you want to look at (maybe I want to see terms that occur 100 times, between 20 and 50 times, less than 20 times etc. etc.) OR by playing around with the map. At the moment though, this is pretty illuminating in the sense that it shows us that the most frequency terms are focused around certain ‘hotspots’ in the area, which in itself is just kind of cool to see.&lt;/p&gt;
&lt;p&gt;Now let’s try out word frequency changes over time: what words were used more or less over the time of data collection? (Help from &lt;a href=&#34;http://tidytextmining.com/twitter.html&#34;&gt;here&lt;/a&gt;) (Also used the &lt;a href=&#34;https://cran.r-project.org/web/packages/lubridate/lubridate.pdf&#34;&gt;lubridate package&lt;/a&gt; to help with time.)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Might have to do this first
tidy_tweets$created2 &amp;lt;- as.POSIXct(tidy_tweets$created, format=&amp;quot;%m/%d/%Y %H:%M&amp;quot;)

words_by_time &amp;lt;- tidy_tweets %&amp;gt;%
  mutate(time_floor = floor_date(created2, unit = &amp;quot;1 week&amp;quot;)) %&amp;gt;%
  count(time_floor, word) %&amp;gt;%
  ungroup() %&amp;gt;%
  group_by(time_floor) %&amp;gt;%
  mutate(time_total = sum(n)) %&amp;gt;%
  group_by(word) %&amp;gt;%
  mutate(word_total = sum(n)) %&amp;gt;%
  ungroup() %&amp;gt;%
  rename(count = n) %&amp;gt;%
  filter(word_total &amp;gt; 100)

words_by_time&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 1,979 × 5
##    time_floor             word count time_total word_total
##        &amp;lt;dttm&amp;gt;            &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;      &amp;lt;int&amp;gt;      &amp;lt;int&amp;gt;
## 1  0016-07-31             #art     7       2729        120
## 2  0016-07-31      #california     9       2729        149
## 3  0016-07-31     #dolorespark     6       2729        168
## 4  0016-07-31         #mission     5       2729        222
## 5  0016-07-31 #missiondistrict     1       2729        158
## 6  0016-07-31    #sanfrancisco    38       2729       1034
## 7  0016-07-31              #sf    23       2729        603
## 8  0016-07-31       #streetart    12       2729        229
## 9  0016-07-31             24th    10       2729        109
## 10 0016-07-31            alamo     3       2729        111
## # ... with 1,969 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Alright, now we want to figure out those words that have changed the most in their frequency over time so as to isolate ones of interests to plot over time. This involves a few steps though.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;nested_data &amp;lt;- words_by_time %&amp;gt;%
  nest(-word) 
nested_data&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 71 × 2
##                word              data
##               &amp;lt;chr&amp;gt;            &amp;lt;list&amp;gt;
## 1              #art &amp;lt;tibble [26 × 4]&amp;gt;
## 2       #california &amp;lt;tibble [28 × 4]&amp;gt;
## 3      #dolorespark &amp;lt;tibble [27 × 4]&amp;gt;
## 4          #mission &amp;lt;tibble [28 × 4]&amp;gt;
## 5  #missiondistrict &amp;lt;tibble [29 × 4]&amp;gt;
## 6     #sanfrancisco &amp;lt;tibble [29 × 4]&amp;gt;
## 7               #sf &amp;lt;tibble [29 × 4]&amp;gt;
## 8        #streetart &amp;lt;tibble [28 × 4]&amp;gt;
## 9              24th &amp;lt;tibble [28 × 4]&amp;gt;
## 10            alamo &amp;lt;tibble [27 × 4]&amp;gt;
## # ... with 61 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Process as described by Silge and Robinson: “This data frame has one row for each person-word combination; the data column is a list column that contains data frames, one for each combination of person and word. Let’s use map() from the purrr library to apply our modeling procedure to each of those little data frames inside our big data frame. This is count data so let’s use glm() with family =”binomial&amp;quot; for modeling. We can think about this modeling procedure answering a question like, “Was a given word mentioned in a given time bin? Yes or no? How does the count of word mentions depend on time?”&amp;quot;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;nested_models &amp;lt;- nested_data %&amp;gt;%
  mutate(models = map(data, ~ glm(cbind(count, time_total) ~ time_floor, ., 
                                  family = &amp;quot;binomial&amp;quot;)))

nested_models&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 71 × 3
##                word              data    models
##               &amp;lt;chr&amp;gt;            &amp;lt;list&amp;gt;    &amp;lt;list&amp;gt;
## 1              #art &amp;lt;tibble [26 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 2       #california &amp;lt;tibble [28 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 3      #dolorespark &amp;lt;tibble [27 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 4          #mission &amp;lt;tibble [28 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 5  #missiondistrict &amp;lt;tibble [29 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 6     #sanfrancisco &amp;lt;tibble [29 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 7               #sf &amp;lt;tibble [29 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 8        #streetart &amp;lt;tibble [28 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 9              24th &amp;lt;tibble [28 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## 10            alamo &amp;lt;tibble [27 × 4]&amp;gt; &amp;lt;S3: glm&amp;gt;
## # ... with 61 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Silge and Robinson: “Now notice that we have a new column for the modeling results; it is another list column and contains glm objects. The next step is to use map() and tidy() from the broom package to pull out the slopes for each of these models and find the important ones. We are comparing many slopes here and some of them are not statistically significant, so let’s apply an adjustment to the p-values for multiple comparisons.”&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;slopes &amp;lt;- nested_models %&amp;gt;%
  unnest(map(models, tidy)) %&amp;gt;%
  filter(term == &amp;quot;time_floor&amp;quot;) %&amp;gt;%
  mutate(adjusted.p.value = p.adjust(p.value))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;“Now let’s find the most important slopes. Which words have changed in frequency at a moderately significant level in our tweets?”&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;top_slopes &amp;lt;- slopes %&amp;gt;% 
  filter(adjusted.p.value &amp;lt; 0.1)

top_slopes&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 16 × 7
##             word       term      estimate    std.error  statistic
##            &amp;lt;chr&amp;gt;      &amp;lt;chr&amp;gt;         &amp;lt;dbl&amp;gt;        &amp;lt;dbl&amp;gt;      &amp;lt;dbl&amp;gt;
## 1   #dolorespark time_floor -1.239208e-07 1.877375e-08  -6.600749
## 2  #sanfrancisco time_floor -3.211771e-08 6.679608e-09  -4.808322
## 3            #sf time_floor -3.483812e-08 8.745735e-09  -3.983441
## 4          alley time_floor -7.976653e-08 1.277341e-08  -6.244732
## 5        clarion time_floor -9.680816e-08 1.449039e-08  -6.680855
## 6       district time_floor  6.701575e-08 5.346673e-09  12.534102
## 7        dolores time_floor -1.132466e-07 7.718189e-09 -14.672698
## 8      francisco time_floor  4.515439e-08 4.642235e-09   9.726865
## 9    manufactory time_floor -7.442008e-08 1.672651e-08  -4.449228
## 10       mission time_floor  3.625027e-08 3.999962e-09   9.062656
## 11          park time_floor -1.134007e-07 7.699185e-09 -14.728916
## 12           san time_floor  4.009249e-08 4.369834e-09   9.174832
## 13            sf time_floor -9.883069e-08 7.359724e-09 -13.428586
## 14        street time_floor -5.396273e-08 1.121572e-08  -4.811349
## 15       tartine time_floor -6.296002e-08 1.209275e-08  -5.206427
## 16      valencia time_floor -8.125194e-08 2.093658e-08  -3.880859
## # ... with 2 more variables: p.value &amp;lt;dbl&amp;gt;, adjusted.p.value &amp;lt;dbl&amp;gt;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s plot them!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;words_by_time %&amp;gt;%
  inner_join(top_slopes, by = c(&amp;quot;word&amp;quot;)) %&amp;gt;%
  ggplot(aes(time_floor, count/time_total, color = word)) +
  geom_line(size = 1.3) +
  labs(x = NULL, y = &amp;quot;Word frequency&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-09-fun-w-tidy-text_files/figure-html/unnamed-chunk-14-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;After looking at some of these features of our data set, it’s time to explore TOPIC MODELING, or (paraphrasing from David Blei) finding structure in more-or-less unstructured documents. To do this we need a document-term matrix. At the moment, the tweets are a little problematic in that they are broken up by words… whereas we actually would like the text of the tweet back as that is what we are treating as our ‘document’. The question at the moment is… do we want to keep the hashtags / can we?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s try by taking our tweets that have been tidied already. First we need to count each word though, and create some kind of column that has 
# This first one is helpful for seeing encodings that need to be removed
tidy_tweets %&amp;gt;%
  count(document, word, sort=TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Source: local data frame [102,888 x 3]
## Groups: document [14,958]
## 
##    document                      word     n
##       &amp;lt;int&amp;gt;                     &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1      5932        facewithtearsofjoy    17
## 2      8849                   balloon     8
## 3     12204 blackquestionmarkornament     8
## 4      7697                       nov     7
## 5      7697                       wed     7
## 6     12204 whitequestionmarkornament     7
## 7      2110              sliceofpizza     6
## 8      2452                nailpolish     6
## 9      2741     facewithnogoodgesture     6
## 10     4014                 earofrice     6
## # ... with 102,878 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Counting words so we can make a dtm with our preserved corpus with hashtags and such
tweet_words &amp;lt;- tidy_tweets %&amp;gt;%  
  count(document, word) %&amp;gt;%
  ungroup()

total_words &amp;lt;- tweet_words %&amp;gt;% 
  group_by(document) %&amp;gt;% 
  summarize(total = sum(n))

post_words &amp;lt;- left_join(tweet_words, total_words)

post_words&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 102,888 × 4
##    document           word     n total
##       &amp;lt;int&amp;gt;          &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;
## 1         1      #nofilter     1     7
## 2         1  #sanfrancisco     1     7
## 3         1      afternoon     1     7
## 4         1        dolores     1     7
## 5         1           park     2     7
## 6         1             sf     1     7
## 7         2 @publicworkssf     1     5
## 8         2    #dustyrhino     1     5
## 9         2          close     1     5
## 10        2           grin     1     5
## # ... with 102,878 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;new_dtm &amp;lt;- post_words %&amp;gt;% 
  cast_dtm(document, word, n)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This seems to have worked :O :O :O Let’s see how topic modeling works here now…&lt;/p&gt;
&lt;p&gt;Visualization in TIDY form also from &lt;a href=&#34;http://tidytextmining.com/topicmodeling.html&#34;&gt;Silge and Robinson&lt;/a&gt;!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;#Set parameters for Gibbs sampling (parameters those used in
#Grun and Hornik 2011)
burnin &amp;lt;- 4000
iter &amp;lt;- 2000
thin &amp;lt;- 500
seed &amp;lt;-list(2003,5,63,100001,765)
nstart &amp;lt;- 5
best &amp;lt;- TRUE
k &amp;lt;- 12
test_lda2 &amp;lt;-LDA(new_dtm,k, method=&amp;quot;Gibbs&amp;quot;, 
             control=list(nstart=nstart, seed = seed, best=best, 
                          burnin = burnin, iter = iter, thin=thin))

# Make that TIDY!!! 
test_lda_td2 &amp;lt;- tidy(test_lda2)

lda_top_terms2 &amp;lt;- test_lda_td2 %&amp;gt;%
  group_by(topic) %&amp;gt;%
  top_n(10, beta) %&amp;gt;%
  ungroup() %&amp;gt;%
  arrange(topic, -beta)

lda_top_terms2 %&amp;gt;%
  mutate(term = reorder(term, beta)) %&amp;gt;%
  ggplot(aes(term, beta, fill = factor(topic))) +
  geom_bar(stat = &amp;quot;identity&amp;quot;, show.legend = FALSE) +
  facet_wrap(~ topic, scales = &amp;quot;free&amp;quot;) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-09-fun-w-tidy-text_files/figure-html/unnamed-chunk-16-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;

</description>
    </item>
    
    <item>
      <title>Building A Website</title>
      <link>/portfolio/2017-03-12-build-website/</link>
      <pubDate>Sun, 12 Mar 2017 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-12-build-website/</guid>
      <description>&lt;!-- BLOGDOWN-HEAD --&gt;
&lt;!-- /BLOGDOWN-HEAD --&gt;

&lt;!-- BLOGDOWN-BODY-BEFORE --&gt;
&lt;!-- /BLOGDOWN-BODY-BEFORE --&gt;
&lt;p&gt;This website was constructed using GitHub, the &lt;a href=&#34;https://github.com/rstudio/blogdown&#34;&gt;blogdown package&lt;/a&gt; for R and Hugo. I got started with &lt;a href=&#34;https://proquestionasker.github.io/blog/Making_Site/&#34;&gt;Amber Thomas’s amazing tutorial&lt;/a&gt; and got some further help from &lt;a href=&#34;http://robertmyles.github.io/2017/02/01/how-to-make-a-github-pages-blog-with-rstudio-and-hugo/&#34;&gt;Robert Myles McDonnell&lt;/a&gt; and more help from &lt;a href=&#34;http://whipperstacker.com/2015/11/27/deploying-a-stand-alone-hugo-site-to-github-pages-mapped-to-a-custom-domain/&#34;&gt;here&lt;/a&gt;. If you are interested in building a site with this configuration, I would start either with Amber’s or Robert’s tutorials as they are very detailed and will give an understanding of how this whole thing works. If, however, the steps detailed in either of these things don’t work for you, try out the process that eventually worked for me.&lt;/p&gt;
&lt;div id=&#34;kates-approach&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Kate’s Approach&lt;/h4&gt;
&lt;p&gt;Let’s start from the very beginning. I was running into a lot of issues (see comments on Robert’s blog and on Amber’s tutorial) but this specific series of steps managed to work for me.&lt;/p&gt;
&lt;p&gt;First, start fresh. Create a USERNAME repo and a special HUGO repo on the ONLINE GitHub.&lt;/p&gt;
&lt;p&gt;Go back to your computer. (I’m working with a Mac).&lt;/p&gt;
&lt;p&gt;Create a new project (I think you can probably do this within a folder, but I went the project route) in R studio. Create a new directory in your own computer and name it whatever you want. Then in R use blogdown to create a site and if you want install a theme etc. Then, &lt;strong&gt;serve_site()&lt;/strong&gt; and &lt;strong&gt;STOP&lt;/strong&gt; R with the little red stop sign located in the upper right corner of the Console window. (I have to do this anyway because whenever I &lt;strong&gt;serve_site()&lt;/strong&gt; or &lt;strong&gt;new_site()&lt;/strong&gt; in R Studio the “&amp;gt;” doesn’t show up in my console afterwards). Update: I just figured out that this is because the site is being continuously “constructed” once you serve it – i.e. you can make some changes and it’ll automatically update in your little Viewer window. Probably doesn’t matter too much, but I’d just stop it just in case.&lt;/p&gt;
&lt;p&gt;Now, in that directory you created with the project will be ALL the website files. Don’t move anything around. I then followed the steps from &lt;a href=&#34;http://whipperstacker.com/2015/11/27/deploying-a-stand-alone-hugo-site-to-github-pages-mapped-to-a-custom-domain/&#34;&gt;this link&lt;/a&gt; mentioned above and did the following in terminal:&lt;/p&gt;
&lt;p&gt;Change directories to the directory where the website files are (let’s call this ‘Local Hugo’). Now remove the public folder (I am &lt;em&gt;pretty&lt;/em&gt; sure that is what this command does)&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;rm -r public/&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;And NOW you are going to link up to your online GitHub.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ git init

$ git remote add origin git@github.com:USERNAME/HUGO.git

$ git submodule add git@github.com:USERNAME/USERNAME.github.io.git public
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;(I get asked for a password here I set up for my .ssh/id_rsa here)&lt;/p&gt;
&lt;p&gt;Then:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ git commit -m &amp;#39;initial commit&amp;#39;

$ git push origin master&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;(Get asked again for password)&lt;/p&gt;
&lt;p&gt;Now I go back to R studio where my project is up and running. I did the customary new_post(‘Hello world’) post to test things out and then made some changes to the config file (put my name etc). Then I &lt;strong&gt;serve_site()&lt;/strong&gt; in R, &lt;strong&gt;STOP&lt;/strong&gt; it again and return to terminal. I then followed the commands listed on Robert’s tutorial.&lt;/p&gt;
&lt;p&gt;Change directories in terminal to your computer Local Hugo folder and then change directories again so you are in the ‘public’ folder in that directory. Then:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;$ git add -A

$ git commit -m &amp;#39;lovely new site&amp;#39;

$ git push origin master&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;(Again I get asked for my password)&lt;/p&gt;
&lt;p&gt;Then go to your username.github.io site. It &lt;em&gt;should&lt;/em&gt; be there! Now when you make changes it’s the same process: do your thing in R studio, &lt;strong&gt;serve_site()&lt;/strong&gt; and then &lt;strong&gt;STOP&lt;/strong&gt; and then go to terminal and run the same commands (navigate to “public” local Hugo folder, add -A, commit with a message and then push).&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Adventures in Text Mining</title>
      <link>/portfolio/2017-03-15-text-mining/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-15-text-mining/</guid>
      <description>&lt;p&gt;There are many wonderful tutorials on how to &lt;a href=&#34;https://blogs.sap.com/2014/03/16/setting-up-twitter-api-to-work-with-r/&#34;&gt;work with Twitter REST APIs&lt;/a&gt; (even a video walk-through &lt;a href=&#34;https://www.youtube.com/watch?v=lT4Kosc_ers&#34;&gt;here&lt;/a&gt;) so I won’t describe that process. Instead, I will show some examples of using the &lt;a href=&#34;https://cran.r-project.org/web/packages/twitteR/twitteR.pdf&#34;&gt;twitteR&lt;/a&gt; and related packages to look at geotagged posts occurring within a specific neighborhood (i.e. how to use the &lt;strong&gt;searchTwitter()&lt;/strong&gt; function in twitteR to search by &lt;em&gt;location&lt;/em&gt;, not specific search term). I will also be using methods described in Julia Silge and David Robinson’s &lt;a href=&#34;http://tidytextmining.com/&#34;&gt;new book&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;packs = c(&amp;quot;twitteR&amp;quot;,&amp;quot;RCurl&amp;quot;,&amp;quot;RJSONIO&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;,&amp;quot;devtools&amp;quot;,&amp;quot;DataCombine&amp;quot;,&amp;quot;ggmap&amp;quot;,
          &amp;quot;topicmodels&amp;quot;,&amp;quot;slam&amp;quot;,&amp;quot;Rmpfr&amp;quot;,&amp;quot;tm&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,
          &amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;,&amp;quot;xlsx&amp;quot;)
lapply(packs, library, character.only=T)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# key = &amp;quot;YOUR KEY HERE&amp;quot;
# secret = &amp;quot;YOUR SECRET HERE&amp;quot;

# tok = &amp;quot;YOUR TOK HERE&amp;quot;
# tok_sec = &amp;quot;YOUR TOK_SEC HERE&amp;quot;

twitter_oauth &amp;lt;- setup_twitter_oauth(key, secret, tok, tok_sec)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;I’m interested in the Mission District neighborhood in San Francisco, California. I obtain a set of coordinates using Google maps and plug that into the ‘geocode’ parameter and then set a radius of 1 kilometer. I know from experience that I only get around 1,000 - 2,000 posts per time I do this, so I set the number of tweets (n) I would like to get from Twitter at ‘7,000’. If you are looking at a more ‘active’ area, or a larger area (more about this later) you can always adjust this number. The API will give you a combination of the most &lt;a href=&#34;https://dev.twitter.com/rest/public/search&#34;&gt;“recent or popular” tweets&lt;/a&gt; that usually extend back about 5 days or so. If you are looking at a smaller area, this means to get any kind of decent tweet corpus you’ll have to spent some time collecting data week after week. Also if you want to look at a larger area than a 3-4 kilometer radius, a lot of times you’ll get a bunch of spam-like posts that don’t have latitude and longitude coordinates associated with them. A work around I thought of (for another project looking at posts in an entire city) is to figure out a series of spots to collect tweets (trying to avoid overlap as much as possible) and stiching those data frames all together and getting rid of any duplicate posts you picked up if your radii overlapped.&lt;/p&gt;
&lt;p&gt;Luckily for the Mission District, however, we are interested in a smaller area and don’t have to worry about multiple sampling points and rbind’ing data frames together, and just run the searchTwitter function once:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geo &amp;lt;- searchTwitter(&amp;#39;&amp;#39;,n=7000, geocode=&amp;#39;37.76,-122.42,1km&amp;#39;,
                     retryOnRateLimit=1)&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;processing-the-data&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Processing the data&lt;/h4&gt;
&lt;p&gt;Now you have a list of tweets. Lists are very difficult to deal with in R, so you convert this into a data frame:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geoDF&amp;lt;-twListToDF(geo)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Chances are there will be emojis in your Twitter data. You can ‘transform’ these emojis into prose using this code as well as a &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;CSV file&lt;/a&gt; I’ve put together of what all of the emojis look like in R. (The idea for this comes from &lt;a href=&#34;http://opiateforthemass.es/articles/emoticons-in-R/&#34;&gt;Jessica Peterka-Bonetta’s work&lt;/a&gt; – she has a list of emojis as well, but it does not include the newest batch of emojis, Unicode Version 9.0, nor the different skin color options for human-based emojis). If you use this emoji list for your own research, please make sure to acknowledge both myself and Jessica.&lt;/p&gt;
&lt;p&gt;Load in the CSV file. You want to make sure it is located in the correct working directory so R can find it when you tell it to read it in.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;emoticons &amp;lt;- read.csv(&amp;quot;Decoded Emojis Col Sep.csv&amp;quot;, header = T)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To transform the emojis, you first need to transform the tweet data into ASCII:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geoDF$text &amp;lt;- iconv(geoDF$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, 
                    sub = &amp;quot;byte&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To ‘count’ the emojis you do a find and replace using the CSV file of ‘Decoded Emojis’ as a reference. Here I am using the &lt;a href=&#34;http://www.inside-r.org/packages/cran/DataCombine/docs/FindReplace&#34;&gt;DataCombine package&lt;/a&gt;. What this does is identifies emojis in the tweets and then replaces them with a prose version. I used whatever description pops up when hovering one’s cursor over an emoji on an Apple emoji keyboard. If not completely the same as other platforms, it provides enough information to find the emoji in question if you are not sure which one was used in the post.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;data &amp;lt;- FindReplace(data = geoDF, Var = &amp;quot;text&amp;quot;, 
                            replaceData = emoticons,
                       from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                       exact = FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now might be a good time to save this file, perhaps in CSV format with the date of when the data was collected:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# write.csv(data,file=paste(&amp;quot;ALL&amp;quot;,Sys.Date(),&amp;quot;.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;visualizing-the-data&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Visualizing the data&lt;/h4&gt;
&lt;p&gt;Now let’s play around with visualizing the data. I want to superimpose different aspects of the tweets I collected on a map. First I have to get a map, which I do using the &lt;a href=&#34;https://cran.r-project.org/web/packages/ggmap/ggmap.pdf&#34;&gt;ggmap package&lt;/a&gt; which interacts with Google Map’s API. When you use this package, be sure to cite it, as it requests you to when you first load the package into your library. (Well, really you should cite every R package you use, right?)&lt;/p&gt;
&lt;p&gt;I request a map of the Mission District, and then check to make sure the map is what I want (in terms of zoom, area covered, etc.)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;map &amp;lt;- get_map(location = &amp;#39;Capp St. and 20th, San Francisco,
               California&amp;#39;, zoom = 15)
# To check out the map
ggmap(map)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-10-1.png&#34; width=&#34;672&#34; /&gt; Looks good to me! Now let’s start to visualize our Twitter data. We can start by seeing where our posts are on a map.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Tell R what we want to map
# Need to specify that lat/lon should be treated like numbers
data$longitude&amp;lt;-as.numeric(data$longitude)
data$latitude&amp;lt;-as.numeric(data$latitude)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For now I just want to look at latitude and longitude, but it is possible to map other aspects as well - it just depends on what you’d like to look at.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;Mission_tweets &amp;lt;- ggmap(map) + geom_point(aes(x=longitude, y=latitude), 
                               data=data, alpha=0.5)

Mission_tweets&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-12-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We can also look at WHEN the posts were generated. We can make a graph of post frequency over time.Graphs constructed with help from &lt;a href=&#34;http://www.cyclismo.org/tutorial/R/time.html&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;https://gist.github.com/stephenturner/3132596&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://stackoverflow.com/questions/27626915/r-graph-frequency-of-observations-over-time-with-small-value-range&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://michaelbommarito.com/2011/03/12/a-quick-look-at-march11-saudi-tweets/&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://stackoverflow.com/questions/31796744/plot-count-frequency-of-tweets-for-word-by-month&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;https://stat.ethz.ch/R-manual/R-devel/library/base/html/as.POSIXlt.html&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://sape.inf.usi.ch/quick-reference/ggplot2/geom&#34;&gt;here&lt;/a&gt; and &lt;a href=&#34;http://stackoverflow.com/questions/3541713/how-to-plot-two-histograms-together-in-r&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Create a data frame with number of tweets per time
d &amp;lt;- as.data.frame(table(data$created))
d &amp;lt;- d[order(d$Freq, decreasing=T), ]
names(d) &amp;lt;- c(&amp;quot;created&amp;quot;,&amp;quot;freq&amp;quot;)
# Combine this with existing data frame
newdata1 &amp;lt;- merge(data,d,by=&amp;quot;created&amp;quot;)
# Tell R that &amp;#39;created&amp;#39; is not an integer or factor but a time.
data$created &amp;lt;- as.POSIXct(data$created)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now plot number of tweets over period of time across 20 minute intervals:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;minutes &amp;lt;- 60
Freq&amp;lt;-data$freq
plot1&amp;lt;-ggplot(data, aes(created)) + geom_freqpoly(binwidth=60*minutes)
plot1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-14-1.png&#34; width=&#34;672&#34; /&gt; This might be more informative if you want to look at specific time periods. We can look at the frequency of posts over the course of a specific day if we want.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;data2 &amp;lt;- data[data$created &amp;lt;= &amp;quot;2017-03-11 00:31:00&amp;quot;, ]
minutes &amp;lt;- 60
Freq&amp;lt;-data2$freq
plot2&amp;lt;-ggplot(data2, aes(created)) + geom_freqpoly(binwidth=60*minutes)
plot2&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-15-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let’s look at other ways to visualize Twitter data. I will be using a larger corpus of posts I’ve been building up for about a year (as mentioned above, I only get about 1,000 posts per searchTwitter per week so it took some time to get a good corpus going).&lt;/p&gt;
&lt;p&gt;Some more processessing needs to be completed before looking at things like most frequent terms or what kind of sentiments seem to be expressed in our corpus. All of the following steps to ‘clean’ the data of URLs, odd characters and ‘stop words’ (a.k.a. words like ‘the’ or ‘and’ that aren’t very informative re. what the post is actually discussing) are taken from &lt;a href=&#34;http://tidytextmining.com/sentiment.html#the-sentiments-dataset&#34;&gt;Silge and Robinson&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets=read.csv(&amp;quot;Col_Sep_INSTACORPUS.csv&amp;quot;, header=T)
# Get rid of stuff particular to the data (here encodings of links
# and such)
# Most of these are characters I don&amp;#39;t have encodings for (other scripts, etc.)
tweets$text = gsub(&amp;quot;Just posted a photo&amp;quot;,&amp;quot;&amp;quot;, tweets$text)
tweets$text = gsub( &amp;quot;&amp;lt;.*?&amp;gt;&amp;quot;, &amp;quot;&amp;quot;, tweets$text)

# Get rid of super frequent spam-y posters
tweets &amp;lt;- tweets[! tweets$screenName %in% c(&amp;quot;4AMSOUNDS&amp;quot;,
      &amp;quot;BruciusTattoo&amp;quot;,&amp;quot;LionsHeartSF&amp;quot;,&amp;quot;hermesalchemist&amp;quot;,&amp;quot;Mrsourmash&amp;quot;),]

# Now for Silge and Robinson&amp;#39;s code. What this is doing is getting rid of 
# URLs, re-tweets (RT) and ampersands. This also gets rid of stop words 
# without having to get rid of hashtags and @ signs by using 
# str_detect and filter! 
reg &amp;lt;- &amp;quot;([^A-Za-z_\\d#@&amp;#39;]|&amp;#39;(?![A-Za-z_\\d#@]))&amp;quot;
tidy_tweets &amp;lt;- tweets %&amp;gt;% 
  filter(!str_detect(text, &amp;quot;^RT&amp;quot;)) %&amp;gt;%
  mutate(text = str_replace_all(text, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, text, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% stop_words$word,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))

# Get rid of stop words by doing an &amp;#39;anti-join&amp;#39; (amazing run-down of what 
# all the joins do is available here:
# http://stat545.com/bit001_dplyr-cheatsheet.html#anti_joinsuperheroes-publishers)
data(stop_words)

tidy_tweets &amp;lt;- tidy_tweets %&amp;gt;%
  anti_join(stop_words)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we can look at things like most frequent words and sentiments expressed in our corpus.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Find most common words in corpus
tidy_tweets %&amp;gt;%
  count(word, sort = TRUE) &lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 27,441 × 2
##             word     n
##            &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1        mission  3327
## 2            san  2751
## 3      francisco  2447
## 4       district  1871
## 5  #sanfrancisco  1175
## 6           park  1060
## 7        dolores  1045
## 8             sf  1031
## 9            #sf   696
## 10           day   597
## # ... with 27,431 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Plot most common words:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tidy_tweets %&amp;gt;%
  count(word, sort = TRUE) %&amp;gt;%
  filter(n &amp;gt; 150) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n)) +
  geom_bar(stat = &amp;quot;identity&amp;quot;) +
  xlab(NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-18-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;What about different sentiments?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# What are the most common &amp;#39;joy&amp;#39; words?
nrcjoy &amp;lt;- sentiments %&amp;gt;% 
  filter(lexicon == &amp;quot;nrc&amp;quot;) %&amp;gt;%
  filter(sentiment == &amp;quot;joy&amp;quot;)

tidy_tweets %&amp;gt;%
  semi_join(nrcjoy) %&amp;gt;%
  count(word, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 355 × 2
##         word     n
##        &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1       love   466
## 2      happy   402
## 3        art   266
## 4   birthday   205
## 5   favorite   194
## 6  beautiful   179
## 7        fun   173
## 8       food   139
## 9      music   126
## 10 chocolate   105
## # ... with 345 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# What are the most common &amp;#39;disgust&amp;#39; words?
nrcdisgust &amp;lt;- sentiments %&amp;gt;% 
  filter(lexicon == &amp;quot;nrc&amp;quot;) %&amp;gt;%
  filter(sentiment == &amp;quot;disgust&amp;quot;)

tidy_tweets %&amp;gt;%
  semi_join(nrcdisgust) %&amp;gt;%
  count(word, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 217 × 2
##       word     n
##      &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1  finally   100
## 2  feeling    55
## 3     gray    50
## 4     tree    50
## 5  hanging    44
## 6      bad    43
## 7      boy    40
## 8     shit    36
## 9    lemon    29
## 10   treat    28
## # ... with 207 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We can also look at counts of negative and positive words
bingsenti &amp;lt;- sentiments %&amp;gt;%
  filter(lexicon ==&amp;quot;bing&amp;quot;)

bing_word_counts &amp;lt;- tidy_tweets %&amp;gt;%
  inner_join(bingsenti) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

# And graph them!
bing_word_counts %&amp;gt;%
  filter(n &amp;gt; 25) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-19-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We can also make pretty word clouds! Code for this taken from &lt;a href=&#34;https://rstudio-pubs-static.s3.amazonaws.com/132792_864e3813b0ec47cb95c7e1e2e2ad83e7.html&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# First have to make a document term matrix, which involves a few steps
tidy_tweets %&amp;gt;%
  count(document, word, sort=TRUE)

tweet_words &amp;lt;- tidy_tweets %&amp;gt;%  
  count(document, word) %&amp;gt;%
  ungroup()

total_words &amp;lt;- tweet_words %&amp;gt;% 
  group_by(document) %&amp;gt;% 
  summarize(total = sum(n))

post_words &amp;lt;- left_join(tweet_words, total_words)

dtm &amp;lt;- post_words %&amp;gt;% 
  cast_dtm(document, word, n)

# Need freq count for word cloud to work
freq = data.frame(sort(colSums(as.matrix(dtm)), decreasing=TRUE))&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;wordcloud(rownames(freq), freq[,1], max.words=70, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-15-text-mining_files/figure-html/unnamed-chunk-21-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;If you are interested in delving even deeper, you can try techniques like topic modeling, a process I describe and demonstrate &lt;a href=&#34;https://lyons7.github.io/portfolio/2017-03-09-fun-w-tidy-text/&#34;&gt;here&lt;/a&gt;!&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Identifying and Visualizing Emojis</title>
      <link>/portfolio/2017-03-10-emoji-maps/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-10-emoji-maps/</guid>
      <description>&lt;p&gt;Dealing with emojis in mined social media data can be tricky for a number of reasons. First, you have to decode them and then… well I guess that is it. After you decode them there is a number of cool things you can look at though!&lt;/p&gt;
&lt;div id=&#34;processing-the-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Processing the data&lt;/h2&gt;
&lt;p&gt;As mentioned, if you are working with social media data, chances are there will be emojis in that data. You can ‘transform’ these emojis into prose using this code as well as a &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;CSV file&lt;/a&gt; I’ve put together of what all of the emojis look like in R. (The idea for this comes from &lt;a href=&#34;http://opiateforthemass.es/articles/emoticons-in-R/&#34;&gt;Jessica Peterka-Bonetta’s work&lt;/a&gt; – she has a list of emojis as well, but it does not include the newest batch of emojis, Unicode Version 9.0, nor the different skin color options for human-based emojis). If you use this emoji list for your own research, please make sure to acknowledge both myself and Jessica.&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;processing-the-data-1&#34; class=&#34;section level2&#34;&gt;

&lt;p&gt;Load in the CSV file. You want to make sure it is located in the correct working directory so R can find it when you tell it to read it in.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets=read.csv(&amp;quot;Col_Sep_INSTACORPUS.csv&amp;quot;, header=T)
emoticons &amp;lt;- read.csv(&amp;quot;Decoded Emojis Col Sep.csv&amp;quot;, header = T)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To transform the emojis, you first need to transform your tweet data into ASCII:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets$text &amp;lt;- iconv(tweets$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, 
                    sub = &amp;quot;byte&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;processing-the-data-2&#34; class=&#34;section level2&#34;&gt;
&lt;p&gt;To ‘count’ the emojis you do a find and replace using the CSV file of ‘Decoded Emojis’ as a reference. Here I am using the &lt;a href=&#34;http://www.inside-r.org/packages/cran/DataCombine/docs/FindReplace&#34;&gt;DataCombine package&lt;/a&gt;. What this does is identifies emojis in the tweeted Instagram posts and then replaces them with a prose version. I used whatever description pops up when hovering one’s cursor over an emoji on an Apple emoji keyboard. If not completely the same as other platforms, it provides enough information to find the emoji in question if you are not sure which one was used in the post.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(DataCombine)
tweets &amp;lt;- FindReplace(data = tweets, Var = &amp;quot;text&amp;quot;, 
                      replaceData = emoticons,
                      from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                      exact = FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now I’m going to subset the data to just look at those posts that have emojis in them. I got help in doing this from &lt;a href=&#34;http://stackoverflow.com/questions/26319567/use-grepl-to-search-either-of-multiple-substrings-in-a-text-in-r&#34;&gt;here&lt;/a&gt;. Again I use my emoji dictionary available &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;emoticons &amp;lt;- read.csv(&amp;quot;Decoded Emojis Col Sep.csv&amp;quot;, header = T)
emogrepl &amp;lt;- grepl(paste(emoticons$Name, collapse = &amp;quot;|&amp;quot;), tweets$text)
emogreplDF&amp;lt;-as.data.frame(emogrepl)
tweets$ID7 &amp;lt;- 1:nrow(tweets)
emogreplDF$ID7 &amp;lt;- 1:nrow(emogreplDF)
tweets &amp;lt;- merge(tweets,emogreplDF,by=&amp;quot;ID7&amp;quot;)
emosub &amp;lt;- tweets[tweets$emogrepl == &amp;quot;TRUE&amp;quot;, ]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now that you have a subset of emojis you can compare posts with emojis vs. posts without etc. etc.!&lt;/p&gt;
&lt;p&gt;How about subsetting BY emoji? Let’s look just at posts that have certain emojis in them, like the red heart emoji or the face with tears of joy.&lt;/p&gt;
&lt;p&gt;First we do pattern matching and replacement. The first command looks through the text of the emosub data frame and finds all instances in which the string ‘HEAVYBLACKHEART’ is present and then generates a list of T/F values&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;heartgrepl &amp;lt;- grepl(paste(&amp;quot; HEAVYBLACKHEART &amp;quot;), emosub$text)
# Turn that list of T/F values into a data frame so we can link it back to the original posts
heartgreplDF&amp;lt;-as.data.frame(heartgrepl)
# Make a new row so as to smush them together (the T/F designation and your data frame of posts)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
heartgreplDF$ID7 &amp;lt;- 1:nrow(heartgreplDF)
emosub &amp;lt;- merge(emosub,heartgreplDF,by=&amp;quot;ID7&amp;quot;)
redheart &amp;lt;- emosub[emosub$heartgrepl == &amp;quot;TRUE&amp;quot;, ]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s do the same with FACEWITHTEARSOFJOY&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;lolfacegrepl &amp;lt;- grepl(paste(&amp;quot; FACEWITHTEARSOFJOY &amp;quot;), emosub$text)
lolfacegreplDF&amp;lt;-as.data.frame(lolfacegrepl)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
lolfacegreplDF$ID7 &amp;lt;- 1:nrow(lolfacegreplDF)
emosub &amp;lt;- merge(emosub,lolfacegreplDF,by=&amp;quot;ID7&amp;quot;)
lolface &amp;lt;- emosub[emosub$lolfacegrepl == &amp;quot;TRUE&amp;quot;, ]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now FACEWITHHEARTSHAPEDEYES&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;hearteyesgrepl &amp;lt;- grepl(paste(&amp;quot; SMILINGFACEWITHHEARTSHAPEDEYES &amp;quot;), emosub$text)
hearteyesgreplDF&amp;lt;-as.data.frame(hearteyesgrepl)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
hearteyesgreplDF$ID7 &amp;lt;- 1:nrow(hearteyesgreplDF)
emosub &amp;lt;- merge(emosub,hearteyesgreplDF,by=&amp;quot;ID7&amp;quot;)
hearteyes &amp;lt;- emosub[emosub$hearteyesgrepl == &amp;quot;TRUE&amp;quot;, ]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Sparkles!!!!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;sparklesgrepl &amp;lt;- grepl(paste(&amp;quot; SPARKLES &amp;quot;), emosub$text)
sparklesgreplDF&amp;lt;-as.data.frame(sparklesgrepl)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
sparklesgreplDF$ID7 &amp;lt;- 1:nrow(sparklesgreplDF)
emosub &amp;lt;- merge(emosub,sparklesgreplDF,by=&amp;quot;ID7&amp;quot;)
sparkles &amp;lt;- emosub[emosub$sparklesgrepl == &amp;quot;TRUE&amp;quot;, ]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Face savouring delicious food!!!!!!!!!!!!!!!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;savourfoodgrepl &amp;lt;- grepl(paste(&amp;quot; FACESAVOURINGDELICIOUSFOOD &amp;quot;), emosub$text)
savourfoodgreplDF&amp;lt;-as.data.frame(savourfoodgrepl)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
savourfoodgreplDF$ID7 &amp;lt;- 1:nrow(savourfoodgreplDF)
emosub &amp;lt;- merge(emosub,savourfoodgreplDF,by=&amp;quot;ID7&amp;quot;)
savourfood &amp;lt;- emosub[emosub$savourfoodgrepl == &amp;quot;TRUE&amp;quot;, ]&lt;/code&gt;&lt;/pre&gt;
&lt;h2&gt;Mapping the data&lt;/h2&gt;
&lt;p&gt;Let’s have a little fun and try to map where some of these emojis occur. I am using the &lt;a href=&#34;https://github.com/dill/emoGG&#34;&gt;emoGG&lt;/a&gt; package.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# devtools::install_github(&amp;quot;dill/emoGG&amp;quot;)
library(emoGG)
# Find the emojis we want to use for a graph (might take a few times to get your search query right)
emoji_search(&amp;quot;heart face&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##                             emoji  code    keyword
## 1                        grinning 1f600       face
## 5                            grin 1f601       face
## 9                             joy 1f602       face
## 15                         smiley 1f603       face
## 19                          smile 1f604       face
## 26                    sweat_smile 1f605       face
## 35                       laughing 1f606       face
## 37                       innocent 1f607       face
## 46                           wink 1f609       face
## 50                          blush 1f60a       face
## 58                        relaxed  263a       face
## 66                            yum 1f60b       face
## 69                       relieved 1f60c       face
## 74                     heart_eyes 1f60d       face
## 81                     sunglasses 1f60e       face
## 86                          smirk 1f60f       face
## 94                 expressionless 1f611       face
## 102                         sweat 1f613       face
## 107                       pensive 1f614       face
## 112                      confused 1f615       face
## 117                    confounded 1f616       face
## 124                       kissing 1f617       face
## 128                 kissing_heart 1f618       face
## 134          kissing_smiling_eyes 1f619       face
## 138           kissing_closed_eyes 1f61a       face
## 144              stuck_out_tongue 1f61b       face
## 150  stuck_out_tongue_winking_eye 1f61c       face
## 156  stuck_out_tongue_closed_eyes 1f61d       face
## 161                  disappointed 1f61e       face
## 165                       worried 1f61f       face
## 169                         angry 1f620       face
## 176                           cry 1f622       face
## 181                     persevere 1f623       face
## 186                       triumph 1f624       face
## 191         disappointed_relieved 1f625       face
## 195                      frowning 1f626       face
## 198                     anguished 1f627       face
## 201                       fearful 1f628       face
## 207                         weary 1f629       face
## 213                        sleepy 1f62a       face
## 221                     grimacing 1f62c       face
## 224                           sob 1f62d       face
## 230                    open_mouth 1f62e       face
## 234                        hushed 1f62f       face
## 237                    cold_sweat 1f630       face
## 239                        scream 1f631       face
## 243                    astonished 1f632       face
## 247                       flushed 1f633       face
## 251                      sleeping 1f634       face
## 259                      no_mouth 1f636       face
## 261                          mask 1f637       face
## 513                           ear 1f442       face
## 514                           ear 1f442       hear
## 515                           ear 1f442      sound
## 516                           ear 1f442     listen
## 526                          kiss 1f48b       face
## 1467                        heart  2764       love
## 1468                        heart  2764       like
## 1469                        heart  2764 valentines
## 1502                        cupid 1f498      heart
## 1667                          art 1f3a8     design
## 1668                          art 1f3a8      paint
## 1669                          art 1f3a8       draw
## 2733                            a 1f170 red-square
## 2734                            a 1f170   alphabet
## 2735                            a 1f170     letter
## 3358                       bowtie             face
## 3366                    neckbeard             face&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We find the code &amp;quot;1f60d&amp;quot; for the smiling face with heart shaped eyes. Let&amp;#39;s try to graph this on a map!
# Using the ggmap package here
map &amp;lt;- get_map(location = &amp;#39;Capp St. and 20th, San Francisco,
               California&amp;#39;, zoom = 15)

lat &amp;lt;- hearteyes$latitude
lon &amp;lt;- hearteyes$longitude

# Without the background
# mapPointshearteyes &amp;lt;-  ggplot(hearteyes, aes(lon,lat)) + geom_emoji(emoji=&amp;quot;1f60d&amp;quot;)
mapPointshearteyes &amp;lt;- ggmap(map) + geom_emoji(aes(x = lon, y = lat), 
                                     data=hearteyes, emoji=&amp;quot;1f60d&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;mapPointshearteyes&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-10-emoji-maps_files/figure-html/unnamed-chunk-12-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Now let’s try multiple emojis at once (help from &lt;a href=&#34;http://blog.revolutionanalytics.com/2015/11/emojis-in-ggplot-graphics.html&#34;&gt;here&lt;/a&gt;).&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Can we do this with plain old layering?
# emoji_search(&amp;quot;sparkles&amp;quot;)
# sparkles = &amp;quot;2728&amp;quot;
# red heart = &amp;quot;2764&amp;quot; 

mapPointsmulti &amp;lt;- ggmap(map) + geom_emoji(aes(x = lon, y = lat), 
                                     data=hearteyes, emoji=&amp;quot;1f60d&amp;quot;) +
                              geom_emoji(aes(x=sparkles$longitude, y=sparkles$latitude),
                                     data=sparkles, emoji=&amp;quot;2728&amp;quot;) +
                              geom_emoji(aes(x=redheart$longitude, y=redheart$latitude),
                                     data=redheart, emoji=&amp;quot;2764&amp;quot;)

mapPointsmulti&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-10-emoji-maps_files/figure-html/unnamed-chunk-13-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;How about emojis that are associated with food?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# apparently called the &amp;#39;yum&amp;#39; emoji: 1f60b

mapPointssavourface &amp;lt;- ggmap(map) + geom_emoji(aes(x=savourfood$longitude,y=savourfood$latitude), 
                                     data=savourfood, emoji=&amp;quot;1f60b&amp;quot;)

mapPointssavourface&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-03-10-emoji-maps_files/figure-html/unnamed-chunk-14-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Thesis Time!</title>
      <link>/portfolio/2017-05-06-thesis-time/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-05-06-thesis-time/</guid>
      <description>&lt;div id=&#34;introduction&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Introduction&lt;/h2&gt;
&lt;p&gt;The following is all of the code used to run analyses used in my dissertation.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;packs = c(&amp;quot;twitteR&amp;quot;,&amp;quot;RCurl&amp;quot;,&amp;quot;RJSONIO&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;,&amp;quot;devtools&amp;quot;,&amp;quot;DataCombine&amp;quot;,&amp;quot;ggmap&amp;quot;,&amp;quot;topicmodels&amp;quot;,&amp;quot;slam&amp;quot;,&amp;quot;Rmpfr&amp;quot;,&amp;quot;tm&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,&amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;,&amp;quot;xlsx&amp;quot;,&amp;quot;ggrepel&amp;quot;,&amp;quot;lubridate&amp;quot;,&amp;quot;purrr&amp;quot;,&amp;quot;broom&amp;quot;, &amp;quot;wordcloud&amp;quot;,&amp;quot;emoGG&amp;quot;,&amp;quot;ldatuning&amp;quot;)

lapply(packs, library, character.only=T)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;getting-the-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Getting the Data&lt;/h2&gt;
&lt;p&gt;To collect data, I used the &lt;a href=&#34;https://cran.r-project.org/web/packages/twitteR/twitteR.pdf&#34;&gt;twitteR package&lt;/a&gt;. I’m interested in the Mission District neighborhood in San Francisco, California. I obtain a set of coordinates using Google maps and plug that into the ‘geocode’ parameter and then set a radius of 1 kilometer. I know from experience that I only get around 1,000 - 2,000 posts per time I do this, so I set the number of tweets (n) I would like to get from Twitter at ‘7,000’.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# key = &amp;quot;YOUR KEY HERE&amp;quot;
# secret = &amp;quot;YOUR SECRET HERE&amp;quot;

# tok = &amp;quot;YOUR TOK HERE&amp;quot;
# tok_sec = &amp;quot;YOUR TOK_SEC HERE&amp;quot;

twitter_oauth &amp;lt;- setup_twitter_oauth(key, secret, tok, tok_sec)

# To collect tweets
geo &amp;lt;- searchTwitter(&amp;#39;&amp;#39;,n=7000, geocode=&amp;#39;37.76,-122.42,1km&amp;#39;,
                     retryOnRateLimit=1)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now I want to identify emojis and separate just those posts that came from Instagram. I then save those to a CSV file and compile it by copy-pasting by hand to get a corpus.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Now you have a list of tweets. Lists are very difficult to deal with in R, so you convert this into a data frame:
geoDF&amp;lt;-twListToDF(geo)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Chances are there will be emojis in your Twitter data. You can ‘transform’ these emojis into prose using this code as well as a &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;CSV file&lt;/a&gt; I’ve put together of what all of the emojis look like in R. (The idea for this comes from &lt;a href=&#34;http://opiateforthemass.es/articles/emoticons-in-R/&#34;&gt;Jessica Peterka-Bonetta’s work&lt;/a&gt; – she has a list of emojis as well, but it does not include the newest batch of emojis, Unicode Version 9.0, nor the different skin color options for human-based emojis). If you use this emoji list for your own research, please make sure to acknowledge both myself and Jessica.&lt;/p&gt;
&lt;p&gt;Load in the CSV file. You want to make sure it is located in the correct working directory so R can find it when you tell it to read it in.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;emoticons &amp;lt;- read.csv(&amp;quot;Decoded Emojis Col Sep.csv&amp;quot;, header = T)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To transform the emojis, you first need to transform the tweet data into ASCII:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geoDF$text &amp;lt;- iconv(geoDF$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, 
                    sub = &amp;quot;byte&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To ‘count’ the emojis you do a find and replace using the &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;CSV file of ‘Decoded Emojis’&lt;/a&gt; as a reference. Here I am using the &lt;a href=&#34;http://www.inside-r.org/packages/cran/DataCombine/docs/FindReplace&#34;&gt;DataCombine package&lt;/a&gt;. What this does is identifies emojis in the tweets and then replaces them with a prose version. I used whatever description pops up when hovering one’s cursor over an emoji on an Apple emoji keyboard. If not completely the same as other platforms, it provides enough information to find the emoji in question if you are not sure which one was used in the post.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;data &amp;lt;- FindReplace(data = geoDF, Var = &amp;quot;text&amp;quot;, 
                      replaceData = emoticons,
                      from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                      exact = FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now might be a good time to save this file, perhaps in CSV format with the date of when the data was collected:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;write.csv(data,file=paste(&amp;quot;ALL&amp;quot;,Sys.Date(),&amp;quot;.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Subset to just those posts that come from Instagram Now you have a data frame which you can manipulate in various ways. For my research, I’m just interested in posts that have occured on Instagram. (Why not just access them via Instagram’s API you ask? Long story short: they are very &lt;em&gt;very&lt;/em&gt; conservative about providing access for academic research). I’ve found a work-around which is filtering mined tweets by those that have Instagram as a source:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;data &amp;lt;- data[data$statusSource == &amp;quot;&amp;lt;a href=\&amp;quot;http://instagram.com\&amp;quot; rel=\&amp;quot;nofollow\&amp;quot;&amp;gt;Instagram&amp;lt;/a&amp;gt;&amp;quot;, ]

#Save this file
write.csv(data,file=paste(&amp;quot;INSTA&amp;quot;,Sys.Date(),&amp;quot;.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Having done this for eight months, we have a nice corpus! Let’s load that in.&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;analyzing-the-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Analyzing the Data&lt;/h2&gt;
&lt;div id=&#34;preparing-data-for-topic-modeling-and-sentiment-analysis&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Preparing data for Topic Modeling and Sentiment Analysis&lt;/h4&gt;
&lt;p&gt;The data need to be processed a bit more in order to analyze them. Let’s try from the start with &lt;a href=&#34;http://tidytextmining.com/&#34;&gt;Silge and Robinson&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Get rid of stuff particular to the data (here encodings of links and such)
# Most of these are characters I don&amp;#39;t have encodings for (other scripts, etc.)

tweets$text = gsub(&amp;quot;Just posted a photo&amp;quot;,&amp;quot;&amp;quot;, tweets$text)
tweets$text = gsub( &amp;quot;&amp;lt;.*?&amp;gt;&amp;quot;, &amp;quot;&amp;quot;, tweets$text)

# Get rid of super frequent spam posters
tweets &amp;lt;- tweets[! tweets$screenName %in% c(&amp;quot;4AMSOUNDS&amp;quot;,&amp;quot;BruciusTattoo&amp;quot;,&amp;quot;LionsHeartSF&amp;quot;,&amp;quot;hermesalchemist&amp;quot;,&amp;quot;Mrsourmash&amp;quot;,&amp;quot;AaronTheEra&amp;quot;,&amp;quot;AmnesiaBar&amp;quot;,&amp;quot;audreymose2&amp;quot;,&amp;quot;audreymosez&amp;quot;,&amp;quot;Bernalcutlery&amp;quot;,&amp;quot;blncdbrkfst&amp;quot;,&amp;quot;BrunosSF&amp;quot;,&amp;quot;chiddythekidd&amp;quot;,&amp;quot;ChurchChills&amp;quot;,&amp;quot;deeXiepoo&amp;quot;,&amp;quot;fabricoutletsf&amp;quot;,&amp;quot;gever&amp;quot;,&amp;quot;miramirasf&amp;quot;,&amp;quot;papalote415&amp;quot;,&amp;quot;HappyHoundsMasg&amp;quot;,&amp;quot;faern_me&amp;quot;),]

# If you want to combine colors, run this at least 3 times over to make sure it &amp;#39;sticks&amp;#39;
# coltweets &amp;lt;- tweets
# coltweets$text &amp;lt;- gsub(&amp;quot; COLONE &amp;quot;, &amp;quot;COLONE&amp;quot;, coltweets$text)
# coltweets$text &amp;lt;- gsub(&amp;quot; COLTWO &amp;quot;, &amp;quot;COLTWO&amp;quot;, coltweets$text)
# coltweets$text &amp;lt;- gsub(&amp;quot; COLTHREE &amp;quot;, &amp;quot;COLTHREE&amp;quot;, coltweets$text)
# coltweets$text &amp;lt;- gsub(&amp;quot; COLFOUR &amp;quot;, &amp;quot;COLFOUR&amp;quot;, coltweets$text)
# coltweets$text &amp;lt;- gsub(&amp;quot; COLFIVE &amp;quot;, &amp;quot;COLFIVE&amp;quot;, coltweets$text)

# Let&amp;#39;s just use this for now. Maybe good to keep these things together
# tweets &amp;lt;- coltweets

# This makes a larger list of stop words combining those from the tm package and tidy text -- even though the tm package stop word list is pretty small anyway, just doing this just in case
data(stop_words)
mystopwords &amp;lt;- c(stopwords(&amp;#39;english&amp;#39;),stop_words$word, stopwords(&amp;#39;spanish&amp;#39;))

# Now for Silge and Robinson&amp;#39;s code. What this is doing is getting rid of 
# URLs, re-tweets (RT) and ampersands. This also gets rid of stop words 
# without having to get rid of hashtags and @ signs by using 
# str_detect and filter! 
reg &amp;lt;- &amp;quot;([^A-Za-z_\\d#@&amp;#39;]|&amp;#39;(?![A-Za-z_\\d#@]))&amp;quot;
tidy_tweets &amp;lt;- tweets %&amp;gt;% 
  filter(!str_detect(text, &amp;quot;^RT&amp;quot;)) %&amp;gt;%
  mutate(text = str_replace_all(text, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, text, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% mystopwords,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div id=&#34;frequency-analysis-and-sentiment-analysis&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Frequency analysis and Sentiment analysis&lt;/h2&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq &amp;lt;- tidy_tweets %&amp;gt;% 
  group_by(latitude,longitude) %&amp;gt;% 
  count(word, sort = TRUE) %&amp;gt;% 
  left_join(tidy_tweets %&amp;gt;% 
              group_by(latitude,longitude) %&amp;gt;% 
              summarise(total = n())) %&amp;gt;%
  mutate(freq = n/total)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The n here is the total number of times this term has shown up, and the total is how many terms there are present in a particular coordinate. Now we have a representation of terms, their frequency and their position. Now I might want to plot this somehow… one way would be to try to plot the most frequent terms (n &amp;gt; 50) (Some help on how to do this was taken from &lt;a href=&#34;http://blog.revolutionanalytics.com/2016/01/avoid-overlapping-labels-in-ggplot2-charts.html&#34;&gt;here&lt;/a&gt; and &lt;a href=&#34;http://stackoverflow.com/questions/14288001/geom-text-not-working-when-ggmap-and-geom-point-used&#34;&gt;here&lt;/a&gt;)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq2 &amp;lt;- subset(freq, n &amp;gt; 50) 

map &amp;lt;- get_map(location = &amp;#39;Valencia St. and 20th, San Francisco,
               California&amp;#39;, zoom = 15)

freq2$longitude&amp;lt;-as.numeric(freq2$longitude)
freq2$latitude&amp;lt;-as.numeric(freq2$latitude)

mapPoints &amp;lt;- ggmap(map) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freq2, aes(x = longitude, y = latitude, label = word),size = 3) &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-14-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let’s zoom into that main central area to see what’s going on!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;map2 &amp;lt;- get_map(location = &amp;#39;Valencia St. and 19th, San Francisco,
               California&amp;#39;, zoom = 16)
mapPoints2 &amp;lt;- ggmap(map2) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freq2, aes(x = longitude, y = latitude, label = word),size = 3) &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-16-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;What about 24th?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Have to go a bit bigger to get more terms
freq3 &amp;lt;- subset(freq, n &amp;gt; 15) 

map3 &amp;lt;- get_map(location = &amp;#39;Folsom St. and 24th, San Francisco,
               California&amp;#39;, zoom = 16)
mapPoints3 &amp;lt;- ggmap(map3) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freq3, aes(x = longitude, y = latitude, label = word),size = 3) &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-18-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;sentiment-analysis&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Sentiment analysis&lt;/h2&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We can also look at counts of negative and positive words
bingsenti &amp;lt;- sentiments %&amp;gt;%
  filter(lexicon ==&amp;quot;bing&amp;quot;)

bing_word_counts &amp;lt;- tidy_tweets %&amp;gt;%
  inner_join(bingsenti) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()
# If you wanted to look at these
# bing_word_counts&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Now we can graph these
bing_word_counts %&amp;gt;%
  filter(n &amp;gt; 25) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-20-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;word-cloud&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Word Cloud&lt;/h2&gt;
&lt;p&gt;In order to do a word cloud we need a document term matrix. This will also be used for topic modeling later.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# First have to make a document term matrix, which involves a few steps
tidy_tweets %&amp;gt;%
  count(document, word, sort=TRUE)

tweet_words &amp;lt;- tidy_tweets %&amp;gt;%  
  count(document, word) %&amp;gt;%
  ungroup()

total_words &amp;lt;- tweet_words %&amp;gt;% 
  group_by(document) %&amp;gt;% 
  summarize(total = sum(n))

post_words &amp;lt;- left_join(tweet_words, total_words)

dtm &amp;lt;- post_words %&amp;gt;% 
  cast_dtm(document, word, n)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freqw = data.frame(sort(colSums(as.matrix(dtm)), decreasing=TRUE))
wordcloud(rownames(freqw), freqw[,1], max.words=100, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-22-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;emojis&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Emojis&lt;/h2&gt;
&lt;p&gt;What if I want to look at just those posts that have emojis in them? Or specific emojis in general?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Identify emojis
emoticons &amp;lt;- read.csv(&amp;quot;Decoded Emojis Col Sep.csv&amp;quot;, header = T)
# This also takes time so I will not run it, but this is how you go through and identify emojis in your corpus and &amp;#39;tag&amp;#39; whether or not they are there!
# emogrepl &amp;lt;- grepl(paste(emoticons$Name, collapse = &amp;quot;|&amp;quot;), tweets$text)
# save(emogrepl,file=paste(&amp;quot;emo.Rda&amp;quot;))
# Emo here: https://www.dropbox.com/s/fqlvqfnx0n8npf2/emo.Rda?dl=0
load(&amp;quot;emo.Rda&amp;quot;)
emogreplDF&amp;lt;-as.data.frame(emogrepl)
tweets$id &amp;lt;- 1:nrow(tweets)
emogreplDF$id &amp;lt;- 1:nrow(emogreplDF)
tweets &amp;lt;- merge(tweets,emogreplDF,by=&amp;quot;id&amp;quot;)
emosub &amp;lt;- tweets[tweets$emogrepl == &amp;quot;TRUE&amp;quot;, ]

# to get JUST emojis, no text

tidy_emos &amp;lt;- emosub %&amp;gt;% 
  mutate(text = str_replace_all(text, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, text, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% mystopwords,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))

# Have to do this so they will recognize each other
tidy_emoticons &amp;lt;- emoticons %&amp;gt;% 
  mutate(Name = str_replace_all(Name, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, Name, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% mystopwords,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))

# I think a semi_join will work: &amp;quot;Return all rows from X where there are matching rows in Y, just keeping columns from X&amp;quot; (http://stat545.com/bit001_dplyr-cheatsheet.html)

emoonly &amp;lt;- semi_join(tidy_emos, tidy_emoticons, by=&amp;quot;word&amp;quot;)

freqe &amp;lt;- emoonly %&amp;gt;% 
  group_by(latitude,longitude) %&amp;gt;% 
  count(word, sort = TRUE) %&amp;gt;% 
  left_join(emoonly %&amp;gt;% 
              group_by(latitude,longitude) %&amp;gt;% 
              summarise(total = n())) %&amp;gt;%
  mutate(freq = n/total)

# freqe

# Map it
freqe2 &amp;lt;- subset(freqe, n &amp;gt; 20) 

map &amp;lt;- get_map(location = &amp;#39;Valencia St. and 20th, San Francisco,
               California&amp;#39;, zoom = 15)

freqe2$longitude&amp;lt;-as.numeric(freqe2$longitude)
freqe2$latitude&amp;lt;-as.numeric(freqe2$latitude)

mapPointse &amp;lt;- ggmap(map) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freqe2, aes(x = longitude, y = latitude, label = word),size = 3) &lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;mapPointse&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-24-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;mapping-emojis&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Mapping Emojis&lt;/h2&gt;
&lt;p&gt;To visualize emojis in our corpus, we use the &lt;a href=&#34;https://github.com/dill/emoGG&#34;&gt;emoGG package&lt;/a&gt;. (See also &lt;a href=&#34;https://lyons7.github.io/portfolio/2017-03-10-emoji-maps/&#34;&gt;here&lt;/a&gt;!) I will do a map of the most common emoji (SPARKLES) and ones related to food. This might be better on a subset so we can try that too…&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s do coffee, the egg pan thing, face savouring delicious food + ice cream?
# ice cream 1f368
# To find the codes for each emoji:
# emoji_search(&amp;quot;ice_cream&amp;quot;)
# First create a subset of just those that have ICE CREAM emoji present
icecreamg &amp;lt;- grepl(paste(&amp;quot; ICECREAM &amp;quot;), emosub$text)
icecreamgD&amp;lt;-as.data.frame(icecreamg)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
icecreamgD$ID7 &amp;lt;- 1:nrow(icecreamgD)
emosub &amp;lt;- merge(emosub,icecreamgD,by=&amp;quot;ID7&amp;quot;)
icecream &amp;lt;- emosub[emosub$icecreamg == &amp;quot;TRUE&amp;quot;, ]

# Same for &amp;#39;Face Savouring Delicious Food&amp;#39;
# savourfood: 1f60b
savourfoodgrepl &amp;lt;- grepl(paste(&amp;quot; FACESAVOURINGDELICIOUSFOOD &amp;quot;), emosub$text)
savourfoodgreplDF&amp;lt;-as.data.frame(savourfoodgrepl)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
savourfoodgreplDF$ID7 &amp;lt;- 1:nrow(savourfoodgreplDF)
emosub &amp;lt;- merge(emosub,savourfoodgreplDF,by=&amp;quot;ID7&amp;quot;)
savourfood &amp;lt;- emosub[emosub$savourfoodgrepl == &amp;quot;TRUE&amp;quot;, ]

#coffee: 2615
hotbevg &amp;lt;- grepl(paste(&amp;quot; HOTBEVERAGE &amp;quot;), emosub$text)
hotbevgD&amp;lt;-as.data.frame(hotbevg)
emosub$id &amp;lt;- 1:nrow(emosub)
hotbevgD$id &amp;lt;- 1:nrow(hotbevgD)
emosub &amp;lt;- merge(emosub,hotbevgD,by=&amp;quot;id&amp;quot;)
coffee &amp;lt;- emosub[emosub$hotbevg == &amp;quot;TRUE&amp;quot;, ] 

#knifeandfork: 1f374
mackg &amp;lt;- grepl(paste(&amp;quot; FORKANDKNIFE &amp;quot;), emosub$text)
mackgD&amp;lt;-as.data.frame(mackg)
emosub$id &amp;lt;- 1:nrow(emosub)
mackgD$id &amp;lt;- 1:nrow(mackgD)
emosub &amp;lt;- merge(emosub,mackgD,by=&amp;quot;id&amp;quot;)
mack &amp;lt;- emosub[emosub$mackg == &amp;quot;TRUE&amp;quot;, ]

#cooking: # Frying pan egg - Food
# 1f373
cookg &amp;lt;- grepl(paste(&amp;quot; COOKING &amp;quot;), emosub$text)
cookgD&amp;lt;-as.data.frame(cookg)
emosub$id &amp;lt;- 1:nrow(emosub)
cookgD$id &amp;lt;- 1:nrow(cookgD)
emosub &amp;lt;- merge(emosub,cookgD,by=&amp;quot;id&amp;quot;)
cook &amp;lt;- emosub[emosub$cookg == &amp;quot;TRUE&amp;quot;, ]


# Map this
foodmap &amp;lt;- ggmap(map) + geom_emoji(aes(x = longitude, y = latitude), 
                                     data=savourfood, emoji=&amp;quot;1f60b&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=cook, emoji=&amp;quot;1f373&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=coffee, emoji=&amp;quot;2615&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=mack, emoji=&amp;quot;1f374&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=icecream, emoji=&amp;quot;1f368&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;foodmap&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-26-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Artist palette
#1f3a8
arg &amp;lt;- grepl(paste(&amp;quot; ARTISTPALETTE &amp;quot;), emosub$text)
argD&amp;lt;-as.data.frame(arg)
emosub$id &amp;lt;- 1:nrow(emosub)
argD$id &amp;lt;- 1:nrow(argD)
emosub &amp;lt;- merge(emosub,argD,by=&amp;quot;id&amp;quot;)
art &amp;lt;- emosub[emosub$arg == &amp;quot;TRUE&amp;quot;, ]

artmap &amp;lt;- ggmap(map) + geom_emoji(aes(x = longitude, y = latitude), 
                                     data=art, emoji=&amp;quot;1f3a8&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;artmap&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-28-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;sparklesgrepl &amp;lt;- grepl(paste(&amp;quot; SPARKLES &amp;quot;), emosub$text)
sparklesgreplDF&amp;lt;-as.data.frame(sparklesgrepl)
emosub$ID7 &amp;lt;- 1:nrow(emosub)
sparklesgreplDF$ID7 &amp;lt;- 1:nrow(sparklesgreplDF)
emosub &amp;lt;- merge(emosub,sparklesgreplDF,by=&amp;quot;ID7&amp;quot;)
sparkles &amp;lt;- emosub[emosub$sparklesgrepl == &amp;quot;TRUE&amp;quot;, ]

sparkplug &amp;lt;- ggmap(map) + geom_emoji(aes(x = longitude, y = latitude), 
                                     data=sparkles, emoji=&amp;quot;2728&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;sparkplug&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-30-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;topic-modeling&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Topic Modeling&lt;/h2&gt;
&lt;div id=&#34;lda-tuning&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;LDA Tuning&lt;/h4&gt;
&lt;p&gt;Before running a topic model, I am going to try the &lt;a href=&#34;http://rpubs.comci/siri/ldatuning&#34;&gt;LDA tuning package&lt;/a&gt; to assess what might be a good number of topics.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# devtools::install_github(&amp;quot;nikita-moor/ldatuning&amp;quot;)
# install.packages(&amp;quot;ldatuning&amp;quot;)
library(&amp;quot;ldatuning&amp;quot;)
library(&amp;quot;topicmodels&amp;quot;)
# I will not run this at the moment because it takes forever!
# result &amp;lt;- FindTopicsNumber(
#   dtm,
#   topics = seq(from = 2, to = 15, by = 1),
#   metrics = c(&amp;quot;Griffiths2004&amp;quot;, &amp;quot;CaoJuan2009&amp;quot;, &amp;quot;Arun2010&amp;quot;, &amp;quot;Deveaud2014&amp;quot;),
#   method = &amp;quot;Gibbs&amp;quot;,
#   control = list(seed = 77),
#   mc.cores = 2L,
#   verbose = TRUE
# )&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When this finally finishes running, we will do the following to look at graphs of results to see ‘best’ topic number. I guess you want that range which is minimize at its lowest and maximize at its highest. So match those up.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# From here: https://www.dropbox.com/s/qplfwb0pazmk7c1/ldatuning.RData?dl=0
load(&amp;quot;ldatuning.RData&amp;quot;)

FindTopicsNumber_plot(result)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-32-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;From this, it appears that the maximum and minimum peak points are about 22. I’ll use that as my number of topics.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# load(&amp;quot;dtm.Rda&amp;quot;)
# Set parameters for Gibbs sampling (parameters those used in
# Grun and Hornik 2011)
# burnin &amp;lt;- 4000
# iter &amp;lt;- 2000
# thin &amp;lt;- 500
# seed &amp;lt;-list(2003,5,63,100001,765)
# nstart &amp;lt;- 5
# best &amp;lt;- TRUE
# k &amp;lt;- 22
# This also takes a while to run, so will just load in results
# lda &amp;lt;-LDA(dtm,k, method=&amp;quot;Gibbs&amp;quot;, 
#              control=list(nstart=nstart, seed = seed, best=best, 
#                           burnin = burnin, iter = iter, thin=thin))
# 
# # Save this (so you don&amp;#39;t have to keep running it all the time)
# save(lda,file=paste(&amp;quot;LDA&amp;quot;,k,&amp;quot;.Rda&amp;quot;))

# Let&amp;#39;s check out the results

# test_lda_td &amp;lt;- tidy(test_lda)
# From here: https://www.dropbox.com/s/4fp81smd3dbrpd6/LDA%2022%20.Rda?dl=0
load(&amp;quot;LDA 22 .Rda&amp;quot;)
# Make it tidy to visualize it, etc.
lda_td &amp;lt;- tidy(lda)

# To graph these results (too many for now, looks messy)
# lda_top_terms &amp;lt;- lda_td %&amp;gt;%
#   group_by(topic) %&amp;gt;%
#   top_n(10, beta) %&amp;gt;%
#   ungroup() %&amp;gt;%
#   arrange(topic, -beta)
# 
# top_terms &amp;lt;- lda_top_terms %&amp;gt;%
#   mutate(term = reorder(term, beta)) %&amp;gt;%
#   ggplot(aes(term, beta, fill = factor(topic))) +
#   geom_bar(stat = &amp;quot;identity&amp;quot;, show.legend = FALSE) +
#   facet_wrap(~ topic, scales = &amp;quot;free&amp;quot;) +
#   coord_flip()
# 
# top_terms&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div id=&#34;pairing-this-back-with-original-tweets&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Pairing this back with original tweets&lt;/h2&gt;
&lt;p&gt;Pair back this information with the original tweets to see how topics are distribtued, learn more about what each topic entails, etc.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Also to link things back
# Look at results
# Maybe a little easier to see than tidy graph
lda.topics &amp;lt;- as.matrix(topics(lda))
terms(lda,10)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##       Topic 1       Topic 2 Topic 3     Topic 4        Topic 5   
##  [1,] &amp;quot;happy&amp;quot;       &amp;quot;home&amp;quot;  &amp;quot;day&amp;quot;       &amp;quot;amazing&amp;quot;      &amp;quot;bar&amp;quot;     
##  [2,] &amp;quot;birthday&amp;quot;    &amp;quot;live&amp;quot;  &amp;quot;beautiful&amp;quot; &amp;quot;chapel&amp;quot;       &amp;quot;time&amp;quot;    
##  [3,] &amp;quot;party&amp;quot;       &amp;quot;black&amp;quot; &amp;quot;favorite&amp;quot;  &amp;quot;ready&amp;quot;        &amp;quot;taqueria&amp;quot;
##  [4,] &amp;quot;fire&amp;quot;        &amp;quot;rose&amp;quot;  &amp;quot;check&amp;quot;     &amp;quot;friend&amp;quot;       &amp;quot;friday&amp;quot;  
##  [5,] &amp;quot;friends&amp;quot;     &amp;quot;music&amp;quot; &amp;quot;days&amp;quot;      &amp;quot;special&amp;quot;      &amp;quot;taco&amp;quot;    
##  [6,] &amp;quot;weekend&amp;quot;     &amp;quot;sweet&amp;quot; &amp;quot;putt&amp;quot;      &amp;quot;water&amp;quot;        &amp;quot;sushi&amp;quot;   
##  [7,] &amp;quot;holiday&amp;quot;     &amp;quot;real&amp;quot;  &amp;quot;urban&amp;quot;     &amp;quot;awesome&amp;quot;      &amp;quot;baby&amp;quot;    
##  [8,] &amp;quot;family&amp;quot;      &amp;quot;days&amp;quot;  &amp;quot;heath&amp;quot;     &amp;quot;flour&amp;quot;        &amp;quot;theater&amp;quot; 
##  [9,] &amp;quot;paypopper&amp;quot;   &amp;quot;sf&amp;quot;    &amp;quot;lovely&amp;quot;    &amp;quot;@thechapelsf&amp;quot; &amp;quot;lunch&amp;quot;   
## [10,] &amp;quot;celebrating&amp;quot; &amp;quot;basil&amp;quot; &amp;quot;ceramics&amp;quot;  &amp;quot;shot&amp;quot;         &amp;quot;ramen&amp;quot;   
##       Topic 6              Topic 7         Topic 8    Topic 9    
##  [1,] &amp;quot;facewithtearsofjoy&amp;quot; &amp;quot;love&amp;quot;          &amp;quot;night&amp;quot;    &amp;quot;francisco&amp;quot;
##  [2,] &amp;quot;armory&amp;quot;             &amp;quot;heavyblackhea&amp;quot; &amp;quot;tonight&amp;quot;  &amp;quot;san&amp;quot;      
##  [3,] &amp;quot;alamo&amp;quot;              &amp;quot;city&amp;quot;          &amp;quot;saturday&amp;quot; &amp;quot;mission&amp;quot;  
##  [4,] &amp;quot;drafthouse&amp;quot;         &amp;quot;building&amp;quot;      &amp;quot;tomorrow&amp;quot; &amp;quot;district&amp;quot; 
##  [5,] &amp;quot;club&amp;quot;               &amp;quot;guys&amp;quot;          &amp;quot;bay&amp;quot;      &amp;quot;#igerssf&amp;quot; 
##  [6,] &amp;quot;life&amp;quot;               &amp;quot;people&amp;quot;        &amp;quot;monday&amp;quot;   &amp;quot;yeah&amp;quot;     
##  [7,] &amp;quot;video&amp;quot;              &amp;quot;time&amp;quot;          &amp;quot;playing&amp;quot;  &amp;quot;cookie&amp;quot;   
##  [8,] &amp;quot;fun&amp;quot;                &amp;quot;techo&amp;quot;         &amp;quot;books&amp;quot;    &amp;quot;streets&amp;quot;  
##  [9,] &amp;quot;kink&amp;quot;               &amp;quot;trip&amp;quot;          &amp;quot;free&amp;quot;     &amp;quot;dr&amp;quot;       
## [10,] &amp;quot;posted&amp;quot;             &amp;quot;hard&amp;quot;          &amp;quot;amnesia&amp;quot;  &amp;quot;beer&amp;quot;     
##       Topic 10                     Topic 11       Topic 12 
##  [1,] &amp;quot;tartine&amp;quot;                    &amp;quot;food&amp;quot;         &amp;quot;dinner&amp;quot; 
##  [2,] &amp;quot;manufactory&amp;quot;                &amp;quot;week&amp;quot;         &amp;quot;photo&amp;quot;  
##  [3,] &amp;quot;bakery&amp;quot;                     &amp;quot;dog&amp;quot;          &amp;quot;bear&amp;quot;   
##  [4,] &amp;quot;stop&amp;quot;                       &amp;quot;cheese&amp;quot;       &amp;quot;lazy&amp;quot;   
##  [5,] &amp;quot;@sfmanufactory&amp;quot;             &amp;quot;school&amp;quot;       &amp;quot;foreign&amp;quot;
##  [6,] &amp;quot;cream&amp;quot;                      &amp;quot;wineglass&amp;quot;    &amp;quot;cinema&amp;quot; 
##  [7,] &amp;quot;bread&amp;quot;                      &amp;quot;trick&amp;quot;        &amp;quot;ladies&amp;quot; 
##  [8,] &amp;quot;ice&amp;quot;                        &amp;quot;perfect&amp;quot;      &amp;quot;#repost&amp;quot;
##  [9,] &amp;quot;facesavouringdeliciousfood&amp;quot; &amp;quot;tour&amp;quot;         &amp;quot;miss&amp;quot;   
## [10,] &amp;quot;pizza&amp;quot;                      &amp;quot;forkandknife&amp;quot; &amp;quot;painted&amp;quot;
##       Topic 13                          Topic 14    Topic 15          
##  [1,] &amp;quot;colone&amp;quot;                          &amp;quot;san&amp;quot;       &amp;quot;#sanfrancisco&amp;quot;   
##  [2,] &amp;quot;sparkles&amp;quot;                        &amp;quot;mission&amp;quot;   &amp;quot;#sf&amp;quot;             
##  [3,] &amp;quot;coltwo&amp;quot;                          &amp;quot;francisco&amp;quot; &amp;quot;#mission&amp;quot;        
##  [4,] &amp;quot;colthree&amp;quot;                        &amp;quot;district&amp;quot;  &amp;quot;#missiondistrict&amp;quot;
##  [5,] &amp;quot;twoheas&amp;quot;                         &amp;quot;#igerssf&amp;quot;  &amp;quot;#california&amp;quot;     
##  [6,] &amp;quot;okhandsign&amp;quot;                      &amp;quot;fran&amp;quot;      &amp;quot;#dolorespark&amp;quot;    
##  [7,] &amp;quot;personraisinghandsincelebration&amp;quot; &amp;quot;reading&amp;quot;   &amp;quot;#bayarea&amp;quot;        
##  [8,] &amp;quot;personwithfoldedhands&amp;quot;           &amp;quot;#sfo&amp;quot;      &amp;quot;#themission&amp;quot;     
##  [9,] &amp;quot;colfour&amp;quot;                         &amp;quot;break&amp;quot;     &amp;quot;#usa&amp;quot;            
## [10,] &amp;quot;signofthehorns&amp;quot;                  &amp;quot;bright&amp;quot;    &amp;quot;#sanfran&amp;quot;        
##       Topic 16                       Topic 17           Topic 18  
##  [1,] &amp;quot;smilingfacewithheashapedeyes&amp;quot; &amp;quot;park&amp;quot;             &amp;quot;mission&amp;quot; 
##  [2,] &amp;quot;studios&amp;quot;                      &amp;quot;dolores&amp;quot;          &amp;quot;street&amp;quot;  
##  [3,] &amp;quot;finally&amp;quot;                      &amp;quot;sf&amp;quot;               &amp;quot;art&amp;quot;     
##  [4,] &amp;quot;sunday&amp;quot;                       &amp;quot;blacksunwithrays&amp;quot; &amp;quot;valencia&amp;quot;
##  [5,] &amp;quot;yesterday&amp;quot;                    &amp;quot;#dolorespark&amp;quot;     &amp;quot;24th&amp;quot;    
##  [6,] &amp;quot;pacific&amp;quot;                      &amp;quot;palmtree&amp;quot;         &amp;quot;station&amp;quot; 
##  [7,] &amp;quot;studio&amp;quot;                       &amp;quot;afternoon&amp;quot;        &amp;quot;st&amp;quot;      
##  [8,] &amp;quot;southern&amp;quot;                     &amp;quot;bridgeatnight&amp;quot;    &amp;quot;16th&amp;quot;    
##  [9,] &amp;quot;pretty&amp;quot;                       &amp;quot;sunny&amp;quot;            &amp;quot;cha&amp;quot;     
## [10,] &amp;quot;fur&amp;quot;                          &amp;quot;summer&amp;quot;           &amp;quot;gray&amp;quot;    
##       Topic 19        Topic 20     Topic 21    Topic 22     
##  [1,] &amp;quot;alley&amp;quot;         &amp;quot;kitchen&amp;quot;    &amp;quot;time&amp;quot;      &amp;quot;coffee&amp;quot;     
##  [2,] &amp;quot;clarion&amp;quot;       &amp;quot;#foodporn&amp;quot;  &amp;quot;elbo&amp;quot;      &amp;quot;morning&amp;quot;    
##  [3,] &amp;quot;#sf&amp;quot;           &amp;quot;restaurant&amp;quot; &amp;quot;chocolate&amp;quot; &amp;quot;cafe&amp;quot;       
##  [4,] &amp;quot;#streetart&amp;quot;    &amp;quot;super&amp;quot;      &amp;quot;fun&amp;quot;       &amp;quot;hotbeverage&amp;quot;
##  [5,] &amp;quot;#art&amp;quot;          &amp;quot;story&amp;quot;      &amp;quot;mission&amp;quot;   &amp;quot;barrel&amp;quot;     
##  [6,] &amp;quot;#graffiti&amp;quot;     &amp;quot;brunch&amp;quot;     &amp;quot;christmas&amp;quot; &amp;quot;#coffee&amp;quot;    
##  [7,] &amp;quot;#clarionalley&amp;quot; &amp;quot;chicken&amp;quot;    &amp;quot;theatre&amp;quot;   &amp;quot;tea&amp;quot;        
##  [8,] &amp;quot;#mural&amp;quot;        &amp;quot;#food&amp;quot;      &amp;quot;house&amp;quot;     &amp;quot;shop&amp;quot;       
##  [9,] &amp;quot;mural&amp;quot;         &amp;quot;thai&amp;quot;       &amp;quot;hot&amp;quot;       &amp;quot;ritual&amp;quot;     
## [10,] &amp;quot;link&amp;quot;          &amp;quot;craftsman&amp;quot;  &amp;quot;church&amp;quot;    &amp;quot;acrylic&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Check at top 50 terms in each topic
# lda.terms &amp;lt;- as.matrix(terms(lda,15))
# Save as CSV file to look at a bit closer
# write.csv(lda.terms,file=paste(&amp;quot;TIDY_LDA&amp;quot;,k,&amp;quot;TopicstoTerms.csv&amp;quot;))

# Actual probabilities
topicProbabilities &amp;lt;- as.data.frame(lda@gamma)
# write.csv(topicProbabilities,
#          file=paste(&amp;quot;TIDYLDA&amp;quot;,k,&amp;quot;TopicProbabilities.csv&amp;quot;))

#Write out the topics to a data frame so you can work with them
test &amp;lt;- as.data.frame(lda.topics)
# We won&amp;#39;t label these topics bc too many, difficult to label. If you wanted to label, however, this is how you would do it. 
# a&amp;lt;-c(&amp;#39;Evaluation&amp;#39;, &amp;#39;Food&amp;#39;,&amp;#39;Performance Promos&amp;#39;, &amp;#39;Leisure&amp;#39;, &amp;#39;Places&amp;#39;,
# &amp;#39;Nightlife&amp;#39;, &amp;#39;Activism/Campaigns&amp;#39;,&amp;#39;Art&amp;#39;,&amp;#39;Outdoors&amp;#39;,&amp;#39;Service/Product Promos&amp;#39;)
# b&amp;lt;-c(1,2,3,4,5,6,7,8,9,10)
# namesdf&amp;lt;-data.frame(&amp;quot;Name&amp;quot;=a,&amp;quot;Number&amp;quot;=b)
# test$V1&amp;lt;-as.factor(test$V1)
# newtopics &amp;lt;- FindReplace(data = test, Var = &amp;quot;V1&amp;quot;, replaceData = namesdf,
#                        from = &amp;quot;Number&amp;quot;, to = &amp;quot;Name&amp;quot;, exact = TRUE)

#Merge topics with tweet corpus
tweets$id &amp;lt;- 1:nrow(tweets)
test$id &amp;lt;- 1:nrow(test)
tweets &amp;lt;- merge(tweets,test,by=&amp;quot;id&amp;quot;)
# Save this
# save(tweets,file=paste(&amp;quot;tweets&amp;quot;,Sys.Date(),&amp;quot;.Rda&amp;quot;))
# load(&amp;quot;tweets 2017-03-22 .Rda&amp;quot;)

#Merge topic probabilities with tweet corpus
topicProbabilities$id &amp;lt;- 1:nrow(topicProbabilities)
tweets &amp;lt;- merge(tweets, topicProbabilities,by=&amp;quot;id&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;visualizing-topic-model-results&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Visualizing Topic Model Results&lt;/h2&gt;
&lt;p&gt;You can now map your posts and see where assigned topics are happening!&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets$longitude&amp;lt;-as.numeric(tweets$longitude)
tweets$latitude &amp;lt;- as.numeric(tweets$latitude)
tweets$V1.x &amp;lt;- factor(tweets$V1.x)
Topics&amp;lt;-tweets$V1.x
mapPointstopics &amp;lt;- ggmap(map) + geom_point(aes(x = longitude, y = latitude, 
                                         color=Topics), 
                                     data=tweets, alpha=0.5, size = 3)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;mapPointstopics&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-36-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;What a mess!&lt;/p&gt;
&lt;p&gt;How about over time?&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;visualizing-the-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Visualizing the data&lt;/h2&gt;
&lt;p&gt;We can also look at WHEN the posts were generated. We can make a graph of post frequency over time.Graphs constructed with help from &lt;a href=&#34;http://www.cyclismo.org/tutorial/R/time.html&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;https://gist.github.com/stephenturner/3132596&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://stackoverflow.com/questions/27626915/r-graph-frequency-of-observations-over-time-with-small-value-range&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://michaelbommarito.com/2011/03/12/a-quick-look-at-march11-saudi-tweets/&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://stackoverflow.com/questions/31796744/plot-count-frequency-of-tweets-for-word-by-month&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;https://stat.ethz.ch/R-manual/R-devel/library/base/html/as.POSIXlt.html&#34;&gt;here&lt;/a&gt;, &lt;a href=&#34;http://sape.inf.usi.ch/quick-reference/ggplot2/geom&#34;&gt;here&lt;/a&gt; and &lt;a href=&#34;http://stackoverflow.com/questions/3541713/how-to-plot-two-histograms-together-in-r&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tweets$created2 &amp;lt;- as.POSIXct(tweets$created, format=&amp;quot;%m/%d/%Y %H:%M&amp;quot;)
tweets$created3&amp;lt;-format(tweets$created2,&amp;#39;%H:%M:%S&amp;#39;)
d3 &amp;lt;- as.data.frame(table(tweets$created3))
d3 &amp;lt;- d3[order(d3$Freq, decreasing=T), ]
names(d3) &amp;lt;- c(&amp;quot;created3&amp;quot;,&amp;quot;freq3&amp;quot;)
tweets &amp;lt;- merge(tweets,d3,by=&amp;quot;created3&amp;quot;)
tweets$created3 &amp;lt;- as.POSIXct(tweets$created3, format=&amp;quot;%H:%M:%S&amp;quot;)
minutes &amp;lt;- 60

Topics&amp;lt;-tweets$V1.x
Time &amp;lt;- tweets$created3&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;ggplot(tweets, aes(Time, color = Topics)) + 
  geom_freqpoly(binwidth=60*minutes)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-38-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# For a more general trend
ggplot(tweets, aes(Time)) + 
  geom_freqpoly(binwidth=60*minutes)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-38-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;matching-tweets-with-ll-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Matching tweets with LL data&lt;/h2&gt;
&lt;p&gt;What we are trying to do is to match up locations in the physical LL with the digital LL and then find the most common topic associated with a physical location. Because we do not have &lt;em&gt;exact&lt;/em&gt; matches, we will try the &lt;a href=&#34;https://cran.r-project.org/web/packages/fuzzyjoin/fuzzyjoin.pdf&#34;&gt;fuzzyjoin package&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(fuzzyjoin)
library(dplyr)
pairsdf &amp;lt;- ll %&amp;gt;%
  geo_inner_join(tweets, unit=&amp;#39;km&amp;#39;,distance_col=&amp;quot;distance&amp;quot;) %&amp;gt;%
  filter(distance &amp;lt;= 0.018288)

# What does this look like on a map?

# mapPointsall &amp;lt;- ggmap(map) + geom_point(aes(x = longitude.x, y = latitude.x), 
#                                     data=pairsdf, alpha=0.5)
# mapPointsall&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now I have a data frame with a row of each time a post has occurred in a 30 foot vicinity of an LL object. What I would like to do is figure out the most common topic that is associated with a particular sign. We’ll use the idea of ‘mode’ here with our topics and the &lt;em&gt;group_by()&lt;/em&gt; function from dplyr as suggested &lt;a href=&#34;http://stackoverflow.com/questions/25198442/how-to-calculate-mean-median-per-group-in-a-dataframe-in-r&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;As R does not have a built in function for mode, we build one. Code for this available &lt;a href=&#34;https://www.tutorialspoint.com/r/r_mean_median_mode.htm&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# To get the mode
getmode &amp;lt;- function(v) {
   uniqv &amp;lt;- unique(v)
   uniqv[which.max(tabulate(match(v, uniqv)))]
}

# Tell R your topic categories are a number so it can deal with them
pairsdf$V1.x&amp;lt;- as.numeric(pairsdf$V1.x)

# Now calculate things about the topics per sign
topicmode &amp;lt;- pairsdf%&amp;gt;%
group_by(SIGN_ID)%&amp;gt;% 
summarise(Mode = getmode(V1.x))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s now combine this with our other data, but just include those instances that have a topic assigned (not all signs got a corresponding tweet)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;topicsigns &amp;lt;- inner_join(ll, topicmode, by = &amp;quot;SIGN_ID&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is kind of messy, so let’s subset the data frame to just have the things we are interested in. Help from &lt;a href=&#34;https://www.r-bloggers.com/taking-a-subset-of-a-data-frame-in-r/&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;topicsigns &amp;lt;- topicsigns[,c(&amp;quot;SIGN_ID&amp;quot;,&amp;quot;latitude&amp;quot;,&amp;quot;longitude&amp;quot;,&amp;quot;LOCATION&amp;quot;,&amp;quot;LANGUAGE&amp;quot;,&amp;quot;COMMUNICATIVE_ROLE&amp;quot;,&amp;quot;MATERIALITY&amp;quot;,&amp;quot;CONTEXT_FRAME&amp;quot;,&amp;quot;YELP&amp;quot;,&amp;quot;CLOSED&amp;quot;,&amp;quot;Mode&amp;quot;)]    # get all rows, only relevant columns

# Rename columns so they make more sense (help from here: http://stackoverflow.com/questions/21502465/replacement-for-rename-in-dplyr/26146202#26146202)
topicsigns &amp;lt;- rename(topicsigns, Topic = Mode)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;gams&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;GAMs!&lt;/h2&gt;
&lt;p&gt;Now onto statistics. We want to see what has the most influence on language displayed in a sign. Let’s use a generalized additive model.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(mgcv)
# Let&amp;#39;s visualize our LL data
# We want to change the order on the plot so it&amp;#39;s easier to look at (help from http://stackoverflow.com/questions/12774210/how-do-you-specifically-order-ggplot2-x-axis-instead-of-alphabetical-order)
ll$LANGUAGE &amp;lt;- as.character(ll$LANGUAGE)
Language &amp;lt;- factor(ll$LANGUAGE, levels=c(&amp;quot;English&amp;quot;, &amp;quot;Eng_Span&amp;quot;,&amp;#39;Equal&amp;#39;,&amp;#39;Spanish&amp;#39;, &amp;#39;Span_Eng&amp;#39;,&amp;quot;Other (Chinese)&amp;quot;,&amp;quot;Other (Thai)&amp;quot;,&amp;quot;Other (Tagalog)&amp;quot;))

# Different colors help from http://stackoverflow.com/questions/19778612/change-color-for-two-geom-point-in-ggplot2

# Colorblind palette (help from http://www.cookbook-r.com/Graphs/Colors_(ggplot2)/#a-colorblind-friendly-palette)
# cbPalette &amp;lt;- c(&amp;quot;#999999&amp;quot;, &amp;quot;#E69F00&amp;quot;, &amp;quot;#56B4E9&amp;quot;, &amp;quot;#009E73&amp;quot;, &amp;quot;#F0E442&amp;quot;, &amp;quot;#0072B2&amp;quot;, &amp;quot;#D55E00&amp;quot;, &amp;quot;#CC79A7&amp;quot;)
# mapPoints &amp;lt;- ggmap(map) + geom_point(aes(x = lon, y = lat,color=Language),data=newdata, alpha = 0.7, size=2) + scale_colour_manual(values=cbPalette)

mapll &amp;lt;- get_map(location = &amp;#39;Van Ness and 22nd, San Francisco,
               California&amp;#39;, zoom = 15)


ll$longitude &amp;lt;- as.numeric(ll$longitude)
ll$latitude &amp;lt;- as.numeric(ll$latitude)

Longitude &amp;lt;- ll$longitude
Latitude &amp;lt;- ll$latitude

mapPointsll &amp;lt;- ggmap(mapll) + geom_point(aes(x = Longitude, y = Latitude,color=Language),data=ll, size=1.5) + scale_colour_manual(values = c(&amp;quot;Spanish&amp;quot; = &amp;quot;blue&amp;quot;, &amp;quot;English&amp;quot; = &amp;quot;magenta&amp;quot;, &amp;quot;Eng_Span&amp;quot; = &amp;quot;red&amp;quot;, &amp;quot;Span_Eng&amp;quot; = &amp;quot;#339900&amp;quot;, &amp;quot;Equal&amp;quot; = &amp;quot;orange&amp;quot;, &amp;quot;Other (Chinese)&amp;quot;=&amp;quot;purple&amp;quot;,&amp;quot;Other (Thai)&amp;quot; =&amp;quot;#FFCC00&amp;quot;,&amp;quot;Other (Tagalog)&amp;quot; = &amp;quot;grey&amp;quot; ))&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;mapPointsll&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-44-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;div id=&#34;model-selection&#34; class=&#34;section level3&#34;&gt;
&lt;h3&gt;Model Selection&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Generalized Linear Models (Logistic Regression, Multinomial Logistic Regression)&lt;/li&gt;
&lt;li&gt;Pros: Enable a categorical output&lt;/li&gt;
&lt;li&gt;Cons: Difficult to capture nonlinear patterns, involves transformation (logit)&lt;/li&gt;
&lt;li&gt;Cons: Difficult to include coordinates&lt;/li&gt;
&lt;li&gt;Generalized Additive Model&lt;/li&gt;
&lt;li&gt;Pros: Relationship between IV and DV not assumed to be linear&lt;/li&gt;
&lt;li&gt;Pros: Can deal with coordinates with a smooth! Allows the trend of DV to be summarized as a function of more than one IV (latitude and longitude)&lt;/li&gt;
&lt;li&gt;Pros: Can deal with my weird time distribution with a smooth as well!&lt;/li&gt;
&lt;/ul&gt;
&lt;div id=&#34;results&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Results&lt;/h4&gt;
&lt;p&gt;Let’s explore the multinom and see what it can tell us about all of these things&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Plotting the data (help on how to manipulate this graph from here: http://r.789695.n4.nabble.com/Ordering-of-stack-in-ggplot-package-ggplot2-td3917159.html)

# Overall counts

dat &amp;lt;- data.frame(table(ll$LOCATION,ll$LANGUAGE))
dat$Var1 &amp;lt;- factor(dat$Var1, levels = c(&amp;quot;Mission&amp;quot;, &amp;quot;24th&amp;quot;, &amp;quot;Valencia&amp;quot;,&amp;quot;18th&amp;quot;))

dat$Var2 &amp;lt;- factor(dat$Var2, levels = c(&amp;quot;English&amp;quot;, &amp;quot;Eng_Span&amp;quot;, &amp;quot;Equal&amp;quot;,&amp;quot;Span_Eng&amp;quot;,&amp;quot;Spanish&amp;quot;,&amp;quot;Other (Chinese)&amp;quot;,&amp;quot;Other (Thai)&amp;quot;,&amp;quot;Other (Tagalog)&amp;quot;))

names(dat) &amp;lt;- c(&amp;quot;Location&amp;quot;,&amp;quot;Language&amp;quot;,&amp;quot;Count&amp;quot;)
# levels(dat$Language)
ggplot(data=dat, aes(x=Location, y=Count, fill=Language)) + geom_bar(stat=&amp;quot;identity&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-45-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Now percentages
please=prop.table(table(ll$LOCATION, ll$LANGUAGE))
please2 &amp;lt;- data.frame(please)
please2$Var1 &amp;lt;- factor(please2$Var1, levels = c(&amp;quot;Mission&amp;quot;, &amp;quot;24th&amp;quot;, &amp;quot;Valencia&amp;quot;,&amp;quot;18th&amp;quot;))

please2$Var2 &amp;lt;- factor(please2$Var2, levels = c(&amp;quot;English&amp;quot;, &amp;quot;Eng_Span&amp;quot;, &amp;quot;Equal&amp;quot;,&amp;quot;Span_Eng&amp;quot;,&amp;quot;Spanish&amp;quot;,&amp;quot;Other (Chinese)&amp;quot;,&amp;quot;Other (Thai)&amp;quot;,&amp;quot;Other (Tagalog)&amp;quot;))

names(please2) &amp;lt;- c(&amp;quot;Location&amp;quot;,&amp;quot;Language&amp;quot;, &amp;quot;Frequency&amp;quot;)
# Help from http://stackoverflow.com/questions/9563368/create-stacked-percent-barplot-in-r
library(scales)
ggplot(please2,aes(x = Location, y = Frequency,fill = Language)) + 
    geom_bar(position = &amp;quot;fill&amp;quot;,stat = &amp;quot;identity&amp;quot;) + 
    scale_y_continuous(labels = percent_format())&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-45-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;div id=&#34;multinomial-logistic-regression&#34; class=&#34;section level5&#34;&gt;
&lt;h5&gt;Multinomial Logistic Regression&lt;/h5&gt;
&lt;p&gt;The results of this are so ugly – the p value also has to computed separately. But here is how it is done.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(nnet)
ll$LANGUAGE &amp;lt;- as.factor(ll$LANGUAGE)
multi &amp;lt;- multinom(LANGUAGE ~ LOCATION, data=ll)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # weights:  40 (28 variable)
## initial  value 2145.983671 
## iter  10 value 1081.616009
## iter  20 value 1027.458362
## iter  30 value 1025.668541
## iter  40 value 1025.609658
## iter  50 value 1025.585372
## final  value 1025.584871 
## converged&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;summary(multi)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Call:
## multinom(formula = LANGUAGE ~ LOCATION, data = ll)
## 
## Coefficients:
##                   (Intercept) LOCATION24th LOCATIONMission
## English          3.713392e+00   -1.6606712      -1.1414267
## Equal            9.161193e-01   -1.6633643      -1.2345010
## Other (Chinese) -9.539999e+00   -2.5206831       7.8354395
## Other (Tagalog) -1.005856e+01   -2.9770588       6.9682681
## Other (Thai)    -9.266575e+00    6.3219050      -0.4741931
## Span_Eng        -1.264373e-04    0.3878855       0.3748708
## Spanish          1.945735e+00   -0.3576242      -0.3272004
##                 LOCATIONValencia
## English               -0.1162223
## Equal                -10.6594671
## Other (Chinese)       -2.5346037
## Other (Tagalog)       -2.6799698
## Other (Thai)          -3.8007603
## Span_Eng             -10.0641616
## Spanish               -1.7226796
## 
## Std. Errors:
##                 (Intercept) LOCATION24th LOCATIONMission LOCATIONValencia
## English           0.7156184    0.7559734       0.7490253        0.8768839
## Equal             0.8366090    0.9293296       0.8988160       65.2725028
## Other (Chinese)  83.3773119  126.7018959      83.3790835      225.3583589
## Other (Tagalog) 108.0554714  189.2203510     108.0603056      311.1603627
## Other (Thai)     72.7242951   72.7315331      77.8555077      351.5400629
## Span_Eng          0.9999461    1.0431848       1.0375929       76.6336199
## Spanish           0.7558726    0.7966966       0.7910811        1.0105941
## 
## Residual Deviance: 2051.17 
## AIC: 2107.17&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Get p vals and coefficients
z &amp;lt;- summary(multi)$coefficients/summary(multi)$standard.errors 
p &amp;lt;- (1 - pnorm(abs(z), 0, 1)) * 2 
p&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##                  (Intercept) LOCATION24th LOCATIONMission LOCATIONValencia
## English         2.113511e-07   0.02803958       0.1275380       0.89455708
## Equal           2.734997e-01   0.07347739       0.1696048       0.87027660
## Other (Chinese) 9.089052e-01   0.98412746       0.9251301       0.99102639
## Other (Tagalog) 9.258344e-01   0.98744717       0.9485841       0.99312804
## Other (Thai)    8.986075e-01   0.93073423       0.9951404       0.99137365
## Span_Eng        9.998991e-01   0.71002077       0.7178835       0.89551562
## Spanish         1.004847e-02   0.65351549       0.6791585       0.08826521&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Get the odds and coefficients
# exp(coef(multi))&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div id=&#34;gams-1&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;GAMs!&lt;/h2&gt;
&lt;p&gt;Let’s turn to GAMs to look at LL distributions.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gamELL= gam(I(LANGUAGE==&amp;quot;English&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED,family=binomial, data=ll)

gamSLL= gam(I(LANGUAGE==&amp;quot;Spanish&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED,family=binomial, data=ll)

gamESLL= gam(I(LANGUAGE==&amp;quot;Eng_Span&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED,family=binomial, data=ll)

gamSELL= gam(I(LANGUAGE==&amp;quot;Span_Eng&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED,family=binomial, data=ll)

gamEQLL= gam(I(LANGUAGE==&amp;quot;Equal&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED,family=binomial, data=ll)&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;gam-diagnostics&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;GAM diagnostics&lt;/h4&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamELL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9993302             0.6214292
## observed 0.9993302             0.3477924
## estimate 0.9993302             0.3560434&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamSLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9993302             0.6214292
## observed 0.9993302             0.2714755
## estimate 0.9993302             0.3560434&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamESLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9993302             0.6214292
## observed 0.9993302             0.1284416
## estimate 0.9993302             0.3560434&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamSELL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9993302             0.6214292
## observed 0.9993302             0.3752692
## estimate 0.9993302             0.3560434&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamEQLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9993302             0.6214292
## observed 0.9993302             0.2880259
## estimate 0.9993302             0.3560434&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamELL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 2 iterations.
## Gradient range [2.937159e-09,2.937159e-09]
## (score 0.07031652 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.002898905,0.002898905].
## Model rank =  106 / 106 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000 21.456   0.881       0&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamSLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 3 iterations.
## Gradient range [-3.796155e-10,-3.796155e-10]
## (score -0.1045091 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.00330123,0.00330123].
## Model rank =  106 / 106 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000 35.459   0.887       0&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamESLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 2 iterations.
## Gradient range [-5.881589e-09,-5.881589e-09]
## (score -0.5780793 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.001517681,0.001517681].
## Model rank =  106 / 106 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000 18.706   0.949     0.5&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamSELL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 4 iterations.
## Gradient range [-6.671218e-08,-6.671218e-08]
## (score -0.5312013 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.001353063,0.001353063].
## Model rank =  106 / 106 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                          k&amp;#39;   edf k-index p-value
## s(latitude,longitude) 59.00  6.55    0.92    0.18&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamEQLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 3 iterations.
## Gradient range [-8.346794e-10,-8.346794e-10]
## (score -0.7151074 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.001447556,0.001447556].
## Model rank =  106 / 106 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000  7.110   0.917    0.24&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-48-1.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-48-2.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-48-3.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-48-4.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-48-5.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;gam-results&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;GAM results&lt;/h4&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# English
summary(gamELL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;English&amp;quot;) ~ s(latitude, longitude, k = 60) + YELP + 
##     COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                    44.0237 51896.9721   0.001
## YELP                                            0.7416     0.1410   5.260
## COMMUNICATIVE_ROLEEstablishment_Description    -1.0166     0.4430  -2.295
## COMMUNICATIVE_ROLEEstablishment_Name           -0.8597     0.4229  -2.033
## COMMUNICATIVE_ROLEGraffiti                      0.1732     0.7926   0.219
## COMMUNICATIVE_ROLEinformation                 -23.1788 24511.2101  -0.001
## COMMUNICATIVE_ROLEInformation                  -1.3411     0.4417  -3.036
## COMMUNICATIVE_ROLEInstructions                  0.3908     1.4229   0.275
## COMMUNICATIVE_ROLELeaflet                      -1.1885     1.4932  -0.796
## COMMUNICATIVE_ROLESlogan                       -0.9822     1.3350  -0.736
## COMMUNICATIVE_ROLEStreet_Signs                  0.5547     1.0081   0.550
## COMMUNICATIVE_ROLETrademark                    18.4011 48196.1446   0.000
## MATERIALITYHand_Written                         0.6733     0.8570   0.786
## MATERIALITYHome_Printed                         0.5714     0.8598   0.665
## MATERIALITYPermanent                            1.7312     1.0364   1.670
## MATERIALITYProfessionally_Printed               1.4481     0.8451   1.714
## CONTEXT_FRAMEAuto_Mechanic                     -0.6429 51897.1058   0.000
## CONTEXT_FRAMEBakery                           -24.9360 19246.8495  -0.001
## CONTEXT_FRAMEBar                              -22.3820 19246.8495  -0.001
## CONTEXT_FRAMEBeauty_Hair_Salon                -23.2895 19246.8494  -0.001
## CONTEXT_FRAMEBusiness                         -22.4638 19246.8494  -0.001
## CONTEXT_FRAMECafe                              -1.4916 20584.5040   0.000
## CONTEXT_FRAMEClothing                         -23.2780 19246.8495  -0.001
## CONTEXT_FRAMECommentary                       -22.1075 19246.8495  -0.001
## CONTEXT_FRAMEExternal                         -22.5786 19246.8494  -0.001
## CONTEXT_FRAMEFlier                            -45.8507 39139.1784  -0.001
## CONTEXT_FRAMEGallery_Museum                   -24.1304 19246.8495  -0.001
## CONTEXT_FRAMEGrocery                           -2.7168 29903.7031   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store             -23.5920 19246.8494  -0.001
## CONTEXT_FRAMEGym_Fitness_Studio                 0.0302 28792.2798   0.000
## CONTEXT_FRAMEHardware                          -1.9022 29882.6610   0.000
## CONTEXT_FRAMEHotel                            -22.6879 19246.8495  -0.001
## CONTEXT_FRAMEInstitution                      -23.0377 19246.8494  -0.001
## CONTEXT_FRAMEJewelry_Store                    -22.3474 19246.8495  -0.001
## CONTEXT_FRAMELandromat                        -22.3911 19246.8495  -0.001
## CONTEXT_FRAMEMenu                              -5.0706 51897.1058   0.000
## CONTEXT_FRAMEMovie_Theater                    -21.4862 19246.8495  -0.001
## CONTEXT_FRAMENightclub                        -22.0964 19246.8495  -0.001
## CONTEXT_FRAMENotary_Financial_Services        -23.2806 19246.8494  -0.001
## CONTEXT_FRAMEResidential                      -45.3584 32847.9910  -0.001
## CONTEXT_FRAMERestaurant                       -23.8055 19246.8494  -0.001
## CONTEXT_FRAMEShop                             -22.5694 19246.8494  -0.001
## CONTEXT_FRAMESpecialty_Foods                  -22.0218 19246.8495  -0.001
## CONTEXT_FRAMESupermarket                      -23.3643 19246.8494  -0.001
## CONTEXT_FRAMETravel_Agency                    -47.3202 28896.6233  -0.002
## CLOSEDFALSE                                   -21.1499 48196.0008   0.000
## CLOSEDTRUE                                    -21.8697 48196.0008   0.000
##                                             Pr(&amp;gt;|z|)    
## (Intercept)                                  0.99932    
## YELP                                        1.44e-07 ***
## COMMUNICATIVE_ROLEEstablishment_Description  0.02174 *  
## COMMUNICATIVE_ROLEEstablishment_Name         0.04206 *  
## COMMUNICATIVE_ROLEGraffiti                   0.82698    
## COMMUNICATIVE_ROLEinformation                0.99925    
## COMMUNICATIVE_ROLEInformation                0.00239 ** 
## COMMUNICATIVE_ROLEInstructions               0.78358    
## COMMUNICATIVE_ROLELeaflet                    0.42607    
## COMMUNICATIVE_ROLESlogan                     0.46189    
## COMMUNICATIVE_ROLEStreet_Signs               0.58216    
## COMMUNICATIVE_ROLETrademark                  0.99970    
## MATERIALITYHand_Written                      0.43208    
## MATERIALITYHome_Printed                      0.50631    
## MATERIALITYPermanent                         0.09483 .  
## MATERIALITYProfessionally_Printed            0.08661 .  
## CONTEXT_FRAMEAuto_Mechanic                   0.99999    
## CONTEXT_FRAMEBakery                          0.99897    
## CONTEXT_FRAMEBar                             0.99907    
## CONTEXT_FRAMEBeauty_Hair_Salon               0.99903    
## CONTEXT_FRAMEBusiness                        0.99907    
## CONTEXT_FRAMECafe                            0.99994    
## CONTEXT_FRAMEClothing                        0.99904    
## CONTEXT_FRAMECommentary                      0.99908    
## CONTEXT_FRAMEExternal                        0.99906    
## CONTEXT_FRAMEFlier                           0.99907    
## CONTEXT_FRAMEGallery_Museum                  0.99900    
## CONTEXT_FRAMEGrocery                         0.99993    
## CONTEXT_FRAMEGrocery_Liquor_Store            0.99902    
## CONTEXT_FRAMEGym_Fitness_Studio              1.00000    
## CONTEXT_FRAMEHardware                        0.99995    
## CONTEXT_FRAMEHotel                           0.99906    
## CONTEXT_FRAMEInstitution                     0.99904    
## CONTEXT_FRAMEJewelry_Store                   0.99907    
## CONTEXT_FRAMELandromat                       0.99907    
## CONTEXT_FRAMEMenu                            0.99992    
## CONTEXT_FRAMEMovie_Theater                   0.99911    
## CONTEXT_FRAMENightclub                       0.99908    
## CONTEXT_FRAMENotary_Financial_Services       0.99903    
## CONTEXT_FRAMEResidential                     0.99890    
## CONTEXT_FRAMERestaurant                      0.99901    
## CONTEXT_FRAMEShop                            0.99906    
## CONTEXT_FRAMESpecialty_Foods                 0.99909    
## CONTEXT_FRAMESupermarket                     0.99903    
## CONTEXT_FRAMETravel_Agency                   0.99869    
## CLOSEDFALSE                                  0.99965    
## CLOSEDTRUE                                   0.99964    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq  p-value    
## s(latitude,longitude) 21.46  28.48  64.21 0.000144 ***
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.266   Deviance explained =   28%
## UBRE = 0.070317  Scale est. = 1         n = 1032&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# To get odds ratios (commented out for clarity)
# exp(coef(gamE))

# Spanish
summary(gamSLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Spanish&amp;quot;) ~ s(latitude, longitude, k = 60) + YELP + 
##     COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -5.187e+01  6.352e+05   0.000
## YELP                                        -6.091e-01  1.745e-01  -3.490
## COMMUNICATIVE_ROLEEstablishment_Description  5.948e-01  5.321e-01   1.118
## COMMUNICATIVE_ROLEEstablishment_Name         5.442e-01  5.122e-01   1.063
## COMMUNICATIVE_ROLEGraffiti                  -2.188e+00  1.106e+00  -1.978
## COMMUNICATIVE_ROLEinformation               -1.127e-01  1.416e+00  -0.080
## COMMUNICATIVE_ROLEInformation                3.734e-01  5.304e-01   0.704
## COMMUNICATIVE_ROLEInstructions               3.725e-01  1.602e+00   0.233
## COMMUNICATIVE_ROLELeaflet                   -1.945e-01  1.445e+00  -0.135
## COMMUNICATIVE_ROLESlogan                     2.172e-01  1.458e+00   0.149
## COMMUNICATIVE_ROLEStreet_Signs              -8.371e-01  1.373e+00  -0.610
## COMMUNICATIVE_ROLETrademark                 -2.306e+01  5.871e+05   0.000
## MATERIALITYHand_Written                     -9.566e-01  9.062e-01  -1.056
## MATERIALITYHome_Printed                     -1.217e+00  9.193e-01  -1.324
## MATERIALITYPermanent                        -2.994e+00  1.254e+00  -2.388
## MATERIALITYProfessionally_Printed           -2.166e+00  8.988e-01  -2.410
## CONTEXT_FRAMEAuto_Mechanic                   9.136e-01  6.351e+05   0.000
## CONTEXT_FRAMEBakery                          2.950e+01  2.420e+05   0.000
## CONTEXT_FRAMEBar                             2.672e+01  2.420e+05   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon               2.752e+01  2.420e+05   0.000
## CONTEXT_FRAMEBusiness                        2.676e+01  2.420e+05   0.000
## CONTEXT_FRAMECafe                            1.552e+00  2.580e+05   0.000
## CONTEXT_FRAMEClothing                        2.849e+01  2.420e+05   0.000
## CONTEXT_FRAMECommentary                      2.818e+01  2.420e+05   0.000
## CONTEXT_FRAMEExternal                        2.729e+01  2.420e+05   0.000
## CONTEXT_FRAMEFlier                           5.829e+01  4.805e+05   0.000
## CONTEXT_FRAMEGallery_Museum                  2.657e+01  2.420e+05   0.000
## CONTEXT_FRAMEGrocery                         3.085e+00  3.651e+05   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store            2.732e+01  2.420e+05   0.000
## CONTEXT_FRAMEGym_Fitness_Studio             -6.323e-02  3.655e+05   0.000
## CONTEXT_FRAMEHardware                        1.891e+00  3.722e+05   0.000
## CONTEXT_FRAMEHotel                           1.565e+00  2.992e+05   0.000
## CONTEXT_FRAMEInstitution                     2.809e+01  2.420e+05   0.000
## CONTEXT_FRAMEJewelry_Store                   2.580e+01  2.420e+05   0.000
## CONTEXT_FRAMELandromat                       2.648e+01  2.420e+05   0.000
## CONTEXT_FRAMEMenu                            6.346e+00  6.351e+05   0.000
## CONTEXT_FRAMEMovie_Theater                   2.658e+01  2.420e+05   0.000
## CONTEXT_FRAMENightclub                       1.394e+00  3.179e+05   0.000
## CONTEXT_FRAMENotary_Financial_Services       2.759e+01  2.420e+05   0.000
## CONTEXT_FRAMEResidential                     5.575e+01  3.985e+05   0.000
## CONTEXT_FRAMERestaurant                      2.805e+01  2.420e+05   0.000
## CONTEXT_FRAMEShop                            2.635e+01  2.420e+05   0.000
## CONTEXT_FRAMESpecialty_Foods                 2.655e+01  2.420e+05   0.000
## CONTEXT_FRAMESupermarket                     2.723e+01  2.420e+05   0.000
## CONTEXT_FRAMETravel_Agency                   3.183e+01  2.420e+05   0.000
## CLOSEDFALSE                                  2.481e+01  5.873e+05   0.000
## CLOSEDTRUE                                   2.529e+01  5.873e+05   0.000
##                                             Pr(&amp;gt;|z|)    
## (Intercept)                                 0.999935    
## YELP                                        0.000482 ***
## COMMUNICATIVE_ROLEEstablishment_Description 0.263683    
## COMMUNICATIVE_ROLEEstablishment_Name        0.288004    
## COMMUNICATIVE_ROLEGraffiti                  0.047951 *  
## COMMUNICATIVE_ROLEinformation               0.936542    
## COMMUNICATIVE_ROLEInformation               0.481422    
## COMMUNICATIVE_ROLEInstructions              0.816079    
## COMMUNICATIVE_ROLELeaflet                   0.892933    
## COMMUNICATIVE_ROLESlogan                    0.881545    
## COMMUNICATIVE_ROLEStreet_Signs              0.541943    
## COMMUNICATIVE_ROLETrademark                 0.999969    
## MATERIALITYHand_Written                     0.291186    
## MATERIALITYHome_Printed                     0.185457    
## MATERIALITYPermanent                        0.016922 *  
## MATERIALITYProfessionally_Printed           0.015944 *  
## CONTEXT_FRAMEAuto_Mechanic                  0.999999    
## CONTEXT_FRAMEBakery                         0.999903    
## CONTEXT_FRAMEBar                            0.999912    
## CONTEXT_FRAMEBeauty_Hair_Salon              0.999909    
## CONTEXT_FRAMEBusiness                       0.999912    
## CONTEXT_FRAMECafe                           0.999995    
## CONTEXT_FRAMEClothing                       0.999906    
## CONTEXT_FRAMECommentary                     0.999907    
## CONTEXT_FRAMEExternal                       0.999910    
## CONTEXT_FRAMEFlier                          0.999903    
## CONTEXT_FRAMEGallery_Museum                 0.999912    
## CONTEXT_FRAMEGrocery                        0.999993    
## CONTEXT_FRAMEGrocery_Liquor_Store           0.999910    
## CONTEXT_FRAMEGym_Fitness_Studio             1.000000    
## CONTEXT_FRAMEHardware                       0.999996    
## CONTEXT_FRAMEHotel                          0.999996    
## CONTEXT_FRAMEInstitution                    0.999907    
## CONTEXT_FRAMEJewelry_Store                  0.999915    
## CONTEXT_FRAMELandromat                      0.999913    
## CONTEXT_FRAMEMenu                           0.999992    
## CONTEXT_FRAMEMovie_Theater                  0.999912    
## CONTEXT_FRAMENightclub                      0.999997    
## CONTEXT_FRAMENotary_Financial_Services      0.999909    
## CONTEXT_FRAMEResidential                    0.999888    
## CONTEXT_FRAMERestaurant                     0.999908    
## CONTEXT_FRAMEShop                           0.999913    
## CONTEXT_FRAMESpecialty_Foods                0.999912    
## CONTEXT_FRAMESupermarket                    0.999910    
## CONTEXT_FRAMETravel_Agency                  0.999895    
## CLOSEDFALSE                                 0.999966    
## CLOSEDTRUE                                  0.999966    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value   
## s(latitude,longitude) 35.46   43.9  72.91 0.00415 **
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.243   Deviance explained = 29.5%
## UBRE = -0.10451  Scale est. = 1         n = 1032&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamES))

# Mostly English with Some Spanish
# Spanish
summary(gamESLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Eng_Span&amp;quot;) ~ s(latitude, longitude, k = 60) + 
##     YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + 
##     CLOSED
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -7.440e+01  4.502e+05   0.000
## YELP                                         4.208e-03  2.567e-01   0.016
## COMMUNICATIVE_ROLEEstablishment_Description  9.883e-01  1.150e+00   0.860
## COMMUNICATIVE_ROLEEstablishment_Name         1.158e+00  1.112e+00   1.041
## COMMUNICATIVE_ROLEGraffiti                   2.157e+00  1.442e+00   1.496
## COMMUNICATIVE_ROLEinformation               -2.275e+01  2.287e+05   0.000
## COMMUNICATIVE_ROLEInformation                1.488e+00  1.131e+00   1.316
## COMMUNICATIVE_ROLEInstructions              -2.221e+01  1.465e+05   0.000
## COMMUNICATIVE_ROLELeaflet                   -1.970e+01  1.068e+05   0.000
## COMMUNICATIVE_ROLESlogan                     3.117e+00  1.712e+00   1.821
## COMMUNICATIVE_ROLEStreet_Signs              -2.277e+01  7.992e+04   0.000
## COMMUNICATIVE_ROLETrademark                 -2.167e+01  4.078e+05   0.000
## MATERIALITYHand_Written                      2.143e+01  8.507e+04   0.000
## MATERIALITYHome_Printed                      2.057e+01  8.507e+04   0.000
## MATERIALITYPermanent                         2.155e+01  8.507e+04   0.000
## MATERIALITYProfessionally_Printed            2.131e+01  8.507e+04   0.000
## CONTEXT_FRAMEAuto_Mechanic                   1.795e+00  4.421e+05   0.000
## CONTEXT_FRAMEBakery                          2.402e+01  1.707e+05   0.000
## CONTEXT_FRAMEBar                             3.610e-01  1.884e+05   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon               2.429e+01  1.707e+05   0.000
## CONTEXT_FRAMEBusiness                        2.354e+01  1.707e+05   0.000
## CONTEXT_FRAMECafe                            5.885e-01  1.834e+05   0.000
## CONTEXT_FRAMEClothing                        6.496e-02  1.935e+05   0.000
## CONTEXT_FRAMECommentary                      2.744e-01  1.982e+05   0.000
## CONTEXT_FRAMEExternal                        2.429e+01  1.707e+05   0.000
## CONTEXT_FRAMEFlier                           2.057e+01  3.517e+05   0.000
## CONTEXT_FRAMEGallery_Museum                  2.447e+01  1.707e+05   0.000
## CONTEXT_FRAMEGrocery                         6.510e-01  2.611e+05   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store            2.496e+01  1.707e+05   0.000
## CONTEXT_FRAMEGym_Fitness_Studio              7.352e-01  2.615e+05   0.000
## CONTEXT_FRAMEHardware                        6.482e-01  2.636e+05   0.000
## CONTEXT_FRAMEHotel                           2.572e+01  1.707e+05   0.000
## CONTEXT_FRAMEInstitution                     1.031e+00  1.925e+05   0.000
## CONTEXT_FRAMEJewelry_Store                   2.431e+01  1.707e+05   0.000
## CONTEXT_FRAMELandromat                       2.533e+01  1.707e+05   0.000
## CONTEXT_FRAMEMenu                            1.820e+00  4.421e+05   0.000
## CONTEXT_FRAMEMovie_Theater                   1.285e-01  2.191e+05   0.000
## CONTEXT_FRAMENightclub                       2.516e+01  1.707e+05   0.000
## CONTEXT_FRAMENotary_Financial_Services       5.360e-01  1.964e+05   0.000
## CONTEXT_FRAMEResidential                     8.533e-01  2.642e+05   0.000
## CONTEXT_FRAMERestaurant                      2.431e+01  1.707e+05   0.000
## CONTEXT_FRAMEShop                            2.440e+01  1.707e+05   0.000
## CONTEXT_FRAMESpecialty_Foods                 9.096e-01  2.038e+05   0.000
## CONTEXT_FRAMESupermarket                     2.522e+01  1.707e+05   0.000
## CONTEXT_FRAMETravel_Agency                   1.447e+00  2.498e+05   0.000
## CLOSEDFALSE                                  2.440e+01  4.078e+05   0.000
## CLOSEDTRUE                                   2.556e+01  4.078e+05   0.000
##                                             Pr(&amp;gt;|z|)  
## (Intercept)                                   0.9999  
## YELP                                          0.9869  
## COMMUNICATIVE_ROLEEstablishment_Description   0.3900  
## COMMUNICATIVE_ROLEEstablishment_Name          0.2980  
## COMMUNICATIVE_ROLEGraffiti                    0.1348  
## COMMUNICATIVE_ROLEinformation                 0.9999  
## COMMUNICATIVE_ROLEInformation                 0.1881  
## COMMUNICATIVE_ROLEInstructions                0.9999  
## COMMUNICATIVE_ROLELeaflet                     0.9999  
## COMMUNICATIVE_ROLESlogan                      0.0686 .
## COMMUNICATIVE_ROLEStreet_Signs                0.9998  
## COMMUNICATIVE_ROLETrademark                   1.0000  
## MATERIALITYHand_Written                       0.9998  
## MATERIALITYHome_Printed                       0.9998  
## MATERIALITYPermanent                          0.9998  
## MATERIALITYProfessionally_Printed             0.9998  
## CONTEXT_FRAMEAuto_Mechanic                    1.0000  
## CONTEXT_FRAMEBakery                           0.9999  
## CONTEXT_FRAMEBar                              1.0000  
## CONTEXT_FRAMEBeauty_Hair_Salon                0.9999  
## CONTEXT_FRAMEBusiness                         0.9999  
## CONTEXT_FRAMECafe                             1.0000  
## CONTEXT_FRAMEClothing                         1.0000  
## CONTEXT_FRAMECommentary                       1.0000  
## CONTEXT_FRAMEExternal                         0.9999  
## CONTEXT_FRAMEFlier                            1.0000  
## CONTEXT_FRAMEGallery_Museum                   0.9999  
## CONTEXT_FRAMEGrocery                          1.0000  
## CONTEXT_FRAMEGrocery_Liquor_Store             0.9999  
## CONTEXT_FRAMEGym_Fitness_Studio               1.0000  
## CONTEXT_FRAMEHardware                         1.0000  
## CONTEXT_FRAMEHotel                            0.9999  
## CONTEXT_FRAMEInstitution                      1.0000  
## CONTEXT_FRAMEJewelry_Store                    0.9999  
## CONTEXT_FRAMELandromat                        0.9999  
## CONTEXT_FRAMEMenu                             1.0000  
## CONTEXT_FRAMEMovie_Theater                    1.0000  
## CONTEXT_FRAMENightclub                        0.9999  
## CONTEXT_FRAMENotary_Financial_Services        1.0000  
## CONTEXT_FRAMEResidential                      1.0000  
## CONTEXT_FRAMERestaurant                       0.9999  
## CONTEXT_FRAMEShop                             0.9999  
## CONTEXT_FRAMESpecialty_Foods                  1.0000  
## CONTEXT_FRAMESupermarket                      0.9999  
## CONTEXT_FRAMETravel_Agency                    1.0000  
## CLOSEDFALSE                                   1.0000  
## CLOSEDTRUE                                    0.9999  
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value
## s(latitude,longitude) 18.71  25.06  23.19   0.568
## 
## R-sq.(adj) =  0.041   Deviance explained = 20.5%
## UBRE = -0.57808  Scale est. = 1         n = 1032&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamES))

# Mostly Spanish with some English
summary(gamSELL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Span_Eng&amp;quot;) ~ s(latitude, longitude, k = 60) + 
##     YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + 
##     CLOSED
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -1.219e+02  3.702e+07   0.000
## YELP                                        -9.442e-01  2.748e-01  -3.436
## COMMUNICATIVE_ROLEEstablishment_Description  1.607e+00  1.093e+00   1.470
## COMMUNICATIVE_ROLEEstablishment_Name         1.024e+00  1.079e+00   0.949
## COMMUNICATIVE_ROLEGraffiti                  -2.759e+01  1.917e+07   0.000
## COMMUNICATIVE_ROLEinformation                2.203e+00  1.728e+00   1.274
## COMMUNICATIVE_ROLEInformation                1.501e+00  1.109e+00   1.354
## COMMUNICATIVE_ROLEInstructions              -9.506e-01  2.834e+07   0.000
## COMMUNICATIVE_ROLELeaflet                   -2.553e+01  3.663e+07   0.000
## COMMUNICATIVE_ROLESlogan                    -3.002e+01  2.740e+07   0.000
## COMMUNICATIVE_ROLEStreet_Signs              -2.743e+01  3.036e+06   0.000
## COMMUNICATIVE_ROLETrademark                 -2.870e+01  6.711e+07   0.000
## MATERIALITYHand_Written                      3.043e+01  2.167e+07   0.000
## MATERIALITYHome_Printed                      3.140e+01  2.167e+07   0.000
## MATERIALITYPermanent                         3.166e+01  2.167e+07   0.000
## MATERIALITYProfessionally_Printed            3.120e+01  2.167e+07   0.000
## CONTEXT_FRAMEAuto_Mechanic                   4.002e+01  7.351e+07   0.000
## CONTEXT_FRAMEBakery                          7.124e+01  3.001e+07   0.000
## CONTEXT_FRAMEBar                             7.258e+01  3.001e+07   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon               7.214e+01  3.001e+07   0.000
## CONTEXT_FRAMEBusiness                        7.179e+01  3.001e+07   0.000
## CONTEXT_FRAMECafe                            4.060e+01  3.221e+07   0.000
## CONTEXT_FRAMEClothing                        7.157e+01  3.001e+07   0.000
## CONTEXT_FRAMECommentary                      4.359e+01  4.121e+07   0.000
## CONTEXT_FRAMEExternal                        4.199e+01  3.107e+07   0.000
## CONTEXT_FRAMEFlier                           6.552e+01  6.704e+07   0.000
## CONTEXT_FRAMEGallery_Museum                  7.347e+01  3.001e+07   0.000
## CONTEXT_FRAMEGrocery                         4.239e+01  4.502e+07   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store            7.224e+01  3.001e+07   0.000
## CONTEXT_FRAMEGym_Fitness_Studio              3.940e+01  4.502e+07   0.000
## CONTEXT_FRAMEHardware                        4.143e+01  4.502e+07   0.000
## CONTEXT_FRAMEHotel                           4.115e+01  3.676e+07   0.000
## CONTEXT_FRAMEInstitution                     3.949e+01  3.373e+07   0.000
## CONTEXT_FRAMEJewelry_Store                   7.331e+01  3.001e+07   0.000
## CONTEXT_FRAMELandromat                       7.152e+01  3.001e+07   0.000
## CONTEXT_FRAMEMenu                            4.482e+01  7.351e+07   0.000
## CONTEXT_FRAMEMovie_Theater                   4.067e+01  3.826e+07   0.000
## CONTEXT_FRAMENightclub                       4.036e+01  3.929e+07   0.000
## CONTEXT_FRAMENotary_Financial_Services       7.042e+01  3.001e+07   0.000
## CONTEXT_FRAMEResidential                     4.161e+01  5.024e+07   0.000
## CONTEXT_FRAMERestaurant                      7.267e+01  3.001e+07   0.000
## CONTEXT_FRAMEShop                            7.233e+01  3.001e+07   0.000
## CONTEXT_FRAMESpecialty_Foods                 4.140e+01  3.580e+07   0.000
## CONTEXT_FRAMESupermarket                     7.308e+01  3.001e+07   0.000
## CONTEXT_FRAMETravel_Agency                   7.418e+01  3.001e+07   0.000
## CLOSEDFALSE                                  1.528e+01  6.242e+03   0.002
## CLOSEDTRUE                                   1.474e+01  6.242e+03   0.002
##                                             Pr(&amp;gt;|z|)    
## (Intercept)                                  1.00000    
## YELP                                         0.00059 ***
## COMMUNICATIVE_ROLEEstablishment_Description  0.14161    
## COMMUNICATIVE_ROLEEstablishment_Name         0.34273    
## COMMUNICATIVE_ROLEGraffiti                   1.00000    
## COMMUNICATIVE_ROLEinformation                0.20249    
## COMMUNICATIVE_ROLEInformation                0.17569    
## COMMUNICATIVE_ROLEInstructions               1.00000    
## COMMUNICATIVE_ROLELeaflet                    1.00000    
## COMMUNICATIVE_ROLESlogan                     1.00000    
## COMMUNICATIVE_ROLEStreet_Signs               0.99999    
## COMMUNICATIVE_ROLETrademark                  1.00000    
## MATERIALITYHand_Written                      1.00000    
## MATERIALITYHome_Printed                      1.00000    
## MATERIALITYPermanent                         1.00000    
## MATERIALITYProfessionally_Printed            1.00000    
## CONTEXT_FRAMEAuto_Mechanic                   1.00000    
## CONTEXT_FRAMEBakery                          1.00000    
## CONTEXT_FRAMEBar                             1.00000    
## CONTEXT_FRAMEBeauty_Hair_Salon               1.00000    
## CONTEXT_FRAMEBusiness                        1.00000    
## CONTEXT_FRAMECafe                            1.00000    
## CONTEXT_FRAMEClothing                        1.00000    
## CONTEXT_FRAMECommentary                      1.00000    
## CONTEXT_FRAMEExternal                        1.00000    
## CONTEXT_FRAMEFlier                           1.00000    
## CONTEXT_FRAMEGallery_Museum                  1.00000    
## CONTEXT_FRAMEGrocery                         1.00000    
## CONTEXT_FRAMEGrocery_Liquor_Store            1.00000    
## CONTEXT_FRAMEGym_Fitness_Studio              1.00000    
## CONTEXT_FRAMEHardware                        1.00000    
## CONTEXT_FRAMEHotel                           1.00000    
## CONTEXT_FRAMEInstitution                     1.00000    
## CONTEXT_FRAMEJewelry_Store                   1.00000    
## CONTEXT_FRAMELandromat                       1.00000    
## CONTEXT_FRAMEMenu                            1.00000    
## CONTEXT_FRAMEMovie_Theater                   1.00000    
## CONTEXT_FRAMENightclub                       1.00000    
## CONTEXT_FRAMENotary_Financial_Services       1.00000    
## CONTEXT_FRAMEResidential                     1.00000    
## CONTEXT_FRAMERestaurant                      1.00000    
## CONTEXT_FRAMEShop                            1.00000    
## CONTEXT_FRAMESpecialty_Foods                 1.00000    
## CONTEXT_FRAMESupermarket                     1.00000    
## CONTEXT_FRAMETravel_Agency                   1.00000    
## CLOSEDFALSE                                  0.99805    
## CLOSEDTRUE                                   0.99812    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value  
## s(latitude,longitude) 6.546  9.113  15.16  0.0895 .
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.0607   Deviance explained = 19.7%
## UBRE = -0.5312  Scale est. = 1         n = 1032&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamSE))

# Equal
summary(gamEQLL)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Equal&amp;quot;) ~ s(latitude, longitude, k = 60) + YELP + 
##     COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -8.083e+01  2.224e+06   0.000
## YELP                                        -1.041e+00  4.483e-01  -2.322
## COMMUNICATIVE_ROLEEstablishment_Description -3.665e-01  1.036e+00  -0.354
## COMMUNICATIVE_ROLEEstablishment_Name        -1.530e-01  9.492e-01  -0.161
## COMMUNICATIVE_ROLEGraffiti                   1.354e+00  1.562e+00   0.867
## COMMUNICATIVE_ROLEinformation                3.844e+00  1.737e+00   2.212
## COMMUNICATIVE_ROLEInformation                1.634e+00  9.271e-01   1.762
## COMMUNICATIVE_ROLEInstructions              -2.572e+01  6.156e+05   0.000
## COMMUNICATIVE_ROLELeaflet                   -2.445e+01  4.656e+05   0.000
## COMMUNICATIVE_ROLESlogan                    -2.640e+01  6.700e+05   0.000
## COMMUNICATIVE_ROLEStreet_Signs               2.472e+00  2.320e+00   1.066
## COMMUNICATIVE_ROLETrademark                 -2.287e+01  2.033e+06   0.000
## MATERIALITYHand_Written                      2.674e+01  4.162e+05   0.000
## MATERIALITYHome_Printed                      2.704e+01  4.162e+05   0.000
## MATERIALITYPermanent                         2.494e+01  4.162e+05   0.000
## MATERIALITYProfessionally_Printed            2.639e+01  4.162e+05   0.000
## CONTEXT_FRAMEAuto_Mechanic                  -2.323e+00  2.184e+06   0.000
## CONTEXT_FRAMEBakery                          2.667e+01  7.987e+05   0.000
## CONTEXT_FRAMEBar                             2.817e-01  8.847e+05   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon              -6.883e-01  8.407e+05   0.000
## CONTEXT_FRAMEBusiness                        2.493e+01  7.987e+05   0.000
## CONTEXT_FRAMECafe                           -2.876e-01  8.560e+05   0.000
## CONTEXT_FRAMEClothing                       -1.417e+00  8.909e+05   0.000
## CONTEXT_FRAMECommentary                     -1.986e+00  9.232e+05   0.000
## CONTEXT_FRAMEExternal                        2.376e+01  7.987e+05   0.000
## CONTEXT_FRAMEFlier                           2.326e+01  1.709e+06   0.000
## CONTEXT_FRAMEGallery_Museum                  2.653e+01  7.987e+05   0.000
## CONTEXT_FRAMEGrocery                         8.718e-01  1.289e+06   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store            2.614e+01  7.987e+05   0.000
## CONTEXT_FRAMEGym_Fitness_Studio             -1.957e+00  1.271e+06   0.000
## CONTEXT_FRAMEHardware                       -3.103e-01  1.247e+06   0.000
## CONTEXT_FRAMEHotel                           2.717e+01  7.987e+05   0.000
## CONTEXT_FRAMEInstitution                     2.406e+01  7.987e+05   0.000
## CONTEXT_FRAMEJewelry_Store                  -1.158e+00  9.500e+05   0.000
## CONTEXT_FRAMELandromat                      -8.332e-01  9.580e+05   0.000
## CONTEXT_FRAMEMenu                            3.488e-01  2.184e+06   0.000
## CONTEXT_FRAMEMovie_Theater                  -1.482e+00  1.035e+06   0.000
## CONTEXT_FRAMENightclub                      -2.489e-01  1.057e+06   0.000
## CONTEXT_FRAMENotary_Financial_Services       2.650e+01  7.987e+05   0.000
## CONTEXT_FRAMEResidential                    -1.179e+00  1.269e+06   0.000
## CONTEXT_FRAMERestaurant                      2.383e+01  7.987e+05   0.000
## CONTEXT_FRAMEShop                            2.496e+01  7.987e+05   0.000
## CONTEXT_FRAMESpecialty_Foods                -1.884e-01  9.262e+05   0.000
## CONTEXT_FRAMESupermarket                     2.449e+01  7.987e+05   0.000
## CONTEXT_FRAMETravel_Agency                  -1.314e+00  1.210e+06   0.000
## CLOSEDFALSE                                  2.602e+01  2.033e+06   0.000
## CLOSEDTRUE                                   1.650e+00  2.046e+06   0.000
##                                             Pr(&amp;gt;|z|)  
## (Intercept)                                   1.0000  
## YELP                                          0.0202 *
## COMMUNICATIVE_ROLEEstablishment_Description   0.7235  
## COMMUNICATIVE_ROLEEstablishment_Name          0.8720  
## COMMUNICATIVE_ROLEGraffiti                    0.3862  
## COMMUNICATIVE_ROLEinformation                 0.0269 *
## COMMUNICATIVE_ROLEInformation                 0.0781 .
## COMMUNICATIVE_ROLEInstructions                1.0000  
## COMMUNICATIVE_ROLELeaflet                     1.0000  
## COMMUNICATIVE_ROLESlogan                      1.0000  
## COMMUNICATIVE_ROLEStreet_Signs                0.2866  
## COMMUNICATIVE_ROLETrademark                   1.0000  
## MATERIALITYHand_Written                       0.9999  
## MATERIALITYHome_Printed                       0.9999  
## MATERIALITYPermanent                          1.0000  
## MATERIALITYProfessionally_Printed             0.9999  
## CONTEXT_FRAMEAuto_Mechanic                    1.0000  
## CONTEXT_FRAMEBakery                           1.0000  
## CONTEXT_FRAMEBar                              1.0000  
## CONTEXT_FRAMEBeauty_Hair_Salon                1.0000  
## CONTEXT_FRAMEBusiness                         1.0000  
## CONTEXT_FRAMECafe                             1.0000  
## CONTEXT_FRAMEClothing                         1.0000  
## CONTEXT_FRAMECommentary                       1.0000  
## CONTEXT_FRAMEExternal                         1.0000  
## CONTEXT_FRAMEFlier                            1.0000  
## CONTEXT_FRAMEGallery_Museum                   1.0000  
## CONTEXT_FRAMEGrocery                          1.0000  
## CONTEXT_FRAMEGrocery_Liquor_Store             1.0000  
## CONTEXT_FRAMEGym_Fitness_Studio               1.0000  
## CONTEXT_FRAMEHardware                         1.0000  
## CONTEXT_FRAMEHotel                            1.0000  
## CONTEXT_FRAMEInstitution                      1.0000  
## CONTEXT_FRAMEJewelry_Store                    1.0000  
## CONTEXT_FRAMELandromat                        1.0000  
## CONTEXT_FRAMEMenu                             1.0000  
## CONTEXT_FRAMEMovie_Theater                    1.0000  
## CONTEXT_FRAMENightclub                        1.0000  
## CONTEXT_FRAMENotary_Financial_Services        1.0000  
## CONTEXT_FRAMEResidential                      1.0000  
## CONTEXT_FRAMERestaurant                       1.0000  
## CONTEXT_FRAMEShop                             1.0000  
## CONTEXT_FRAMESpecialty_Foods                  1.0000  
## CONTEXT_FRAMESupermarket                      1.0000  
## CONTEXT_FRAMETravel_Agency                    1.0000  
## CLOSEDFALSE                                   1.0000  
## CLOSEDTRUE                                    1.0000  
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                        edf Ref.df Chi.sq p-value
## s(latitude,longitude) 7.11  9.628  15.22   0.102
## 
## R-sq.(adj) =  0.144   Deviance explained = 31.5%
## UBRE = -0.71511  Scale est. = 1         n = 1032&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamEQ))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We can see from these results a lot of information – what is significant, deviance explained, coefficients, etc. But it is also useful to plot probabilities.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Plot probabilities? (Adapted from http://myweb.uiowa.edu/pbreheny/publications/visreg.pdf)
library(visreg)
# We will just look at those flagged as &amp;#39;significant&amp;#39;
# Probability of English by coordinate
visreg2d(gamELL, &amp;quot;longitude&amp;quot;, &amp;quot;latitude&amp;quot;, plot.type=&amp;quot;image&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-50-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Spanish
visreg2d(gamSLL, &amp;quot;longitude&amp;quot;, &amp;quot;latitude&amp;quot;, plot.type=&amp;quot;image&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-50-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;combining-ll-and-instagram-data&#34; class=&#34;section level3&#34;&gt;
&lt;h3&gt;Combining LL and Instagram Data&lt;/h3&gt;
&lt;p&gt;Remember to remind R that your ‘Mode’ is actually a category, not a continuous variable.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;topicsigns$YELP &amp;lt;- as.factor(topicsigns$YELP)
topicsigns$Topic &amp;lt;- as.factor(topicsigns$Topic)

# Subset to get rid of Trademark with has no observations
topicsigns&amp;lt;-subset(topicsigns, COMMUNICATIVE_ROLE==&amp;quot;Establishment_Name&amp;quot; | COMMUNICATIVE_ROLE ==&amp;quot;Establishment_Description&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Graffiti&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Advertisement&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Information&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Instructions&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Leaflet&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Slogan&amp;quot;| COMMUNICATIVE_ROLE==&amp;quot;Street_Signs&amp;quot;)

# On to the first GAM!
# We have adjusted k to 40.
# English
gamE= gam(I(LANGUAGE==&amp;quot;English&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + Topic,family=binomial, data=topicsigns)

# Spanish
gamS= gam(I(LANGUAGE==&amp;quot;Spanish&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + Topic,family=binomial, data=topicsigns)

# Mostly English with some Spanish
gamES = gam(I(LANGUAGE==&amp;quot;Eng_Span&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + Topic,family=binomial, data=topicsigns)

# Mostly Spanish with some English
gamSE = gam(I(LANGUAGE==&amp;quot;Span_Eng&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + Topic,family=binomial, data=topicsigns)

# Equal
gamEQ = gam(I(LANGUAGE==&amp;quot;Equal&amp;quot;)~s(latitude,longitude, k=60) + YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + Topic,family=binomial, data=topicsigns)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;model-checking&#34; class=&#34;section level3&#34;&gt;
&lt;h3&gt;Model checking&lt;/h3&gt;
&lt;p&gt;It’s also time to check for concurvity, to see if &lt;a href=&#34;https://stat.ethz.ch/R-manual/R-devel/library/mgcv/html/concurvity.html&#34;&gt;“a smooth term in [my] model could be approximated by one or more of the other smooth terms in the model”&lt;/a&gt;. I think I am potentially at risk for this perhaps as “this is often the case when a smooth of space is included in a model, along with smooths of other covariates that also vary more or less smoothly in space”.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9995429             0.8130711
## observed 0.9995429             0.4924878
## estimate 0.9995429             0.4007821&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamS)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9995429             0.8130711
## observed 0.9995429             0.4332782
## estimate 0.9995429             0.4007821&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamES)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9995429             0.8130711
## observed 0.9995429             0.3382393
## estimate 0.9995429             0.4007821&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamSE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9995429             0.8130711
## observed 0.9995429             0.5602935
## estimate 0.9995429             0.4007821&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;concurvity(gamEQ)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9995429             0.8130711
## observed 0.9995429             0.4685437
## estimate 0.9995429             0.4007821&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Concurvity measures suggest that my smooths are okay – they are all pretty far away from 1.&lt;/p&gt;
&lt;p&gt;Now onto gam.check to look at more diagnostics. There is an issue here where the k-index is less than 1 for these models, but this doesn’t get solved until k is up to around 300 or so, which would not be the best solution (would make the model prone to over-fitting!). So, while these suggest k is too low, I keep k as is to not over fit the model.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 3 iterations.
## Gradient range [2.738027e-09,2.738027e-09]
## (score 0.04053542 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.001719122,0.001719122].
## Model rank =  126 / 126 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000 20.107   0.938    0.02&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamS)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 9 iterations.
## Gradient range [2.463495e-07,2.463495e-07]
## (score -0.172996 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.002485206,0.002485206].
## Model rank =  126 / 126 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                          k&amp;#39;   edf k-index p-value
## s(latitude,longitude) 59.00 57.96    1.07    0.98&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamES)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 3 iterations.
## Gradient range [-5.25853e-09,-5.25853e-09]
## (score -0.5617174 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.002522951,0.002522951].
## Model rank =  126 / 126 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                          k&amp;#39;   edf k-index p-value
## s(latitude,longitude) 59.00 17.64    1.11       1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamSE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 7 iterations.
## Gradient range [-6.972236e-07,-6.972236e-07]
## (score -0.5066816 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [6.96943e-07,6.96943e-07].
## Model rank =  126 / 126 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                          k&amp;#39;   edf k-index p-value
## s(latitude,longitude) 59.00  2.00    1.03    0.84&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(gamEQ)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## step failed after 35 iterations.
## Gradient range [3.18731e-05,3.18731e-05]
## (score -0.7286916 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.001106209,0.001106209].
## Model rank =  126 / 126 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                          k&amp;#39;   edf k-index p-value
## s(latitude,longitude) 59.00 12.76    1.26       1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-53-1.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-53-2.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-53-3.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-53-4.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-53-5.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;/div&gt;
&lt;div id=&#34;results-1&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Results&lt;/h2&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# English
summary(gamE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;English&amp;quot;) ~ s(latitude, longitude, k = 60) + YELP + 
##     COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + 
##     Topic
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                  4.990e+01  5.036e+05   0.000
## YELP1                                        2.984e-01  3.220e-01   0.927
## YELP2                                        1.588e+00  4.304e-01   3.688
## YELP3                                        2.082e+00  1.254e+00   1.660
## YELP4                                        2.518e+01  2.451e+05   0.000
## COMMUNICATIVE_ROLEEstablishment_Description -1.026e+00  6.068e-01  -1.690
## COMMUNICATIVE_ROLEEstablishment_Name        -7.267e-01  5.730e-01  -1.268
## COMMUNICATIVE_ROLEGraffiti                   8.023e-01  1.078e+00   0.744
## COMMUNICATIVE_ROLEInformation               -8.694e-01  5.884e-01  -1.478
## COMMUNICATIVE_ROLEInstructions               2.497e+01  1.296e+05   0.000
## COMMUNICATIVE_ROLELeaflet                   -3.804e-01  1.913e+00  -0.199
## COMMUNICATIVE_ROLESlogan                    -1.360e+00  1.391e+00  -0.978
## COMMUNICATIVE_ROLEStreet_Signs               1.046e+00  1.450e+00   0.722
## MATERIALITYHand_Written                      6.629e-01  1.495e+00   0.443
## MATERIALITYHome_Printed                      7.129e-01  1.525e+00   0.468
## MATERIALITYPermanent                         2.042e+00  1.721e+00   1.186
## MATERIALITYProfessionally_Printed            1.567e+00  1.500e+00   1.044
## CONTEXT_FRAMEBakery                         -2.696e+01  3.561e+05   0.000
## CONTEXT_FRAMEBar                            -2.457e+01  3.561e+05   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon              -2.574e+01  3.561e+05   0.000
## CONTEXT_FRAMEBusiness                       -2.529e+01  3.561e+05   0.000
## CONTEXT_FRAMECafe                            1.666e-02  3.606e+05   0.000
## CONTEXT_FRAMEClothing                       -2.576e+01  3.561e+05   0.000
## CONTEXT_FRAMECommentary                     -2.541e+01  3.561e+05   0.000
## CONTEXT_FRAMEExternal                       -2.548e+01  3.561e+05   0.000
## CONTEXT_FRAMEFlier                          -5.344e+01  4.362e+05   0.000
## CONTEXT_FRAMEGallery_Museum                 -2.710e+01  3.561e+05   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store           -2.641e+01  3.561e+05   0.000
## CONTEXT_FRAMEGym_Fitness_Studio              4.657e-01  3.964e+05   0.000
## CONTEXT_FRAMEHardware                       -1.595e+00  4.191e+05   0.000
## CONTEXT_FRAMEHotel                          -2.606e+01  3.561e+05   0.000
## CONTEXT_FRAMEInstitution                    -2.557e+01  3.561e+05   0.000
## CONTEXT_FRAMEJewelry_Store                  -2.339e+01  3.561e+05   0.000
## CONTEXT_FRAMELandromat                      -4.914e-01  4.305e+05   0.000
## CONTEXT_FRAMEMenu                           -5.225e+00  5.036e+05   0.000
## CONTEXT_FRAMEMovie_Theater                  -2.367e+01  3.561e+05   0.000
## CONTEXT_FRAMENightclub                      -4.828e-01  4.089e+05   0.000
## CONTEXT_FRAMENotary_Financial_Services      -2.855e+01  3.561e+05   0.000
## CONTEXT_FRAMEResidential                    -7.777e+01  4.673e+05   0.000
## CONTEXT_FRAMERestaurant                     -2.664e+01  3.561e+05   0.000
## CONTEXT_FRAMEShop                           -2.552e+01  3.561e+05   0.000
## CONTEXT_FRAMESpecialty_Foods                -2.583e+01  3.561e+05   0.000
## CONTEXT_FRAMESupermarket                    -2.544e+01  3.561e+05   0.000
## CONTEXT_FRAMETravel_Agency                  -5.426e+01  3.901e+05   0.000
## CLOSEDFALSE                                 -2.463e+01  3.561e+05   0.000
## CLOSEDTRUE                                  -2.430e+01  3.561e+05   0.000
## Topic2                                      -1.673e-01  5.049e-01  -0.331
## Topic3                                       3.205e-01  6.109e-01   0.525
## Topic4                                       9.554e-01  4.934e-01   1.936
## Topic5                                       3.184e-01  5.598e-01   0.569
## Topic6                                       3.246e-01  7.153e-01   0.454
## Topic7                                       1.838e+00  1.023e+00   1.798
## Topic8                                       1.427e+00  8.291e-01   1.721
## Topic9                                       7.251e-01  5.262e-01   1.378
## Topic10                                      1.284e+00  6.455e-01   1.989
## Topic11                                      3.955e-01  1.101e+00   0.359
## Topic12                                     -9.880e-02  6.304e-01  -0.157
## Topic13                                      2.616e+01  1.375e+05   0.000
## Topic14                                      2.721e+01  1.291e+05   0.000
## Topic15                                     -1.580e+00  1.345e+00  -1.175
## Topic16                                      2.033e+00  9.328e-01   2.179
## Topic17                                      7.820e-01  7.100e-01   1.101
## Topic18                                     -1.331e-01  7.560e-01  -0.176
## Topic19                                     -4.293e-01  6.751e-01  -0.636
## Topic20                                      2.557e+01  1.684e+05   0.000
## Topic21                                     -1.319e+00  9.210e-01  -1.433
## Topic22                                      1.759e+00  1.237e+00   1.421
##                                             Pr(&amp;gt;|z|)    
## (Intercept)                                 0.999921    
## YELP1                                       0.353962    
## YELP2                                       0.000226 ***
## YELP3                                       0.096956 .  
## YELP4                                       0.999918    
## COMMUNICATIVE_ROLEEstablishment_Description 0.091033 .  
## COMMUNICATIVE_ROLEEstablishment_Name        0.204729    
## COMMUNICATIVE_ROLEGraffiti                  0.456689    
## COMMUNICATIVE_ROLEInformation               0.139481    
## COMMUNICATIVE_ROLEInstructions              0.999846    
## COMMUNICATIVE_ROLELeaflet                   0.842426    
## COMMUNICATIVE_ROLESlogan                    0.328013    
## COMMUNICATIVE_ROLEStreet_Signs              0.470596    
## MATERIALITYHand_Written                     0.657435    
## MATERIALITYHome_Printed                     0.640091    
## MATERIALITYPermanent                        0.235460    
## MATERIALITYProfessionally_Printed           0.296401    
## CONTEXT_FRAMEBakery                         0.999940    
## CONTEXT_FRAMEBar                            0.999945    
## CONTEXT_FRAMEBeauty_Hair_Salon              0.999942    
## CONTEXT_FRAMEBusiness                       0.999943    
## CONTEXT_FRAMECafe                           1.000000    
## CONTEXT_FRAMEClothing                       0.999942    
## CONTEXT_FRAMECommentary                     0.999943    
## CONTEXT_FRAMEExternal                       0.999943    
## CONTEXT_FRAMEFlier                          0.999902    
## CONTEXT_FRAMEGallery_Museum                 0.999939    
## CONTEXT_FRAMEGrocery_Liquor_Store           0.999941    
## CONTEXT_FRAMEGym_Fitness_Studio             0.999999    
## CONTEXT_FRAMEHardware                       0.999997    
## CONTEXT_FRAMEHotel                          0.999942    
## CONTEXT_FRAMEInstitution                    0.999943    
## CONTEXT_FRAMEJewelry_Store                  0.999948    
## CONTEXT_FRAMELandromat                      0.999999    
## CONTEXT_FRAMEMenu                           0.999992    
## CONTEXT_FRAMEMovie_Theater                  0.999947    
## CONTEXT_FRAMENightclub                      0.999999    
## CONTEXT_FRAMENotary_Financial_Services      0.999936    
## CONTEXT_FRAMEResidential                    0.999867    
## CONTEXT_FRAMERestaurant                     0.999940    
## CONTEXT_FRAMEShop                           0.999943    
## CONTEXT_FRAMESpecialty_Foods                0.999942    
## CONTEXT_FRAMESupermarket                    0.999943    
## CONTEXT_FRAMETravel_Agency                  0.999889    
## CLOSEDFALSE                                 0.999945    
## CLOSEDTRUE                                  0.999946    
## Topic2                                      0.740359    
## Topic3                                      0.599904    
## Topic4                                      0.052811 .  
## Topic5                                      0.569532    
## Topic6                                      0.649956    
## Topic7                                      0.072237 .  
## Topic8                                      0.085337 .  
## Topic9                                      0.168237    
## Topic10                                     0.046698 *  
## Topic11                                     0.719444    
## Topic12                                     0.875468    
## Topic13                                     0.999848    
## Topic14                                     0.999832    
## Topic15                                     0.240117    
## Topic16                                     0.029330 *  
## Topic17                                     0.270734    
## Topic18                                     0.860228    
## Topic19                                     0.524893    
## Topic20                                     0.999879    
## Topic21                                     0.151954    
## Topic22                                     0.155201    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value  
## s(latitude,longitude) 20.11  26.84  41.76  0.0355 *
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.335   Deviance explained = 38.2%
## UBRE = 0.040535  Scale est. = 1         n = 700&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# To get odds ratios (commented out for clarity)
# exp(coef(gamE))

# Spanish
summary(gamS)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Spanish&amp;quot;) ~ s(latitude, longitude, k = 60) + YELP + 
##     COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + 
##     Topic
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -6.405e+02  9.491e+07   0.000
## YELP1                                       -3.300e-01  5.326e-01  -0.620
## YELP2                                       -2.298e+00  8.522e-01  -2.696
## YELP3                                        5.881e-02  2.561e+00   0.023
## YELP4                                       -4.461e+01  4.745e+07   0.000
## COMMUNICATIVE_ROLEEstablishment_Description  2.528e+00  1.290e+00   1.959
## COMMUNICATIVE_ROLEEstablishment_Name         2.189e+00  1.264e+00   1.732
## COMMUNICATIVE_ROLEGraffiti                  -1.206e+00  1.800e+00  -0.670
## COMMUNICATIVE_ROLEInformation                1.447e+00  1.239e+00   1.168
## COMMUNICATIVE_ROLEInstructions              -2.627e+02  3.032e+07   0.000
## COMMUNICATIVE_ROLELeaflet                    1.924e+00  2.592e+00   0.742
## COMMUNICATIVE_ROLESlogan                     2.383e+00  2.011e+00   1.185
## COMMUNICATIVE_ROLEStreet_Signs              -1.667e+02  1.628e+07   0.000
## MATERIALITYHand_Written                      3.010e+00  3.356e+00   0.897
## MATERIALITYHome_Printed                      2.265e+00  3.597e+00   0.630
## MATERIALITYPermanent                        -1.213e+00  3.832e+00  -0.317
## MATERIALITYProfessionally_Printed            6.240e-02  3.447e+00   0.018
## CONTEXT_FRAMEBakery                          3.139e+02  6.711e+07   0.000
## CONTEXT_FRAMEBar                             2.685e+02  6.946e+07   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon               3.114e+02  6.711e+07   0.000
## CONTEXT_FRAMEBusiness                        3.129e+02  6.711e+07   0.000
## CONTEXT_FRAMECafe                            2.752e+02  6.838e+07   0.000
## CONTEXT_FRAMEClothing                        3.139e+02  6.711e+07   0.000
## CONTEXT_FRAMECommentary                      3.167e+02  6.711e+07   0.000
## CONTEXT_FRAMEExternal                        3.146e+02  6.711e+07   0.000
## CONTEXT_FRAMEFlier                           3.613e+02  8.219e+07   0.000
## CONTEXT_FRAMEGallery_Museum                  3.003e+02  6.711e+07   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store            3.121e+02  6.711e+07   0.000
## CONTEXT_FRAMEGym_Fitness_Studio              2.746e+02  7.503e+07   0.000
## CONTEXT_FRAMEHardware                        2.675e+02  8.219e+07   0.000
## CONTEXT_FRAMEHotel                           2.680e+02  7.249e+07   0.000
## CONTEXT_FRAMEInstitution                     3.120e+02  6.711e+07   0.000
## CONTEXT_FRAMEJewelry_Store                   3.081e+02  6.711e+07   0.000
## CONTEXT_FRAMELandromat                       7.204e+01  8.268e+07   0.000
## CONTEXT_FRAMEMenu                            6.990e+03  9.491e+07   0.000
## CONTEXT_FRAMEMovie_Theater                   3.129e+02  6.711e+07   0.000
## CONTEXT_FRAMENightclub                       2.521e+02  7.749e+07   0.000
## CONTEXT_FRAMENotary_Financial_Services       3.141e+02  6.711e+07   0.000
## CONTEXT_FRAMEResidential                     4.009e+02  8.878e+07   0.000
## CONTEXT_FRAMERestaurant                      3.149e+02  6.711e+07   0.000
## CONTEXT_FRAMEShop                            3.119e+02  6.711e+07   0.000
## CONTEXT_FRAMESpecialty_Foods                 3.495e+02  7.045e+07   0.000
## CONTEXT_FRAMESupermarket                     3.127e+02  6.711e+07   0.000
## CONTEXT_FRAMETravel_Agency                   3.212e+02  6.711e+07   0.000
## CLOSEDFALSE                                  5.548e+01  6.711e+07   0.000
## CLOSEDTRUE                                   5.434e+01  6.711e+07   0.000
## Topic2                                      -1.388e+00  1.392e+00  -0.997
## Topic3                                      -1.043e+00  1.299e+00  -0.803
## Topic4                                       3.978e-01  1.390e+00   0.286
## Topic5                                      -3.210e+00  1.909e+00  -1.682
## Topic6                                      -1.250e+00  1.590e+00  -0.786
## Topic7                                      -7.038e-01  3.401e+00  -0.207
## Topic8                                      -6.842e+00  2.277e+00  -3.005
## Topic9                                      -3.714e+00  1.972e+00  -1.883
## Topic10                                     -2.015e+00  1.988e+00  -1.014
## Topic11                                      1.331e+00  2.402e+00   0.554
## Topic12                                      3.456e-01  1.050e+00   0.329
## Topic13                                     -4.195e+01  3.047e+07   0.000
## Topic14                                     -4.890e+01  3.001e+07   0.000
## Topic15                                     -4.179e+00  2.514e+00  -1.662
## Topic16                                     -7.043e+00  2.499e+00  -2.818
## Topic17                                     -7.767e+00  2.821e+00  -2.753
## Topic18                                     -3.160e+00  2.059e+00  -1.534
## Topic19                                     -1.841e+00  2.599e+00  -0.708
## Topic20                                     -5.276e+01  3.355e+07   0.000
## Topic21                                     -2.558e-01  2.339e+00  -0.109
## Topic22                                     -4.039e+01  1.799e+07   0.000
##                                             Pr(&amp;gt;|z|)   
## (Intercept)                                  0.99999   
## YELP1                                        0.53552   
## YELP2                                        0.00702 **
## YELP3                                        0.98168   
## YELP4                                        1.00000   
## COMMUNICATIVE_ROLEEstablishment_Description  0.05014 . 
## COMMUNICATIVE_ROLEEstablishment_Name         0.08326 . 
## COMMUNICATIVE_ROLEGraffiti                   0.50272   
## COMMUNICATIVE_ROLEInformation                0.24273   
## COMMUNICATIVE_ROLEInstructions               0.99999   
## COMMUNICATIVE_ROLELeaflet                    0.45787   
## COMMUNICATIVE_ROLESlogan                     0.23612   
## COMMUNICATIVE_ROLEStreet_Signs               0.99999   
## MATERIALITYHand_Written                      0.36988   
## MATERIALITYHome_Printed                      0.52884   
## MATERIALITYPermanent                         0.75151   
## MATERIALITYProfessionally_Printed            0.98556   
## CONTEXT_FRAMEBakery                          1.00000   
## CONTEXT_FRAMEBar                             1.00000   
## CONTEXT_FRAMEBeauty_Hair_Salon               1.00000   
## CONTEXT_FRAMEBusiness                        1.00000   
## CONTEXT_FRAMECafe                            1.00000   
## CONTEXT_FRAMEClothing                        1.00000   
## CONTEXT_FRAMECommentary                      1.00000   
## CONTEXT_FRAMEExternal                        1.00000   
## CONTEXT_FRAMEFlier                           1.00000   
## CONTEXT_FRAMEGallery_Museum                  1.00000   
## CONTEXT_FRAMEGrocery_Liquor_Store            1.00000   
## CONTEXT_FRAMEGym_Fitness_Studio              1.00000   
## CONTEXT_FRAMEHardware                        1.00000   
## CONTEXT_FRAMEHotel                           1.00000   
## CONTEXT_FRAMEInstitution                     1.00000   
## CONTEXT_FRAMEJewelry_Store                   1.00000   
## CONTEXT_FRAMELandromat                       1.00000   
## CONTEXT_FRAMEMenu                            0.99994   
## CONTEXT_FRAMEMovie_Theater                   1.00000   
## CONTEXT_FRAMENightclub                       1.00000   
## CONTEXT_FRAMENotary_Financial_Services       1.00000   
## CONTEXT_FRAMEResidential                     1.00000   
## CONTEXT_FRAMERestaurant                      1.00000   
## CONTEXT_FRAMEShop                            1.00000   
## CONTEXT_FRAMESpecialty_Foods                 1.00000   
## CONTEXT_FRAMESupermarket                     1.00000   
## CONTEXT_FRAMETravel_Agency                   1.00000   
## CLOSEDFALSE                                  1.00000   
## CLOSEDTRUE                                   1.00000   
## Topic2                                       0.31881   
## Topic3                                       0.42176   
## Topic4                                       0.77471   
## Topic5                                       0.09262 . 
## Topic6                                       0.43168   
## Topic7                                       0.83605   
## Topic8                                       0.00266 **
## Topic9                                       0.05965 . 
## Topic10                                      0.31071   
## Topic11                                      0.57937   
## Topic12                                      0.74211   
## Topic13                                      1.00000   
## Topic14                                      1.00000   
## Topic15                                      0.09648 . 
## Topic16                                      0.00483 **
## Topic17                                      0.00591 **
## Topic18                                      0.12492   
## Topic19                                      0.47879   
## Topic20                                      1.00000   
## Topic21                                      0.91292   
## Topic22                                      1.00000   
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value
## s(latitude,longitude) 57.96  58.61  64.08   0.294
## 
## R-sq.(adj) =  0.438   Deviance explained = 53.4%
## UBRE = -0.173  Scale est. = 1         n = 700&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamES))

# Mostly English with Some Spanish
# Spanish
summary(gamES)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Eng_Span&amp;quot;) ~ s(latitude, longitude, k = 60) + 
##     YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + 
##     CLOSED + Topic
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -9.157e+01  6.723e+07   0.000
## YELP1                                        1.712e+00  1.024e+00   1.672
## YELP2                                       -3.042e-01  1.165e+00  -0.261
## YELP3                                       -3.202e+01  1.331e+07   0.000
## YELP4                                        9.460e-01  5.054e+07   0.000
## COMMUNICATIVE_ROLEEstablishment_Description -2.680e-02  1.475e+00  -0.018
## COMMUNICATIVE_ROLEEstablishment_Name         2.765e-02  1.424e+00   0.019
## COMMUNICATIVE_ROLEGraffiti                   3.348e+01  8.920e+06   0.000
## COMMUNICATIVE_ROLEInformation               -9.775e-01  1.595e+00  -0.613
## COMMUNICATIVE_ROLEInstructions              -2.237e+00  3.135e+07   0.000
## COMMUNICATIVE_ROLELeaflet                    1.298e+01  1.978e+07   0.000
## COMMUNICATIVE_ROLESlogan                     2.938e+00  2.178e+00   1.349
## COMMUNICATIVE_ROLEStreet_Signs              -4.105e+00  1.867e+07   0.000
## MATERIALITYHand_Written                      2.620e+01  2.552e+06   0.000
## MATERIALITYHome_Printed                      2.584e+01  2.552e+06   0.000
## MATERIALITYPermanent                         2.910e+01  2.552e+06   0.000
## MATERIALITYProfessionally_Printed            2.669e+01  2.552e+06   0.000
## CONTEXT_FRAMEBakery                          2.951e+01  6.994e+07   0.000
## CONTEXT_FRAMEBar                             3.607e+01  6.949e+07   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon               6.884e+01  6.711e+07   0.000
## CONTEXT_FRAMEBusiness                        6.671e+01  6.711e+07   0.000
## CONTEXT_FRAMECafe                            3.592e+01  6.835e+07   0.000
## CONTEXT_FRAMEClothing                        3.707e+01  6.932e+07   0.000
## CONTEXT_FRAMECommentary                      5.441e+00  7.104e+07   0.000
## CONTEXT_FRAMEExternal                        3.740e+01  6.770e+07   0.000
## CONTEXT_FRAMEFlier                           2.216e+01  8.454e+07   0.000
## CONTEXT_FRAMEGallery_Museum                  7.062e+01  6.711e+07   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store            6.907e+01  6.711e+07   0.000
## CONTEXT_FRAMEGym_Fitness_Studio              3.604e+01  7.503e+07   0.000
## CONTEXT_FRAMEHardware                        3.687e+01  8.219e+07   0.000
## CONTEXT_FRAMEHotel                           7.043e+01  6.711e+07   0.000
## CONTEXT_FRAMEInstitution                     3.782e+01  7.127e+07   0.000
## CONTEXT_FRAMEJewelry_Store                   3.541e+01  7.045e+07   0.000
## CONTEXT_FRAMELandromat                       4.067e+01  8.232e+07   0.000
## CONTEXT_FRAMEMenu                            4.285e+01  9.491e+07   0.000
## CONTEXT_FRAMEMovie_Theater                   3.530e+01  7.187e+07   0.000
## CONTEXT_FRAMENightclub                       3.814e+01  7.786e+07   0.000
## CONTEXT_FRAMENotary_Financial_Services       3.813e+01  7.351e+07   0.000
## CONTEXT_FRAMEResidential                     6.612e+01  8.893e+07   0.000
## CONTEXT_FRAMERestaurant                      6.800e+01  6.711e+07   0.000
## CONTEXT_FRAMEShop                            6.791e+01  6.711e+07   0.000
## CONTEXT_FRAMESpecialty_Foods                 3.632e+01  7.062e+07   0.000
## CONTEXT_FRAMESupermarket                     6.970e+01  6.711e+07   0.000
## CONTEXT_FRAMETravel_Agency                   3.740e+01  7.351e+07   0.000
## CLOSEDFALSE                                 -7.310e+00  3.126e+06   0.000
## CLOSEDTRUE                                  -7.519e+00  3.126e+06   0.000
## Topic2                                       2.534e-01  1.140e+00   0.222
## Topic3                                      -5.211e-01  1.581e+00  -0.330
## Topic4                                      -2.192e+00  1.360e+00  -1.611
## Topic5                                      -3.224e+01  1.140e+07   0.000
## Topic6                                       9.174e-01  1.343e+00   0.683
## Topic7                                       1.400e+00  1.672e+00   0.838
## Topic8                                      -3.084e+01  3.122e+06   0.000
## Topic9                                       8.210e-01  1.016e+00   0.808
## Topic10                                     -1.965e+00  1.470e+00  -1.336
## Topic11                                     -3.118e+01  2.340e+07   0.000
## Topic12                                     -3.255e+01  1.469e+07   0.000
## Topic13                                     -3.089e+01  3.046e+07   0.000
## Topic14                                     -2.990e+01  3.006e+07   0.000
## Topic15                                      6.170e+00  1.775e+00   3.476
## Topic16                                     -5.035e-01  1.513e+00  -0.333
## Topic17                                     -3.304e+01  1.684e+07   0.000
## Topic18                                      1.425e-01  2.359e+00   0.060
## Topic19                                     -3.344e-01  1.301e+00  -0.257
## Topic20                                     -3.063e+01  3.395e+07   0.000
## Topic21                                     -3.061e+01  1.837e+07   0.000
## Topic22                                     -5.892e+01  9.203e+06   0.000
##                                             Pr(&amp;gt;|z|)    
## (Intercept)                                 0.999999    
## YELP1                                       0.094554 .  
## YELP2                                       0.793932    
## YELP3                                       0.999998    
## YELP4                                       1.000000    
## COMMUNICATIVE_ROLEEstablishment_Description 0.985504    
## COMMUNICATIVE_ROLEEstablishment_Name        0.984507    
## COMMUNICATIVE_ROLEGraffiti                  0.999997    
## COMMUNICATIVE_ROLEInformation               0.539858    
## COMMUNICATIVE_ROLEInstructions              1.000000    
## COMMUNICATIVE_ROLELeaflet                   0.999999    
## COMMUNICATIVE_ROLESlogan                    0.177245    
## COMMUNICATIVE_ROLEStreet_Signs              1.000000    
## MATERIALITYHand_Written                     0.999992    
## MATERIALITYHome_Printed                     0.999992    
## MATERIALITYPermanent                        0.999991    
## MATERIALITYProfessionally_Printed           0.999992    
## CONTEXT_FRAMEBakery                         1.000000    
## CONTEXT_FRAMEBar                            1.000000    
## CONTEXT_FRAMEBeauty_Hair_Salon              0.999999    
## CONTEXT_FRAMEBusiness                       0.999999    
## CONTEXT_FRAMECafe                           1.000000    
## CONTEXT_FRAMEClothing                       1.000000    
## CONTEXT_FRAMECommentary                     1.000000    
## CONTEXT_FRAMEExternal                       1.000000    
## CONTEXT_FRAMEFlier                          1.000000    
## CONTEXT_FRAMEGallery_Museum                 0.999999    
## CONTEXT_FRAMEGrocery_Liquor_Store           0.999999    
## CONTEXT_FRAMEGym_Fitness_Studio             1.000000    
## CONTEXT_FRAMEHardware                       1.000000    
## CONTEXT_FRAMEHotel                          0.999999    
## CONTEXT_FRAMEInstitution                    1.000000    
## CONTEXT_FRAMEJewelry_Store                  1.000000    
## CONTEXT_FRAMELandromat                      1.000000    
## CONTEXT_FRAMEMenu                           1.000000    
## CONTEXT_FRAMEMovie_Theater                  1.000000    
## CONTEXT_FRAMENightclub                      1.000000    
## CONTEXT_FRAMENotary_Financial_Services      1.000000    
## CONTEXT_FRAMEResidential                    0.999999    
## CONTEXT_FRAMERestaurant                     0.999999    
## CONTEXT_FRAMEShop                           0.999999    
## CONTEXT_FRAMESpecialty_Foods                1.000000    
## CONTEXT_FRAMESupermarket                    0.999999    
## CONTEXT_FRAMETravel_Agency                  1.000000    
## CLOSEDFALSE                                 0.999998    
## CLOSEDTRUE                                  0.999998    
## Topic2                                      0.824161    
## Topic3                                      0.741752    
## Topic4                                      0.107155    
## Topic5                                      0.999998    
## Topic6                                      0.494642    
## Topic7                                      0.402244    
## Topic8                                      0.999992    
## Topic9                                      0.418832    
## Topic10                                     0.181480    
## Topic11                                     0.999999    
## Topic12                                     0.999998    
## Topic13                                     0.999999    
## Topic14                                     0.999999    
## Topic15                                     0.000509 ***
## Topic16                                     0.739367    
## Topic17                                     0.999998    
## Topic18                                     0.951830    
## Topic19                                     0.797135    
## Topic20                                     0.999999    
## Topic21                                     0.999999    
## Topic22                                     0.999995    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value
## s(latitude,longitude) 17.64  23.38  23.22   0.467
## 
## R-sq.(adj) =  0.256   Deviance explained = 47.1%
## UBRE = -0.56172  Scale est. = 1         n = 700&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamES))

# Mostly Spanish with some English
summary(gamSE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Span_Eng&amp;quot;) ~ s(latitude, longitude, k = 60) + 
##     YELP + COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + 
##     CLOSED + Topic
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                 -8.388e+06  6.712e+07  -0.125
## YELP1                                       -5.546e-01  5.436e-01  -1.020
## YELP2                                       -1.772e+00  9.086e-01  -1.950
## YELP3                                       -2.471e+01  1.482e+05   0.000
## YELP4                                       -2.476e+01  7.382e+05   0.000
## COMMUNICATIVE_ROLEEstablishment_Description  6.912e-01  1.237e+00   0.559
## COMMUNICATIVE_ROLEEstablishment_Name         2.719e-02  1.204e+00   0.023
## COMMUNICATIVE_ROLEGraffiti                  -2.871e-02  3.738e+05   0.000
## COMMUNICATIVE_ROLEInformation                9.792e-01  1.248e+00   0.785
## COMMUNICATIVE_ROLEInstructions              -1.003e-01  3.930e+05   0.000
## COMMUNICATIVE_ROLELeaflet                   -2.249e+01  3.507e+05   0.000
## COMMUNICATIVE_ROLESlogan                     1.047e+00  4.646e+05   0.000
## COMMUNICATIVE_ROLEStreet_Signs               2.317e+01  2.708e+05   0.000
## MATERIALITYHand_Written                     -2.686e+00  3.420e+05   0.000
## MATERIALITYHome_Printed                     -1.065e+00  3.420e+05   0.000
## MATERIALITYPermanent                        -2.458e+01  3.602e+05   0.000
## MATERIALITYProfessionally_Printed           -8.574e-01  3.420e+05   0.000
## CONTEXT_FRAMEBakery                          8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEBar                             8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEBeauty_Hair_Salon               8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEBusiness                        8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMECafe                            8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEClothing                        8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMECommentary                      8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEExternal                        8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEFlier                           8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEGallery_Museum                  8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEGrocery_Liquor_Store            8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEGym_Fitness_Studio              8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEHardware                        8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEHotel                           8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEInstitution                     8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEJewelry_Store                   8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMELandromat                       8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEMenu                            8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEMovie_Theater                   8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMENightclub                       8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMENotary_Financial_Services       8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEResidential                     8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMERestaurant                      8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMEShop                            8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMESpecialty_Foods                 8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMESupermarket                     8.388e+06  6.711e+07   0.125
## CONTEXT_FRAMETravel_Agency                   8.388e+06  6.711e+07   0.125
## CLOSEDFALSE                                  1.256e+00  1.179e+06   0.000
## CLOSEDTRUE                                   1.712e+00  1.179e+06   0.000
## Topic2                                       2.975e-01  7.916e-01   0.376
## Topic3                                      -9.652e-01  1.170e+00  -0.825
## Topic4                                      -1.114e+00  8.622e-01  -1.292
## Topic5                                       1.321e+00  6.948e-01   1.902
## Topic6                                      -5.116e+01  1.801e+05   0.000
## Topic7                                      -2.609e+01  2.034e+05   0.000
## Topic8                                      -2.550e+01  2.359e+05   0.000
## Topic9                                      -2.607e+01  1.266e+05   0.000
## Topic10                                     -1.364e+00  1.061e+00  -1.285
## Topic11                                     -2.577e+01  2.807e+05   0.000
## Topic12                                      1.118e-01  9.142e-01   0.122
## Topic13                                     -2.633e+01  4.527e+05   0.000
## Topic14                                     -2.553e+01  4.402e+05   0.000
## Topic15                                     -2.618e+01  4.851e+05   0.000
## Topic16                                     -2.533e+01  1.231e+05   0.000
## Topic17                                     -2.112e-01  1.219e+00  -0.173
## Topic18                                     -2.023e+00  1.422e+00  -1.423
## Topic19                                     -6.111e-01  9.939e-01  -0.615
## Topic20                                     -2.548e+01  4.378e+05   0.000
## Topic21                                      1.126e+00  1.005e+00   1.120
## Topic22                                     -2.517e+01  2.460e+05   0.000
##                                             Pr(&amp;gt;|z|)  
## (Intercept)                                   0.9005  
## YELP1                                         0.3076  
## YELP2                                         0.0512 .
## YELP3                                         0.9999  
## YELP4                                         1.0000  
## COMMUNICATIVE_ROLEEstablishment_Description   0.5763  
## COMMUNICATIVE_ROLEEstablishment_Name          0.9820  
## COMMUNICATIVE_ROLEGraffiti                    1.0000  
## COMMUNICATIVE_ROLEInformation                 0.4326  
## COMMUNICATIVE_ROLEInstructions                1.0000  
## COMMUNICATIVE_ROLELeaflet                     0.9999  
## COMMUNICATIVE_ROLESlogan                      1.0000  
## COMMUNICATIVE_ROLEStreet_Signs                0.9999  
## MATERIALITYHand_Written                       1.0000  
## MATERIALITYHome_Printed                       1.0000  
## MATERIALITYPermanent                          0.9999  
## MATERIALITYProfessionally_Printed             1.0000  
## CONTEXT_FRAMEBakery                           0.9005  
## CONTEXT_FRAMEBar                              0.9005  
## CONTEXT_FRAMEBeauty_Hair_Salon                0.9005  
## CONTEXT_FRAMEBusiness                         0.9005  
## CONTEXT_FRAMECafe                             0.9005  
## CONTEXT_FRAMEClothing                         0.9005  
## CONTEXT_FRAMECommentary                       0.9005  
## CONTEXT_FRAMEExternal                         0.9005  
## CONTEXT_FRAMEFlier                            0.9005  
## CONTEXT_FRAMEGallery_Museum                   0.9005  
## CONTEXT_FRAMEGrocery_Liquor_Store             0.9005  
## CONTEXT_FRAMEGym_Fitness_Studio               0.9005  
## CONTEXT_FRAMEHardware                         0.9005  
## CONTEXT_FRAMEHotel                            0.9005  
## CONTEXT_FRAMEInstitution                      0.9005  
## CONTEXT_FRAMEJewelry_Store                    0.9005  
## CONTEXT_FRAMELandromat                        0.9005  
## CONTEXT_FRAMEMenu                             0.9005  
## CONTEXT_FRAMEMovie_Theater                    0.9005  
## CONTEXT_FRAMENightclub                        0.9005  
## CONTEXT_FRAMENotary_Financial_Services        0.9005  
## CONTEXT_FRAMEResidential                      0.9005  
## CONTEXT_FRAMERestaurant                       0.9005  
## CONTEXT_FRAMEShop                             0.9005  
## CONTEXT_FRAMESpecialty_Foods                  0.9005  
## CONTEXT_FRAMESupermarket                      0.9005  
## CONTEXT_FRAMETravel_Agency                    0.9005  
## CLOSEDFALSE                                   1.0000  
## CLOSEDTRUE                                    1.0000  
## Topic2                                        0.7070  
## Topic3                                        0.4094  
## Topic4                                        0.1962  
## Topic5                                        0.0572 .
## Topic6                                        0.9998  
## Topic7                                        0.9999  
## Topic8                                        0.9999  
## Topic9                                        0.9998  
## Topic10                                       0.1986  
## Topic11                                       0.9999  
## Topic12                                       0.9026  
## Topic13                                       1.0000  
## Topic14                                       1.0000  
## Topic15                                       1.0000  
## Topic16                                       0.9998  
## Topic17                                       0.8625  
## Topic18                                       0.1548  
## Topic19                                       0.5387  
## Topic20                                       1.0000  
## Topic21                                       0.2626  
## Topic22                                       0.9999  
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value  
## s(latitude,longitude) 2.001  2.001  5.263  0.0721 .
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.074   Deviance explained = 31.1%
## UBRE = -0.50668  Scale est. = 1         n = 700&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamSE))

# Equal
summary(gamEQ)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(LANGUAGE == &amp;quot;Equal&amp;quot;) ~ s(latitude, longitude, k = 60) + YELP + 
##     COMMUNICATIVE_ROLE + MATERIALITY + CONTEXT_FRAME + CLOSED + 
##     Topic
## 
## Parametric coefficients:
##                                               Estimate Std. Error z value
## (Intercept)                                  2.852e+02  1.036e+08   0.000
## YELP1                                       -2.500e+01  1.868e+03  -0.013
## YELP2                                       -2.659e+01  3.278e+03  -0.008
## YELP3                                       -1.986e+03  1.424e+07   0.000
## YELP4                                        3.948e+02  5.071e+07   0.000
## COMMUNICATIVE_ROLEEstablishment_Description  2.430e+01  2.547e+05   0.000
## COMMUNICATIVE_ROLEEstablishment_Name         2.329e+01  2.547e+05   0.000
## COMMUNICATIVE_ROLEGraffiti                  -1.301e+02  2.252e+07   0.000
## COMMUNICATIVE_ROLEInformation                2.568e+01  2.547e+05   0.000
## COMMUNICATIVE_ROLEInstructions              -1.092e+03  3.071e+07   0.000
## COMMUNICATIVE_ROLELeaflet                   -2.776e+02  4.703e+07   0.000
## COMMUNICATIVE_ROLESlogan                    -1.116e+01  3.092e+07   0.000
## COMMUNICATIVE_ROLEStreet_Signs               4.654e+01  1.937e+07   0.000
## MATERIALITYHand_Written                     -8.454e+01  4.152e+07   0.000
## MATERIALITYHome_Printed                     -8.813e+01  4.152e+07   0.000
## MATERIALITYPermanent                        -3.476e+02  4.277e+07   0.000
## MATERIALITYProfessionally_Printed           -2.691e+02  4.152e+07   0.000
## CONTEXT_FRAMEBakery                         -6.589e+02  6.794e+07   0.000
## CONTEXT_FRAMEBar                            -4.177e+02  6.957e+07   0.000
## CONTEXT_FRAMEBeauty_Hair_Salon              -4.185e+02  6.851e+07   0.000
## CONTEXT_FRAMEBusiness                       -2.529e+02  6.711e+07   0.000
## CONTEXT_FRAMECafe                           -4.759e+02  6.842e+07   0.000
## CONTEXT_FRAMEClothing                       -4.444e+02  6.929e+07   0.000
## CONTEXT_FRAMECommentary                     -4.744e+02  7.478e+07   0.000
## CONTEXT_FRAMEExternal                       -2.628e+02  6.711e+07   0.000
## CONTEXT_FRAMEFlier                           3.668e+02  9.538e+07   0.000
## CONTEXT_FRAMEGallery_Museum                  7.260e+02  6.711e+07   0.000
## CONTEXT_FRAMEGrocery_Liquor_Store           -5.364e+02  6.794e+07   0.000
## CONTEXT_FRAMEGym_Fitness_Studio             -3.441e+02  7.543e+07   0.000
## CONTEXT_FRAMEHardware                       -2.966e+02  8.248e+07   0.000
## CONTEXT_FRAMEHotel                          -2.207e+02  6.711e+07   0.000
## CONTEXT_FRAMEInstitution                    -4.852e+02  7.126e+07   0.000
## CONTEXT_FRAMEJewelry_Store                  -5.329e+02  7.048e+07   0.000
## CONTEXT_FRAMELandromat                      -5.956e+02  8.219e+07   0.000
## CONTEXT_FRAMEMenu                            1.656e+03  9.553e+07   0.000
## CONTEXT_FRAMEMovie_Theater                  -3.023e+02  7.194e+07   0.000
## CONTEXT_FRAMENightclub                       8.276e+02  7.817e+07   0.000
## CONTEXT_FRAMENotary_Financial_Services      -2.366e+02  7.449e+07   0.000
## CONTEXT_FRAMEResidential                    -2.834e+02  8.922e+07   0.000
## CONTEXT_FRAMERestaurant                     -5.356e+02  6.740e+07   0.000
## CONTEXT_FRAMEShop                           -2.492e+02  6.711e+07   0.000
## CONTEXT_FRAMESpecialty_Foods                -2.741e+02  7.066e+07   0.000
## CONTEXT_FRAMESupermarket                    -4.127e+02  6.711e+07   0.000
## CONTEXT_FRAMETravel_Agency                   3.185e+02  7.351e+07   0.000
## CLOSEDFALSE                                 -2.562e+02  6.711e+07   0.000
## CLOSEDTRUE                                  -1.812e+02  6.895e+07   0.000
## Topic2                                       1.122e+02  5.694e+03   0.020
## Topic3                                      -7.341e+01  1.513e+04  -0.005
## Topic4                                       2.010e+01  1.017e+04   0.002
## Topic5                                      -9.792e+01  1.166e+07   0.000
## Topic6                                      -1.909e+02  1.354e+07   0.000
## Topic7                                      -2.990e+02  1.544e+07   0.000
## Topic8                                      -4.571e+01  1.567e+04  -0.003
## Topic9                                      -3.349e+02  9.304e+06   0.000
## Topic10                                     -2.491e+02  1.086e+07   0.000
## Topic11                                      3.518e+02  3.937e+04   0.009
## Topic12                                     -1.031e+02  1.503e+07   0.000
## Topic13                                      1.268e+02  3.076e+07   0.000
## Topic14                                     -4.679e+02  3.017e+07   0.000
## Topic15                                      1.865e+02  3.031e+07   0.000
## Topic16                                     -1.588e+02  1.731e+07   0.000
## Topic17                                      2.371e+02  1.731e+07   0.000
## Topic18                                      2.497e+02  1.060e+07   0.000
## Topic19                                     -1.098e+00  1.019e+04   0.000
## Topic20                                      2.688e+02  3.471e+07   0.000
## Topic21                                      1.669e+02  3.945e+04   0.004
## Topic22                                      3.523e+02  9.783e+03   0.036
##                                             Pr(&amp;gt;|z|)
## (Intercept)                                    1.000
## YELP1                                          0.989
## YELP2                                          0.994
## YELP3                                          1.000
## YELP4                                          1.000
## COMMUNICATIVE_ROLEEstablishment_Description    1.000
## COMMUNICATIVE_ROLEEstablishment_Name           1.000
## COMMUNICATIVE_ROLEGraffiti                     1.000
## COMMUNICATIVE_ROLEInformation                  1.000
## COMMUNICATIVE_ROLEInstructions                 1.000
## COMMUNICATIVE_ROLELeaflet                      1.000
## COMMUNICATIVE_ROLESlogan                       1.000
## COMMUNICATIVE_ROLEStreet_Signs                 1.000
## MATERIALITYHand_Written                        1.000
## MATERIALITYHome_Printed                        1.000
## MATERIALITYPermanent                           1.000
## MATERIALITYProfessionally_Printed              1.000
## CONTEXT_FRAMEBakery                            1.000
## CONTEXT_FRAMEBar                               1.000
## CONTEXT_FRAMEBeauty_Hair_Salon                 1.000
## CONTEXT_FRAMEBusiness                          1.000
## CONTEXT_FRAMECafe                              1.000
## CONTEXT_FRAMEClothing                          1.000
## CONTEXT_FRAMECommentary                        1.000
## CONTEXT_FRAMEExternal                          1.000
## CONTEXT_FRAMEFlier                             1.000
## CONTEXT_FRAMEGallery_Museum                    1.000
## CONTEXT_FRAMEGrocery_Liquor_Store              1.000
## CONTEXT_FRAMEGym_Fitness_Studio                1.000
## CONTEXT_FRAMEHardware                          1.000
## CONTEXT_FRAMEHotel                             1.000
## CONTEXT_FRAMEInstitution                       1.000
## CONTEXT_FRAMEJewelry_Store                     1.000
## CONTEXT_FRAMELandromat                         1.000
## CONTEXT_FRAMEMenu                              1.000
## CONTEXT_FRAMEMovie_Theater                     1.000
## CONTEXT_FRAMENightclub                         1.000
## CONTEXT_FRAMENotary_Financial_Services         1.000
## CONTEXT_FRAMEResidential                       1.000
## CONTEXT_FRAMERestaurant                        1.000
## CONTEXT_FRAMEShop                              1.000
## CONTEXT_FRAMESpecialty_Foods                   1.000
## CONTEXT_FRAMESupermarket                       1.000
## CONTEXT_FRAMETravel_Agency                     1.000
## CLOSEDFALSE                                    1.000
## CLOSEDTRUE                                     1.000
## Topic2                                         0.984
## Topic3                                         0.996
## Topic4                                         0.998
## Topic5                                         1.000
## Topic6                                         1.000
## Topic7                                         1.000
## Topic8                                         0.998
## Topic9                                         1.000
## Topic10                                        1.000
## Topic11                                        0.993
## Topic12                                        1.000
## Topic13                                        1.000
## Topic14                                        1.000
## Topic15                                        1.000
## Topic16                                        1.000
## Topic17                                        1.000
## Topic18                                        1.000
## Topic19                                        1.000
## Topic20                                        1.000
## Topic21                                        0.997
## Topic22                                        0.971
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value
## s(latitude,longitude) 12.76  13.06  0.002       1
## 
## R-sq.(adj) =  0.698   Deviance explained = 83.3%
## UBRE = -0.72869  Scale est. = 1         n = 700&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Odds ratios
# exp(coef(gamEQ))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We can see from these results a lot of information – what is significant, deviance explained, coefficients, etc. But it is also useful to plot probabilities.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Plot probabilities? (Adapted from http://myweb.uiowa.edu/pbreheny/publications/visreg.pdf)
library(visreg)
# We will just look at those flagged as &amp;#39;significant&amp;#39;
# Probability of English by coordinate
visreg2d(gamE, &amp;quot;longitude&amp;quot;, &amp;quot;latitude&amp;quot;, plot.type=&amp;quot;image&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-55-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Spanish
visreg2d(gamS, &amp;quot;longitude&amp;quot;, &amp;quot;latitude&amp;quot;, plot.type=&amp;quot;image&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-55-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;gams-for-instagram-only&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;GAMs for Instagram only&lt;/h2&gt;
&lt;p&gt;Let’s look at GAMs within our social media data set.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;emogam &amp;lt;- gam(I(emogrepl==&amp;quot;TRUE&amp;quot;)~s(latitude,longitude, k=60) + V1.x, family=binomial,data=tweets)
concurvity(emogam)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9168534            0.09941700
## observed 0.9168534            0.02028529
## estimate 0.9168534            0.01043618&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Concurvity seems ok
gam.check(emogam)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 3 iterations.
## Gradient range [5.004738e-08,5.004738e-08]
## (score -0.008444863 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.0001851222,0.0001851222].
## Model rank =  81 / 81 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000 36.914   0.915       0&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Same issue with gam.check but will keep k on the lower side to avoid over fitting

# Results!
summary(emogam)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(emogrepl == &amp;quot;TRUE&amp;quot;) ~ s(latitude, longitude, k = 60) + V1.x
## 
## Parametric coefficients:
##             Estimate Std. Error z value Pr(&amp;gt;|z|)    
## (Intercept) -1.51773    0.06987 -21.723  &amp;lt; 2e-16 ***
## V1.x2       -0.10551    0.10678  -0.988 0.323073    
## V1.x3        0.07772    0.10716   0.725 0.468282    
## V1.x4        0.01201    0.11055   0.109 0.913452    
## V1.x5        0.00548    0.11107   0.049 0.960649    
## V1.x6        0.18434    0.11212   1.644 0.100142    
## V1.x7        0.38492    0.11253   3.421 0.000625 ***
## V1.x8        0.11688    0.11947   0.978 0.327903    
## V1.x9       -0.01001    0.09944  -0.101 0.919822    
## V1.x10       0.24011    0.11187   2.146 0.031849 *  
## V1.x11       0.20051    0.12685   1.581 0.113942    
## V1.x12       0.16066    0.12227   1.314 0.188856    
## V1.x13       0.78768    0.11795   6.678 2.43e-11 ***
## V1.x14       0.07260    0.10874   0.668 0.504380    
## V1.x15       0.05380    0.12258   0.439 0.660701    
## V1.x16       0.27443    0.13654   2.010 0.044448 *  
## V1.x17       0.15745    0.10518   1.497 0.134416    
## V1.x18      -0.02877    0.13462  -0.214 0.830771    
## V1.x19       0.12386    0.12564   0.986 0.324213    
## V1.x20       0.09485    0.14550   0.652 0.514487    
## V1.x21       0.04794    0.15142   0.317 0.751550    
## V1.x22       0.09597    0.13291   0.722 0.470255    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq p-value    
## s(latitude,longitude) 36.91     46  193.8  &amp;lt;2e-16 ***
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.0158   Deviance explained = 1.87%
## UBRE = -0.0084449  Scale est. = 1         n = 16744&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Now Sparkles emoji
sparky &amp;lt;- grepl(paste(&amp;quot; SPARKLES &amp;quot;), tweets$text)
sparkyDF&amp;lt;-as.data.frame(sparky)
tweets$id &amp;lt;- 1:nrow(tweets)
sparkyDF$id &amp;lt;- 1:nrow(sparkyDF)
tweets &amp;lt;- merge(tweets,sparkyDF,by=&amp;quot;id&amp;quot;)

sparkygam &amp;lt;- gam(I(sparky==&amp;quot;TRUE&amp;quot;)~s(latitude,longitude, k=60) + V1.x, family=binomial, data=tweets)
concurvity(sparkygam)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##               para s(latitude,longitude)
## worst    0.9168534           0.099417003
## observed 0.9168534           0.009350418
## estimate 0.9168534           0.010436183&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;gam.check(sparkygam)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Method: UBRE   Optimizer: outer newton
## full convergence after 3 iterations.
## Gradient range [7.38181e-08,7.38181e-08]
## (score -0.8937008 &amp;amp; scale 1).
## Hessian positive definite, eigenvalue range [0.0003198463,0.0003198463].
## Model rank =  81 / 81 
## 
## Basis dimension (k) checking results. Low p-value (k-index&amp;lt;1) may
## indicate that k is too low, especially if edf is close to k&amp;#39;.
## 
##                           k&amp;#39;    edf k-index p-value
## s(latitude,longitude) 59.000 31.750   0.859    0.14&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;summary(sparkygam)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## 
## Family: binomial 
## Link function: logit 
## 
## Formula:
## I(sparky == &amp;quot;TRUE&amp;quot;) ~ s(latitude, longitude, k = 60) + V1.x
## 
## Parametric coefficients:
##              Estimate Std. Error z value Pr(&amp;gt;|z|)    
## (Intercept) -5.043769   0.308248 -16.363  &amp;lt; 2e-16 ***
## V1.x2        0.374535   0.421174   0.889 0.373860    
## V1.x3        0.521652   0.414222   1.259 0.207903    
## V1.x4        0.540606   0.421866   1.281 0.200031    
## V1.x5       -0.333582   0.542760  -0.615 0.538817    
## V1.x6        0.406962   0.453892   0.897 0.369929    
## V1.x7        0.610551   0.441826   1.382 0.167008    
## V1.x8        0.459427   0.469569   0.978 0.327876    
## V1.x9       -0.006097   0.429996  -0.014 0.988687    
## V1.x10      -0.073187   0.513893  -0.142 0.886751    
## V1.x11      -0.088493   0.588622  -0.150 0.880497    
## V1.x12       0.105301   0.543732   0.194 0.846439    
## V1.x13       1.363123   0.403352   3.379 0.000726 ***
## V1.x14       0.396858   0.430658   0.922 0.356781    
## V1.x15      -0.551737   0.654946  -0.842 0.399555    
## V1.x16      -1.265384   1.047887  -1.208 0.227217    
## V1.x17       0.066183   0.454785   0.146 0.884297    
## V1.x18      -0.010056   0.588844  -0.017 0.986375    
## V1.x19      -0.449025   0.657493  -0.683 0.494648    
## V1.x20       0.043034   0.656281   0.066 0.947718    
## V1.x21      -0.942557   1.047574  -0.900 0.368252    
## V1.x22      -0.214021   0.655523  -0.326 0.744055    
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## Approximate significance of smooth terms:
##                         edf Ref.df Chi.sq  p-value    
## s(latitude,longitude) 31.75  40.41  86.64 3.35e-05 ***
## ---
## Signif. codes:  0 &amp;#39;***&amp;#39; 0.001 &amp;#39;**&amp;#39; 0.01 &amp;#39;*&amp;#39; 0.05 &amp;#39;.&amp;#39; 0.1 &amp;#39; &amp;#39; 1
## 
## R-sq.(adj) =  0.0116   Deviance explained = 7.43%
## UBRE = -0.8937  Scale est. = 1         n = 16744&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-56-1.png&#34; width=&#34;672&#34; /&gt;&lt;img src=&#34;/portfolio/2017-05-06-thesis-time_files/figure-html/unnamed-chunk-56-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Census Data</title>
      <link>/portfolio/2017-03-12-census-data/</link>
      <pubDate>Sun, 12 Mar 2017 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-03-12-census-data/</guid>
      <description>&lt;script src=&#34;/rmarkdown-libs/htmlwidgets/htmlwidgets.js&#34;&gt;&lt;/script&gt;
&lt;script src=&#34;/rmarkdown-libs/jquery/jquery.min.js&#34;&gt;&lt;/script&gt;
&lt;link href=&#34;/rmarkdown-libs/leaflet/leaflet.css&#34; rel=&#34;stylesheet&#34; /&gt;
&lt;script src=&#34;/rmarkdown-libs/leaflet/leaflet.js&#34;&gt;&lt;/script&gt;
&lt;link href=&#34;/rmarkdown-libs/leafletfix/leafletfix.css&#34; rel=&#34;stylesheet&#34; /&gt;
&lt;link href=&#34;/rmarkdown-libs/leaflet-label/leaflet.label.css&#34; rel=&#34;stylesheet&#34; /&gt;
&lt;script src=&#34;/rmarkdown-libs/leaflet-label/leaflet.label.js&#34;&gt;&lt;/script&gt;
&lt;script src=&#34;/rmarkdown-libs/Proj4Leaflet/proj4-compressed.js&#34;&gt;&lt;/script&gt;
&lt;script src=&#34;/rmarkdown-libs/Proj4Leaflet/proj4leaflet.js&#34;&gt;&lt;/script&gt;
&lt;script src=&#34;/rmarkdown-libs/leaflet-binding/leaflet.js&#34;&gt;&lt;/script&gt;
&lt;script src=&#34;/rmarkdown-libs/leaflet-providers/leaflet-providers.js&#34;&gt;&lt;/script&gt;
&lt;script src=&#34;/rmarkdown-libs/leaflet-providers-plugin/leaflet-providers-plugin.js&#34;&gt;&lt;/script&gt;

&lt;p&gt;Profound gratitude is expressed to &lt;a href=&#34;http://zevross.com/blog/2015/10/14/manipulating-and-mapping-us-census-data-in-r-using-the-acs-tigris-and-leaflet-packages-3/#census-data-the-easyer-way&#34;&gt;this tutorial&lt;/a&gt; and the creators of the &lt;a href=&#34;https://cran.r-project.org/web/packages/acs/acs.pdf&#34;&gt;acs&lt;/a&gt; and &lt;a href=&#34;https://cran.r-project.org/web/packages/tigris/tigris.pdf&#34;&gt;tigris&lt;/a&gt; packages. If you’ve had the pleasure of stumbling around the U.S. Census website and awkwardly trying to transcribe and tabulate things in Excel you will quickly appreciate how very much better the R based approach is. Plus you can make pretty &lt;em&gt;and&lt;/em&gt; interactive maps! :O&lt;/p&gt;
&lt;div id=&#34;getting-census-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Getting census data&lt;/h2&gt;
&lt;p&gt;About 95% of the steps came from the &lt;a href=&#34;http://zevross.com/blog/2015/10/14/manipulating-and-mapping-us-census-data-in-r-using-the-acs-tigris-and-leaflet-packages-3/#census-data-the-easyer-way&#34;&gt;lovely tutorial&lt;/a&gt; mentioned above, but I’ll reproduce them here just to show how the code might be applied to another example. Everything from that original tutorial is in quotes. As mentioned, the main packages at work here are the &lt;a href=&#34;https://cran.r-project.org/web/packages/acs/acs.pdf&#34;&gt;acs&lt;/a&gt; and &lt;a href=&#34;https://cran.r-project.org/web/packages/tigris/tigris.pdf&#34;&gt;tigris&lt;/a&gt; packages.&lt;/p&gt;
&lt;p&gt;Some points about what these packages do:&lt;/p&gt;
&lt;p&gt;— The tigris package uses &lt;a href=&#34;https://www.census.gov/geo/maps-data/data/tiger-line.html&#34;&gt;TIGER/Line Shapefiles etc.&lt;/a&gt; which have geographic entity codes which enables them to be linked back to census data.&lt;/p&gt;
&lt;p&gt;— The acs package is how you download the census data. Maybe we want specific data sets (like JUST Decennial data, not ACS data). The package literature says about this: “By default, acs.fetch will download 5-year ACS, but as of version 2.0 users must specify a specific”endyear“. Users may also select 1- or 3-year ACS data using the”span=&amp;quot; option, as well as Decennial data using the “dataset” option. (When dataset=“sf1” or “sf3”, span will be reset to 0 regardless of any explict or default options.) At present, the API provides five-, three- and one-year data for a variety of different endyears, and Decennial data for 2010, 2000, and 1990; see the chart below and/or visit &lt;a href=&#34;http://www.census.gov/data/developers/data-sets.html&#34; class=&#34;uri&#34;&gt;http://www.census.gov/data/developers/data-sets.html&lt;/a&gt; to learn more about what is available through the API. (Warning: support for 1990 is a bit unreliable as of the date of this version, due to non-standard variable lookup tables.)“.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(tigris)
library(acs)
library(stringr) # to &amp;quot;pad fips codes&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Get spatial data
# &amp;quot;note that you can use county names in the tigris package but 
# not in the acs.fetch function from the acs package so I&amp;#39;m using
# fips numbers here.&amp;quot; 

# &amp;quot;grab the spatial data (tigris)&amp;quot;
# Get information on the FIPS codes from here: https://www.census.gov/geo/reference/codes/cou.html
# It&amp;#39;s formatted in a specific way, but the county number one you need is the third one
# Example: IL,17,031,Cook County,H1
# Or you can just do an easy look up with tigris package
lookup_code(&amp;quot;California&amp;quot;, &amp;quot;San Francisco&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;The code for California is &amp;#39;06&amp;#39; and the code for San Francisco County is &amp;#39;075&amp;#39;.&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;counties &amp;lt;- 075
tracts &amp;lt;- tracts(state = &amp;#39;CA&amp;#39;, county = 075, cb=TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You have to get a census API key &lt;a href=&#34;http://api.census.gov/data/key_signup.html&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Install your key
# api.key.install(key=&amp;quot;YOUR CENSUS KEY HERE&amp;quot;)

# &amp;quot;create a geographic set to grab tabular data (acs)&amp;quot;
# IF YOU WANT MORE THAN ONE COUNTY, DO county=c(#, #, #, #)
geo&amp;lt;-geo.make(state=&amp;quot;CA&amp;quot;,
              county=075, tract=&amp;quot;*&amp;quot;)

# &amp;quot;!!!! important note -- the package has not been updated to 2013&amp;quot;
# &amp;quot;data so I&amp;#39;m using the five year span that ends in 2012&amp;quot;

# if you want to change the data you are getting you change the table.number variable. This table.number comes from the ID field in the American Fact Finder tool from the census bureau. This one here has &amp;quot;Household Income in the Past 12 Months&amp;quot;

income&amp;lt;-acs.fetch(endyear = 2012, span = 5, geography = geo,
                table.number = &amp;quot;B19001&amp;quot;, col.names = &amp;quot;pretty&amp;quot;)

# &amp;quot;use of col.names = &amp;quot;pretty&amp;quot; above gives the full column definitions&amp;quot;
# &amp;quot;if you want Census variable IDs use col.names=&amp;quot;auto&amp;quot;. Here are the&amp;quot;
# &amp;quot;variables we want with pretty and auto results.&amp;quot;
#&amp;quot;&amp;quot;Household Income: Total:&amp;quot; (&amp;quot;B19001_001&amp;quot;)&amp;quot;
#&amp;quot;&amp;quot;Household Income: $200,000 or more&amp;quot; (&amp;quot;B19001_017&amp;quot;)&amp;quot;


# &amp;quot;the resulting &amp;quot;income&amp;quot; object is not a data.frame it&amp;#39;s a list&amp;quot;
# &amp;quot;to see what&amp;#39;s available&amp;quot;

# names(attributes(income))
##  [1] &amp;quot;endyear&amp;quot;        &amp;quot;span&amp;quot;           &amp;quot;acs.units&amp;quot;      &amp;quot;currency.year&amp;quot; 
##  [5] &amp;quot;modified&amp;quot;       &amp;quot;geography&amp;quot;      &amp;quot;acs.colnames&amp;quot;   &amp;quot;estimate&amp;quot;      
##  [9] &amp;quot;standard.error&amp;quot; &amp;quot;class&amp;quot;
# attr(income, &amp;quot;acs.colnames&amp;quot;)
##  [1] &amp;quot;Household Income: Total:&amp;quot;              
##  [2] &amp;quot;Household Income: Less than $10,000&amp;quot;   
##  [3] &amp;quot;Household Income: $10,000 to $14,999&amp;quot;  
##  [4] &amp;quot;Household Income: $15,000 to $19,999&amp;quot;  
##  [5] &amp;quot;Household Income: $20,000 to $24,999&amp;quot;  
##  [6] &amp;quot;Household Income: $25,000 to $29,999&amp;quot;  
##  [7] &amp;quot;Household Income: $30,000 to $34,999&amp;quot;  
##  [8] &amp;quot;Household Income: $35,000 to $39,999&amp;quot;  
##  [9] &amp;quot;Household Income: $40,000 to $44,999&amp;quot;  
## [10] &amp;quot;Household Income: $45,000 to $49,999&amp;quot;  
## [11] &amp;quot;Household Income: $50,000 to $59,999&amp;quot;  
## [12] &amp;quot;Household Income: $60,000 to $74,999&amp;quot;  
## [13] &amp;quot;Household Income: $75,000 to $99,999&amp;quot;  
## [14] &amp;quot;Household Income: $100,000 to $124,999&amp;quot;
## [15] &amp;quot;Household Income: $125,000 to $149,999&amp;quot;
## [16] &amp;quot;Household Income: $150,000 to $199,999&amp;quot;
## [17] &amp;quot;Household Income: $200,000 or more&amp;quot;

# &amp;quot;convert to a data.frame for merging&amp;quot;
income_df &amp;lt;- data.frame(paste0(str_pad(income@geography$state, 2, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(income@geography$county, 3, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(income@geography$tract, 6, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;)), 
                        income@estimate[,c(&amp;quot;Household Income: Total:&amp;quot;,
&amp;quot;Household Income: $200,000 or more&amp;quot;)], 
                        stringsAsFactors = FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You’ll need the &lt;a href=&#34;https://cran.r-project.org/web/packages/dplyr/index.html&#34;&gt;dplyr package&lt;/a&gt; for this next bit&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(dplyr)
income_df &amp;lt;- select(income_df, 1:3)
rownames(income_df)&amp;lt;-1:nrow(income_df)
names(income_df)&amp;lt;-c(&amp;quot;GEOID&amp;quot;, &amp;quot;total&amp;quot;, &amp;quot;over_200&amp;quot;)
income_df$percent &amp;lt;- 100*(income_df$over_200/income_df$total)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now as the tutorial says, ‘do the merge’.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;income_merged&amp;lt;- geo_join(tracts, income_df, &amp;quot;GEOID&amp;quot;, &amp;quot;GEOID&amp;quot;)
# there are some tracts with no land that we should exclude
income_merged &amp;lt;- income_merged[income_merged$ALAND&amp;gt;0,]&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;mapping-census-data&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Mapping Census Data&lt;/h2&gt;
&lt;p&gt;Now we use something called leaflet to make a map…&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;library(leaflet)
popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, income_merged$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent of Households above $200k: &amp;quot;, round(income_merged$percent,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = income_merged$percent
)

map3&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = income_merged, 
              fillColor = ~pal(percent), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = income_merged$percent, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent of Households&amp;lt;br&amp;gt;above $200k&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
map3&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-fac0309a6755fb81944f&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-fac0309a6755fb81944f&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.420612315115,-122.407452,-122.399638840473,-122.401787,-122.403495,-122.405174,-122.406625,-122.408332,-122.411581,-122.414877,-122.416888,-122.418718,-122.419224,-122.420612315115],&#34;lat&#34;:[37.8111121700604,37.811441,37.8065658502084,37.805314,37.805089,37.804763,37.804535,37.804367,37.803963,37.803542,37.805212,37.805932,37.808453,37.8111121700604]}]],[[{&#34;lng&#34;:[-122.411019,-122.40777,-122.406199,-122.406024,-122.405645,-122.402421,-122.402041,-122.405513,-122.406668,-122.407083,-122.407869,-122.409014,-122.411376,-122.411019],&#34;lat&#34;:[37.80117,37.801571,37.80172,37.800835,37.798969,37.799382,37.797505,37.797065,37.797865,37.798152,37.798697,37.79949,37.801125,37.80117]}]],[[{&#34;lng&#34;:[-122.422077,-122.415372,-122.415185,-122.414995,-122.418271,-122.421461,-122.421858,-122.422077],&#34;lat&#34;:[37.788474,37.78932,37.788388,37.787454,37.787038,37.786632,37.787535,37.788474]}]],[[{&#34;lng&#34;:[-122.441905,-122.437138,-122.436799,-122.436274,-122.441177,-122.441368,-122.441754,-122.441905],&#34;lat&#34;:[37.803744,37.804352,37.802715,37.800814,37.80019,37.801126,37.802985,37.803744]}]],[[{&#34;lng&#34;:[-122.459476,-122.45274,-122.4463,-122.446117,-122.445934,-122.445758,-122.445589,-122.445402,-122.450118,-122.454774,-122.45505,-122.459179,-122.459388,-122.459476],&#34;lat&#34;:[37.789711,37.790974,37.791879,37.790999,37.790119,37.789248,37.788364,37.787438,37.786838,37.786244,37.786209,37.785682,37.788405,37.789711]}]],[[{&#34;lng&#34;:[-122.458878,-122.456304,-122.454186,-122.453385,-122.455217,-122.454683,-122.458371,-122.458505,-122.458878],&#34;lat&#34;:[37.781326,37.781449,37.781651,37.777827,37.777599,37.774755,37.774309,37.77619,37.781326]}]],[[{&#34;lng&#34;:[-122.424678,-122.422838,-122.421256,-122.421069,-122.420689,-122.420358,-122.423541,-122.424108,-122.424678],&#34;lat&#34;:[37.785291,37.784547,37.784745,37.783816,37.781955,37.780072,37.779674,37.782477,37.785291]}]],[[{&#34;lng&#34;:[-122.433219,-122.429929,-122.426641,-122.426263,-122.425888,-122.429178,-122.430825,-122.432467,-122.433219],&#34;lat&#34;:[37.77749,37.777909,37.778329,37.776464,37.774599,37.774181,37.773973,37.773757,37.77749]}]],[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.408171,-122.407272,-122.406665,-122.4061,-122.403071,-122.402103,-122.400435,-122.399615,-122.398863,-122.398515,-122.398328,-122.400455,-122.402546,-122.402154,-122.402188,-122.403693,-122.406189,-122.410433,-122.408171],&#34;lat&#34;:[37.716157,37.71782,37.719215,37.719837,37.718465,37.718721,37.719162,37.716278,37.718683,37.717664,37.716259,37.713118,37.712266,37.712656,37.714731,37.714341,37.715244,37.716414,37.716157]}]],[[{&#34;lng&#34;:[-122.472672,-122.471457,-122.468017,-122.468036,-122.466235,-122.46264,-122.462615,-122.462602,-122.462569,-122.462556,-122.460858,-122.465994,-122.468939,-122.469236,-122.471319,-122.471215,-122.471355,-122.471472,-122.472672],&#34;lat&#34;:[37.717248,37.716085,37.716103,37.7179,37.717912,37.717936,37.715619,37.714308,37.71315,37.711254,37.710586,37.710154,37.708232,37.708232,37.708305,37.708939,37.712802,37.713427,37.717248]}]],[[{&#34;lng&#34;:[-122.478714,-122.472583,-122.472516,-122.472379,-122.472264,-122.47831,-122.478444,-122.478579,-122.478714],&#34;lat&#34;:[37.786031,37.786317,37.784453,37.782581,37.780747,37.780449,37.782308,37.784174,37.786031]}]],[[{&#34;lng&#34;:[-122.408431,-122.405513,-122.402041,-122.400681,-122.400149,-122.404613,-122.404796,-122.407903,-122.408253,-122.408431],&#34;lat&#34;:[37.796704,37.797065,37.797505,37.796777,37.794134,37.793565,37.794453,37.79406,37.795824,37.796704]}]],[[{&#34;lng&#34;:[-122.423481,-122.421768,-122.416839,-122.416653,-122.416473,-122.416292,-122.416113,-122.422801,-122.423143,-122.423283,-122.423481],&#34;lat&#34;:[37.795739,37.795945,37.796596,37.79567,37.794777,37.793892,37.793014,37.792161,37.793923,37.79481,37.795739]}]],[[{&#34;lng&#34;:[-122.407903,-122.404796,-122.404613,-122.404418,-122.407536,-122.407903],&#34;lat&#34;:[37.79406,37.794453,37.793565,37.792639,37.792249,37.79406]}]],[[{&#34;lng&#34;:[-122.421461,-122.418271,-122.417901,-122.417707,-122.421069,-122.421256,-122.421461],&#34;lat&#34;:[37.786632,37.787038,37.785167,37.784236,37.783816,37.784745,37.786632]}]],[[{&#34;lng&#34;:[-122.41443,-122.412782,-122.41114,-122.410952,-122.410765,-122.414054,-122.414242,-122.41443],&#34;lat&#34;:[37.784657,37.784866,37.785075,37.784141,37.783214,37.782794,37.783724,37.784657]}]],[[{&#34;lng&#34;:[-122.430045,-122.425209,-122.424835,-122.424464,-122.42929,-122.429667,-122.430045],&#34;lat&#34;:[37.803518,37.804133,37.802268,37.800402,37.799788,37.801654,37.803518]}]],[[{&#34;lng&#34;:[-122.453902,-122.4506,-122.445717,-122.440735,-122.440358,-122.443312,-122.442765,-122.44839,-122.450522,-122.453352,-122.453902],&#34;lat&#34;:[37.771054,37.77146,37.77208,37.772713,37.770847,37.77047,37.769661,37.768865,37.768593,37.768246,37.771054]}]],[[{&#34;lng&#34;:[-122.449433,-122.448274,-122.447635,-122.448013,-122.44839,-122.442765,-122.443356,-122.443866,-122.443533,-122.442575,-122.443347,-122.443538,-122.446247,-122.446384,-122.446783,-122.44912,-122.449433],&#34;lat&#34;:[37.7632,37.764196,37.765139,37.767007,37.768865,37.769661,37.768944,37.767536,37.766691,37.765928,37.765333,37.765323,37.762247,37.761816,37.761781,37.761648,37.7632]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.44462,-122.443421,-122.443255,-122.438474,-122.43409,-122.433769,-122.433697,-122.435916,-122.43809,-122.438246,-122.441408,-122.443961,-122.444556,-122.44462],&#34;lat&#34;:[37.747081,37.750099,37.750769,37.751024,37.751304,37.748097,37.747297,37.747179,37.74705,37.748665,37.74861,37.746765,37.746687,37.747081]}]],[[{&#34;lng&#34;:[-122.40648,-122.405105,-122.40509,-122.404497,-122.398703,-122.398461,-122.398338,-122.398216,-122.39797,-122.403689,-122.406389,-122.40648],&#34;lat&#34;:[37.760731,37.763852,37.764628,37.764664,37.765014,37.762463,37.761184,37.759909,37.757359,37.757015,37.759804,37.760731]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.421703,-122.415939,-122.414388,-122.409798,-122.408779,-122.408217,-122.406853,-122.406551,-122.406295,-122.405608,-122.408509,-122.40756,-122.410519,-122.41446,-122.415193,-122.416637,-122.419486,-122.419514,-122.421703],&#34;lat&#34;:[37.731807,37.732072,37.732417,37.734828,37.735817,37.737612,37.73805,37.735721,37.735307,37.732439,37.731552,37.729261,37.728488,37.727457,37.729223,37.728809,37.729046,37.73149,37.731807]}]],[[{&#34;lng&#34;:[-122.440141,-122.437502,-122.43715,-122.433985,-122.432385,-122.433555,-122.431154,-122.432777,-122.437046,-122.440418,-122.440141],&#34;lat&#34;:[37.717692,37.721176,37.721575,37.720063,37.719298,37.717744,37.716594,37.714437,37.716008,37.717245,37.717692]}]],[[{&#34;lng&#34;:[-122.453394,-122.453411,-122.453425,-122.448873,-122.439752,-122.438816,-122.438392,-122.435825,-122.433104,-122.432422,-122.435892,-122.439558,-122.439483,-122.443075,-122.443664,-122.446308,-122.447878,-122.449546,-122.451596,-122.452335,-122.452382,-122.454002,-122.453394],&#34;lat&#34;:[37.729837,37.731568,37.73304,37.733069,37.733111,37.733087,37.73442,37.733924,37.733419,37.733176,37.731577,37.730191,37.730112,37.728634,37.728304,37.725978,37.723005,37.723011,37.723033,37.72318,37.727868,37.729166,37.729837]}]],[[{&#34;lng&#34;:[-122.486906,-122.484763,-122.484627,-122.484496,-122.484365,-122.47687,-122.476699,-122.4766,-122.479817,-122.484104,-122.486248,-122.486382,-122.486512,-122.48664,-122.48677,-122.486906],&#34;lat&#34;:[37.765054,37.76516,37.763231,37.76137,37.759501,37.759832,37.757968,37.756108,37.755966,37.755777,37.755683,37.757542,37.759407,37.761276,37.763137,37.765054]}]],[[{&#34;lng&#34;:[-122.486179,-122.485302,-122.48193,-122.480015,-122.474972,-122.474634,-122.474745,-122.475347,-122.478988,-122.479631,-122.484909,-122.483811,-122.486606,-122.486179],&#34;lat&#34;:[37.730879,37.730946,37.731172,37.731148,37.731072,37.728992,37.726936,37.720914,37.720844,37.719663,37.724032,37.727029,37.729555,37.730879]}]],[[{&#34;lng&#34;:[-122.50987,-122.502979,-122.502844,-122.502713,-122.502582,-122.509111,-122.509382,-122.509622,-122.50987],&#34;lat&#34;:[37.76409,37.764344,37.762427,37.760567,37.7587,37.758414,37.760272,37.762127,37.76409]}]],[[{&#34;lng&#34;:[-122.48408,-122.478714,-122.478579,-122.478444,-122.47831,-122.479388,-122.482598,-122.483667,-122.483804,-122.483936,-122.48408],&#34;lat&#34;:[37.785795,37.786031,37.784174,37.782308,37.780449,37.780403,37.780265,37.780207,37.782077,37.783931,37.785795]}]],[[{&#34;lng&#34;:[-122.497455,-122.493168,-122.493305,-122.491172,-122.489019,-122.488876,-122.488741,-122.488606,-122.488471,-122.494903,-122.495039,-122.495175,-122.497319,-122.497455],&#34;lat&#34;:[37.777579,37.777774,37.779752,37.779869,37.779977,37.777972,37.776107,37.774241,37.772374,37.772081,37.773947,37.775813,37.775715,37.777579]}]],[[{&#34;lng&#34;:[-122.424478,-122.422811,-122.422241,-122.421475,-122.420908,-122.415635,-122.416,-122.416648,-122.419348,-122.420083,-122.420082,-122.42389,-122.423377,-122.424478],&#34;lat&#34;:[37.71011,37.710511,37.711317,37.712201,37.713774,37.712733,37.711915,37.710473,37.709947,37.708311,37.708231,37.708236,37.709203,37.71011]}]],[[{&#34;lng&#34;:[-122.510781,-122.507484,-122.509842514521,-122.50531,-122.493048716406,-122.493338,-122.493491,-122.493446,-122.498801,-122.502012,-122.505226,-122.506509,-122.509879,-122.510781],&#34;lat&#34;:[37.782059,37.782185,37.7846145393916,37.788312,37.7879341073778,37.787403,37.783498,37.781627,37.781377,37.781227,37.781077,37.780857,37.780627,37.782059]}]],[[{&#34;lng&#34;:[-122.417218,-122.412283,-122.411911,-122.411719,-122.411364,-122.415218,-122.416292,-122.416473,-122.416653,-122.416839,-122.417218],&#34;lat&#34;:[37.798472,37.799096,37.797245,37.796291,37.794527,37.794029,37.793892,37.794777,37.79567,37.796596,37.798472]}]],[[{&#34;lng&#34;:[-122.443183,-122.438206,-122.436557,-122.434918,-122.431635,-122.428336,-122.425047,-122.424704,-122.431291,-122.434562,-122.437863,-122.438901,-122.44281,-122.443183],&#34;lat&#34;:[37.784827,37.785459,37.785668,37.785876,37.786293,37.786713,37.78713,37.785418,37.784621,37.784152,37.783791,37.78345,37.782971,37.784827]}]],[[{&#34;lng&#34;:[-122.442525,-122.440345,-122.438864,-122.437769,-122.434546,-122.434317,-122.43424,-122.43409,-122.438474,-122.443255,-122.442525],&#34;lat&#34;:[37.75238,37.755228,37.755434,37.755541,37.756093,37.75369,37.752891,37.751304,37.751024,37.750769,37.75238]}]],[[{&#34;lng&#34;:[-122.402105,-122.399867,-122.396409,-122.392699,-122.393284,-122.396682,-122.397025,-122.398515,-122.398863,-122.400785,-122.401183,-122.402105],&#34;lat&#34;:[37.72779,37.730292,37.729801,37.729284,37.727764,37.720266,37.719515,37.717664,37.718683,37.723616,37.724588,37.72779]}]],[[{&#34;lng&#34;:[-122.448582,-122.447878,-122.44473,-122.444486,-122.438787,-122.43702,-122.435385,-122.43715,-122.437502,-122.440141,-122.440418,-122.440999,-122.441896,-122.443706,-122.446945,-122.44796,-122.448727,-122.448582],&#34;lat&#34;:[37.718411,37.723005,37.722987,37.722974,37.723637,37.723723,37.723909,37.721575,37.721176,37.717692,37.717245,37.716488,37.715308,37.713592,37.716187,37.717,37.71745,37.718411]}]],[[{&#34;lng&#34;:[-122.485727,-122.479296,-122.479426,-122.476221,-122.476131,-122.476007,-122.475854,-122.475726,-122.485335,-122.485466,-122.485597,-122.485727],&#34;lat&#34;:[37.74822,37.748504,37.750368,37.750509,37.748643,37.746778,37.744915,37.743049,37.742626,37.744493,37.746354,37.74822]}]],[[{&#34;lng&#34;:[-122.488741,-122.482318,-122.479106,-122.478972,-122.478837,-122.482049,-122.488471,-122.488606,-122.488741],&#34;lat&#34;:[37.776107,37.7764,37.776546,37.774681,37.772815,37.772667,37.772374,37.774241,37.776107]}]],[[{&#34;lng&#34;:[-122.41575,-122.412461,-122.412266,-122.412076,-122.415372,-122.415559,-122.41575],&#34;lat&#34;:[37.791206,37.791627,37.790673,37.78974,37.78932,37.790254,37.791206]}]],[[{&#34;lng&#34;:[-122.437326,-122.435678,-122.434036,-122.432393,-122.430747,-122.427457,-122.424275,-122.423705,-122.426893,-122.430183,-122.433469,-122.435113,-122.43695,-122.437326],&#34;lat&#34;:[37.797811,37.798021,37.79823,37.798439,37.798648,37.799067,37.799472,37.796678,37.796272,37.795854,37.795435,37.795226,37.795949,37.797811]}]],[[{&#34;lng&#34;:[-122.455217,-122.453385,-122.454186,-122.447302,-122.445826,-122.442793,-122.441512,-122.440958,-122.442053,-122.441866,-122.441488,-122.44478,-122.446471,-122.448068,-122.451358,-122.454683,-122.455217],&#34;lat&#34;:[37.777599,37.777827,37.781651,37.782392,37.782512,37.782885,37.782096,37.779364,37.779244,37.7783,37.776435,37.776017,37.775802,37.775599,37.775175,37.774755,37.777599]}]],[[{&#34;lng&#34;:[-122.398703,-122.393848,-122.392714,-122.39252,-122.392319,-122.392494,-122.398216,-122.398338,-122.398461,-122.398703],&#34;lat&#34;:[37.765014,37.765307,37.764084,37.762819,37.76106,37.760245,37.759909,37.761184,37.762463,37.765014]}]],[[{&#34;lng&#34;:[-122.502979,-122.496085,-122.49595,-122.495822,-122.495692,-122.495555,-122.495433,-122.495307,-122.495165,-122.499914,-122.499784,-122.50193,-122.50206,-122.502191,-122.502318,-122.50245,-122.502582,-122.502713,-122.502844,-122.502979],&#34;lat&#34;:[37.764344,37.764675,37.762732,37.760871,37.759001,37.757137,37.755273,37.75341,37.751543,37.751334,37.749472,37.749377,37.75124,37.753106,37.754953,37.756835,37.7587,37.760567,37.762427,37.764344]}]],[[{&#34;lng&#34;:[-122.514483,-122.509842514521,-122.507484,-122.510781,-122.509879,-122.509593,-122.509228,-122.506022,-122.501733,-122.498521,-122.498386,-122.49825,-122.498114,-122.505611,-122.511124,-122.513067688424,-122.513128,-122.514483],&#34;lat&#34;:[37.780829,37.7846145393916,37.782185,37.782059,37.780627,37.780173,37.777044,37.77719,37.777385,37.777531,37.775667,37.7738,37.771934,37.771592,37.771323,37.7707830331714,37.771214,37.780829]}]],[[{&#34;lng&#34;:[-122.415372,-122.412076,-122.408786,-122.408402,-122.411699,-122.413354,-122.414995,-122.415185,-122.415372],&#34;lat&#34;:[37.78932,37.78974,37.79016,37.788293,37.787872,37.787664,37.787454,37.788388,37.78932]}]],[[{&#34;lng&#34;:[-122.447466,-122.441177,-122.436274,-122.435678,-122.437326,-122.43695,-122.438591,-122.440234,-122.446728,-122.447015,-122.447303,-122.447466],&#34;lat&#34;:[37.79939,37.80019,37.800814,37.798021,37.797811,37.795949,37.795741,37.795533,37.794706,37.796579,37.798459,37.79939]}]],[[{&#34;lng&#34;:[-122.400022,-122.398488,-122.396515,-122.392776,-122.393904,-122.391617,-122.391891,-122.392699,-122.396409,-122.399867,-122.399423,-122.400022],&#34;lat&#34;:[37.732311,37.733808,37.737156,37.735038,37.733788,37.732487,37.731684,37.729284,37.729801,37.730292,37.731703,37.732311]}]],[[{&#34;lng&#34;:[-122.496085,-122.493336,-122.486906,-122.48677,-122.48664,-122.486512,-122.486382,-122.486248,-122.492675,-122.495433,-122.495555,-122.495692,-122.495822,-122.49595,-122.496085],&#34;lat&#34;:[37.764675,37.764774,37.765054,37.763137,37.761276,37.759407,37.757542,37.755683,37.755398,37.755273,37.757137,37.759001,37.760871,37.762732,37.764675]}]],[[{&#34;lng&#34;:[-122.489019,-122.483667,-122.482598,-122.479388,-122.479241,-122.479106,-122.482318,-122.488741,-122.488876,-122.489019],&#34;lat&#34;:[37.779977,37.780207,37.780265,37.780403,37.778408,37.776546,37.7764,37.776107,37.777972,37.779977]}]],[[{&#34;lng&#34;:[-122.418718,-122.416888,-122.414877,-122.412908,-122.412849,-122.412471,-122.412283,-122.417218,-122.417385,-122.417587,-122.417799,-122.418281,-122.418718],&#34;lat&#34;:[37.805932,37.805212,37.803542,37.802184,37.801893,37.800032,37.799096,37.798472,37.799414,37.800341,37.801267,37.804023,37.805932]}]],[[{&#34;lng&#34;:[-122.422801,-122.416113,-122.415936,-122.41575,-122.415559,-122.415372,-122.422077,-122.422237,-122.42242,-122.422611,-122.422801],&#34;lat&#34;:[37.792161,37.793014,37.792133,37.791206,37.790254,37.78932,37.788474,37.789399,37.790358,37.791277,37.792161]}]],[[{&#34;lng&#34;:[-122.411719,-122.408431,-122.408253,-122.407903,-122.407536,-122.409171,-122.409541,-122.409722,-122.411364,-122.411719],&#34;lat&#34;:[37.796291,37.796704,37.795824,37.79406,37.792249,37.792043,37.793852,37.794737,37.794527,37.796291]}]],[[{&#34;lng&#34;:[-122.414995,-122.413354,-122.411699,-122.408402,-122.408227,-122.411516,-122.411326,-122.41297,-122.414617,-122.414995],&#34;lat&#34;:[37.787454,37.787664,37.787872,37.788293,37.787359,37.78694,37.785996,37.78579,37.785582,37.787454]}]],[[{&#34;lng&#34;:[-122.421069,-122.417707,-122.417339,-122.417146,-122.415505,-122.413866,-122.413365,-122.416292,-122.417615,-122.418704,-122.419219,-122.419334,-122.419359,-122.419765,-122.420358,-122.420689,-122.421069],&#34;lat&#34;:[37.783816,37.784236,37.782379,37.781447,37.781654,37.781863,37.779862,37.777494,37.7766,37.775645,37.775316,37.77521,37.775432,37.777291,37.780072,37.781955,37.783816]}]],[[{&#34;lng&#34;:[-122.436799,-122.431679,-122.430045,-122.429667,-122.42929,-122.424464,-122.424275,-122.427457,-122.430747,-122.432393,-122.434036,-122.435678,-122.436274,-122.436799],&#34;lat&#34;:[37.802715,37.803267,37.803518,37.801654,37.799788,37.800402,37.799472,37.799067,37.798648,37.798439,37.79823,37.798021,37.800814,37.802715]}]],[[{&#34;lng&#34;:[-122.430183,-122.426893,-122.426335,-122.425802,-122.422611,-122.42242,-122.425616,-122.428905,-122.42927,-122.429625,-122.430183],&#34;lat&#34;:[37.795854,37.796272,37.793517,37.790874,37.791277,37.790358,37.789952,37.789534,37.791339,37.793098,37.795854]}]],[[{&#34;lng&#34;:[-122.446728,-122.440234,-122.438591,-122.43695,-122.435113,-122.433469,-122.430183,-122.429625,-122.434558,-122.436204,-122.437846,-122.439488,-122.446117,-122.4463,-122.446728],&#34;lat&#34;:[37.794706,37.795533,37.795741,37.795949,37.795226,37.795435,37.795854,37.793098,37.792472,37.792263,37.792054,37.791843,37.790999,37.791879,37.794706]}]],[[{&#34;lng&#34;:[-122.446117,-122.439488,-122.437846,-122.436204,-122.435849,-122.435486,-122.438773,-122.440488,-122.443753,-122.445402,-122.445589,-122.445758,-122.445934,-122.446117],&#34;lat&#34;:[37.790999,37.791843,37.792054,37.792263,37.790502,37.788697,37.788278,37.788061,37.787647,37.787438,37.788364,37.789248,37.790119,37.790999]}]],[[{&#34;lng&#34;:[-122.443753,-122.440488,-122.438773,-122.435486,-122.434918,-122.436557,-122.438206,-122.443183,-122.44337,-122.443753],&#34;lat&#34;:[37.787647,37.788061,37.788278,37.788697,37.785876,37.785668,37.785459,37.784827,37.785759,37.787647]}]],[[{&#34;lng&#34;:[-122.459179,-122.45505,-122.454774,-122.450118,-122.445402,-122.443753,-122.44337,-122.443183,-122.44281,-122.442793,-122.445826,-122.447302,-122.454186,-122.456304,-122.458878,-122.458998,-122.459179],&#34;lat&#34;:[37.785682,37.786209,37.786244,37.786838,37.787438,37.787647,37.785759,37.784827,37.782971,37.782885,37.782512,37.782392,37.781651,37.781449,37.781326,37.783189,37.785682]}]],[[{&#34;lng&#34;:[-122.44281,-122.438901,-122.437863,-122.434562,-122.433973,-122.433596,-122.433407,-122.436697,-122.437072,-122.440958,-122.441512,-122.442793,-122.44281],&#34;lat&#34;:[37.782971,37.78345,37.783791,37.784152,37.781219,37.779355,37.778422,37.778006,37.779867,37.779364,37.782096,37.782885,37.782971]}]],[[{&#34;lng&#34;:[-122.433973,-122.430696,-122.427396,-122.424108,-122.423541,-122.423354,-122.424998,-122.426641,-122.429929,-122.433219,-122.433407,-122.433596,-122.433973],&#34;lat&#34;:[37.781219,37.781635,37.782057,37.782477,37.779674,37.778747,37.778538,37.778329,37.777909,37.77749,37.778422,37.779355,37.781219]}]],[[{&#34;lng&#34;:[-122.426641,-122.424998,-122.423354,-122.423541,-122.420358,-122.419765,-122.419359,-122.424246,-122.425888,-122.426263,-122.426641],&#34;lat&#34;:[37.778329,37.778538,37.778747,37.779674,37.780072,37.777291,37.775432,37.774811,37.774599,37.776464,37.778329]}]],[[{&#34;lng&#34;:[-122.435794,-122.433572,-122.43157,-122.42822,-122.426402,-122.426948,-122.428635,-122.428949,-122.430824,-122.435193,-122.435472,-122.435794],&#34;lat&#34;:[37.769058,37.769178,37.769307,37.769441,37.769596,37.769175,37.767846,37.767504,37.766014,37.762727,37.765727,37.769058]}]],[[{&#34;lng&#34;:[-122.413161,-122.411606,-122.409387,-122.407159,-122.405719,-122.404933,-122.403387,-122.401843,-122.40407,-122.408516,-122.41248,-122.415561,-122.413161],&#34;lat&#34;:[37.774992,37.776221,37.777977,37.779739,37.780878,37.7815,37.780265,37.779032,37.777273,37.77376,37.77063,37.773101,37.774992]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.431477,-122.427045,-122.42485,-122.421393,-122.421704,-122.422174,-122.422709,-122.423361,-122.430888,-122.431022,-122.431256,-122.431477],&#34;lat&#34;:[37.747434,37.747696,37.747825,37.748124,37.746412,37.745614,37.74394,37.74229,37.741833,37.742631,37.745032,37.747434]}]],[[{&#34;lng&#34;:[-122.45084,-122.449363,-122.447399,-122.445679,-122.444148,-122.440049,-122.43556,-122.435559,-122.43332,-122.433259,-122.430888,-122.430741,-122.431759,-122.428726,-122.431349,-122.432443,-122.434697,-122.434401,-122.435825,-122.438392,-122.439915,-122.443829,-122.443649,-122.442489,-122.442525,-122.445987,-122.446658,-122.449861,-122.451692,-122.45084],&#34;lat&#34;:[37.745904,37.746129,37.746596,37.744139,37.743297,37.745299,37.74155,37.743161,37.743296,37.741903,37.741833,37.740424,37.739803,37.738326,37.736618,37.735879,37.737256,37.736199,37.733924,37.73442,37.734901,37.736134,37.736519,37.737289,37.739448,37.74118,37.742442,37.743265,37.745629,37.745904]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.387399,-122.38717,-122.388529,-122.385282,-122.381558,-122.384047,-122.382919,-122.380186,-122.381284,-122.379932,-122.379779,-122.381439,-122.38209,-122.383431,-122.385287,-122.389035,-122.387399],&#34;lat&#34;:[37.734738,37.735698,37.736475,37.74024,37.738123,37.737514,37.734511,37.73266,37.734034,37.733381,37.730799,37.730582,37.730635,37.729738,37.730791,37.732919,37.734738]}]],[[{&#34;lng&#34;:[-122.392699,-122.391891,-122.391617,-122.39106,-122.389035,-122.385287,-122.383431,-122.379697,-122.377833,-122.382176,-122.385024,-122.386899,-122.384517,-122.38521,-122.38793,-122.389801,-122.393284,-122.392699],&#34;lat&#34;:[37.729284,37.731684,37.732487,37.734066,37.732919,37.730791,37.729738,37.727618,37.72656,37.722246,37.724098,37.724289,37.722862,37.723088,37.724729,37.725791,37.727764,37.729284]}]],[[{&#34;lng&#34;:[-122.431294,-122.430199,-122.430165,-122.429304,-122.428038,-122.426104,-122.423982,-122.421703,-122.419514,-122.419486,-122.416637,-122.415193,-122.41446,-122.418404,-122.418888,-122.419875,-122.421847,-122.423336,-122.424199,-122.43039,-122.431294],&#34;lat&#34;:[37.72873,37.729872,37.729851,37.730742,37.732016,37.731651,37.731552,37.731807,37.73149,37.729046,37.728809,37.729223,37.727457,37.726425,37.724966,37.72471,37.724199,37.727792,37.728736,37.728547,37.72873]}]],[[{&#34;lng&#34;:[-122.410519,-122.40756,-122.408509,-122.405608,-122.406295,-122.40506,-122.404657,-122.402752,-122.402105,-122.401183,-122.400785,-122.402061,-122.4026,-122.405572,-122.408529,-122.410519],&#34;lat&#34;:[37.728488,37.729261,37.731552,37.732439,37.735307,37.733652,37.732949,37.729357,37.72779,37.724588,37.723616,37.72392,37.725235,37.724461,37.723688,37.728488]}]],[[{&#34;lng&#34;:[-122.432385,-122.431205,-122.430026,-122.428837,-122.428074,-122.424848,-122.424555,-122.42639,-122.42667,-122.427189,-122.431154,-122.433555,-122.432385],&#34;lat&#34;:[37.719298,37.720856,37.722398,37.723955,37.723591,37.722053,37.721648,37.718561,37.717078,37.715748,37.716594,37.717744,37.719298]}]],[[{&#34;lng&#34;:[-122.465994,-122.460858,-122.457312,-122.456376,-122.455619,-122.455028,-122.452884,-122.449627,-122.448727,-122.44796,-122.446945,-122.443706,-122.444571,-122.446091,-122.449309,-122.452665,-122.453983,-122.458509,-122.45905,-122.468939,-122.465994],&#34;lat&#34;:[37.710154,37.710586,37.710648,37.710825,37.711066,37.711317,37.712829,37.715739,37.71745,37.717,37.716187,37.713592,37.712783,37.711592,37.709946,37.708806,37.708232,37.708231,37.708231,37.708232,37.710154]}]],[[{&#34;lng&#34;:[-122.476762,-122.470335,-122.469263,-122.463912,-122.463782,-122.463658,-122.46377,-122.463361,-122.463433,-122.464462,-122.4666,-122.468335,-122.469782,-122.47068,-122.470908,-122.47121,-122.473683,-122.4766,-122.476699,-122.47687,-122.476762],&#34;lat&#34;:[37.759837,37.760124,37.760171,37.760405,37.758538,37.756764,37.754142,37.753039,37.753028,37.752899,37.752803,37.753244,37.752675,37.752376,37.752616,37.755194,37.756237,37.756108,37.757968,37.759832,37.759837]}]],[[{&#34;lng&#34;:[-122.46147,-122.456091,-122.455537,-122.454172,-122.454078,-122.453872,-122.451367,-122.448535,-122.447915,-122.44695,-122.445348,-122.447943,-122.446734,-122.44709,-122.449423,-122.445697,-122.447794,-122.449447,-122.45084,-122.451692,-122.453897,-122.453959,-122.454275,-122.454637,-122.458743,-122.459174,-122.458635,-122.46147],&#34;lat&#34;:[37.751522,37.751804,37.753774,37.756507,37.757027,37.757378,37.758576,37.759045,37.757589,37.75655,37.755619,37.755532,37.753573,37.751505,37.750412,37.748855,37.749408,37.74666,37.745904,37.745629,37.745742,37.745765,37.745908,37.746142,37.746876,37.747286,37.747769,37.751522]}]],[[{&#34;lng&#34;:[-122.459592,-122.458849,-122.458082,-122.454932,-122.45464,-122.454238,-122.453773,-122.452306,-122.449861,-122.446658,-122.445987,-122.442525,-122.442489,-122.443649,-122.443829,-122.439915,-122.438392,-122.438816,-122.439752,-122.448873,-122.453425,-122.456156,-122.459505,-122.460282,-122.459592],&#34;lat&#34;:[37.738533,37.739158,37.739099,37.741777,37.742326,37.742835,37.743219,37.742819,37.743265,37.742442,37.74118,37.739448,37.737289,37.736519,37.736134,37.734901,37.73442,37.733087,37.733111,37.733069,37.73304,37.734212,37.734291,37.737855,37.738533]}]],[[{&#34;lng&#34;:[-122.484763,-122.477261,-122.477116,-122.476974,-122.476762,-122.47687,-122.484365,-122.484496,-122.484627,-122.484763],&#34;lat&#34;:[37.76516,37.765486,37.763562,37.761701,37.759837,37.759832,37.759501,37.76137,37.763231,37.76516]}]],[[{&#34;lng&#34;:[-122.485124,-122.484832,-122.483143,-122.483205,-122.479981,-122.478674,-122.47724,-122.475818,-122.474383,-122.474528,-122.472672,-122.471472,-122.471355,-122.472269,-122.477016,-122.485275,-122.485124],&#34;lat&#34;:[37.718575,37.718571,37.718517,37.717585,37.716942,37.717903,37.717709,37.716891,37.717507,37.719065,37.717248,37.713427,37.712802,37.712838,37.714593,37.714823,37.718575]}]],[[{&#34;lng&#34;:[-122.464781,-122.463581,-122.459476,-122.459388,-122.459179,-122.458998,-122.458878,-122.461068,-122.464284,-122.464551,-122.464781],&#34;lat&#34;:[37.788564,37.788806,37.789711,37.788405,37.785682,37.783189,37.781326,37.781227,37.781089,37.784792,37.788564]}]],[[{&#34;lng&#34;:[-122.493338,-122.493048716406,-122.492883,-122.485827333182,-122.483985,-122.483799,-122.479845,-122.472352,-122.472583,-122.478714,-122.48408,-122.487286,-122.487151,-122.490364,-122.493491,-122.493338],&#34;lat&#34;:[37.787403,37.7879341073778,37.787929,37.7906121409026,37.789769,37.78774,37.786796,37.787234,37.786317,37.786031,37.785795,37.785649,37.783785,37.78364,37.783498,37.787403]}]],[[{&#34;lng&#34;:[-122.479388,-122.47831,-122.472264,-122.472103,-122.471967,-122.471842,-122.471706,-122.478837,-122.478972,-122.479106,-122.479241,-122.479388],&#34;lat&#34;:[37.780403,37.780449,37.780747,37.778731,37.776871,37.775006,37.773141,37.772815,37.774681,37.776546,37.778408,37.780403]}]],[[{&#34;lng&#34;:[-122.485783,-122.480296,-122.478083,-122.470336,-122.463793,-122.448241130218,-122.448241,-122.44946,-122.44782,-122.447466,-122.447303,-122.447015,-122.446728,-122.4463,-122.45274,-122.459476,-122.463581,-122.464781,-122.466924,-122.470912,-122.472352,-122.479845,-122.483799,-122.483985,-122.485827333182,-122.485783],&#34;lat&#34;:[37.790629,37.801293,37.810828,37.808671,37.804653,37.8072521685356,37.804478,37.802454,37.801633,37.79939,37.798459,37.796579,37.794706,37.791879,37.790974,37.789711,37.788806,37.788564,37.788132,37.787221,37.787234,37.786796,37.78774,37.789769,37.7906121409026,37.790629]}]],[[{&#34;lng&#34;:[-122.40499,-122.404391,-122.401657,-122.399433,-122.400982,-122.398755,-122.397206,-122.39275,-122.390527,-122.389168,-122.388423,-122.387421,-122.383505768697,-122.381232,-122.380462169293,-122.385823,-122.387917,-122.388164,-122.393968,-122.393848,-122.398703,-122.404497,-122.40499],&#34;lat&#34;:[37.769715,37.76975,37.771817,37.773573,37.774809,37.776568,37.775332,37.778852,37.780607,37.781649,37.781798,37.78279,37.7830792797042,37.773514,37.7675163525902,37.767255,37.764379,37.766958,37.766614,37.765307,37.765014,37.764664,37.769715]}]],[[{&#34;lng&#34;:[-122.513067688424,-122.511124,-122.505611,-122.498114,-122.494903,-122.488471,-122.482049,-122.478837,-122.471706,-122.466952,-122.465332,-122.463749,-122.460553,-122.458371,-122.454683,-122.454083,-122.450787,-122.447497,-122.444216,-122.440922,-122.440735,-122.445717,-122.4506,-122.453902,-122.453352,-122.452947,-122.456913,-122.45779,-122.458405,-122.46109,-122.464297,-122.47073,-122.477261,-122.484763,-122.486906,-122.493336,-122.496085,-122.502979,-122.50987,-122.512070759187,-122.513067688424],&#34;lat&#34;:[37.7707830331714,37.771323,37.771592,37.771934,37.772081,37.772374,37.772667,37.772815,37.773141,37.773351,37.773423,37.773624,37.774031,37.774309,37.774755,37.77193,37.772368,37.77279,37.773212,37.773635,37.772713,37.77208,37.77146,37.771054,37.768246,37.766374,37.765874,37.766015,37.76616,37.766132,37.765963,37.765779,37.765486,37.76516,37.765054,37.764774,37.764675,37.764344,37.76409,37.7636593024008,37.7707830331714]}]],[[{&#34;lng&#34;:[-122.426722,-122.422688,-122.422121,-122.420224,-122.420163,-122.41676,-122.416824,-122.414313,-122.412447,-122.410513,-122.409798,-122.414388,-122.415939,-122.421703,-122.423982,-122.426104,-122.428038,-122.426158,-122.428573,-122.426722],&#34;lat&#34;:[37.736372,37.735205,37.735153,37.735058,37.735777,37.735605,37.734888,37.734763,37.73467,37.734592,37.734828,37.732417,37.732072,37.731807,37.731552,37.731651,37.732016,37.733971,37.735082,37.736372]}]],[[{&#34;lng&#34;:[-122.418888,-122.418404,-122.41446,-122.410519,-122.408529,-122.405572,-122.404059,-122.403078,-122.402103,-122.403071,-122.4061,-122.406665,-122.407036,-122.41099,-122.412468,-122.412969,-122.416632,-122.419875,-122.418888],&#34;lat&#34;:[37.724966,37.726425,37.727457,37.728488,37.723688,37.724461,37.720864,37.72113,37.718721,37.718465,37.719837,37.719215,37.720085,37.719074,37.722658,37.723858,37.722157,37.72471,37.724966]}]],[[{&#34;lng&#34;:[-122.431801,-122.431294,-122.43039,-122.424199,-122.423336,-122.421847,-122.423807,-122.424848,-122.428074,-122.426912,-122.432447,-122.431801],&#34;lat&#34;:[37.728249,37.72873,37.728547,37.728736,37.727792,37.724199,37.723691,37.722053,37.723591,37.725151,37.727637,37.728249]}]],[[{&#34;lng&#34;:[-122.444571,-122.443706,-122.441896,-122.440999,-122.440418,-122.437046,-122.432777,-122.433682,-122.436144,-122.436203,-122.436642,-122.437811,-122.443335,-122.444097,-122.444571],&#34;lat&#34;:[37.712783,37.713592,37.715308,37.716488,37.717245,37.716008,37.714437,37.713181,37.714254,37.714171,37.71355,37.71192,37.710555,37.711689,37.712783]}]],[[{&#34;lng&#34;:[-122.452665,-122.449309,-122.446091,-122.444571,-122.444097,-122.443335,-122.437811,-122.43837,-122.436993,-122.436771,-122.435382,-122.440781,-122.442079,-122.444482,-122.445875,-122.449751,-122.452183,-122.453983,-122.452665],&#34;lat&#34;:[37.708806,37.709946,37.711592,37.712783,37.711689,37.710555,37.71192,37.711133,37.710746,37.710156,37.708132,37.708304,37.708231,37.708332,37.708233,37.708213,37.708132,37.708232,37.708806]}]],[[{&#34;lng&#34;:[-122.463782,-122.463226,-122.462972,-122.460843,-122.457536,-122.455466,-122.452569,-122.4524,-122.451958,-122.44912,-122.446783,-122.446394,-122.446395,-122.447915,-122.448535,-122.451367,-122.453872,-122.454078,-122.454172,-122.455537,-122.456091,-122.46147,-122.462108,-122.463361,-122.46377,-122.463658,-122.463782],&#34;lat&#34;:[37.758538,37.758562,37.762316,37.762628,37.763566,37.764161,37.764509,37.763669,37.761478,37.761648,37.761781,37.760907,37.758856,37.757589,37.759045,37.758576,37.757378,37.757027,37.756507,37.753774,37.751804,37.751522,37.751972,37.753039,37.754142,37.756764,37.758538]}]],[[{&#34;lng&#34;:[-122.47073,-122.464297,-122.464172,-122.464042,-122.463912,-122.469263,-122.470335,-122.470468,-122.470595,-122.47073],&#34;lat&#34;:[37.765779,37.765963,37.764131,37.762269,37.760405,37.760171,37.760124,37.762027,37.763849,37.765779]}]],[[{&#34;lng&#34;:[-122.4766,-122.473683,-122.47121,-122.470908,-122.47068,-122.469782,-122.468335,-122.4666,-122.464462,-122.463442,-122.464186,-122.465403,-122.466469,-122.466336,-122.469549,-122.470595,-122.476131,-122.476221,-122.476355,-122.4766],&#34;lat&#34;:[37.756108,37.756237,37.755194,37.752616,37.752376,37.752675,37.753244,37.752803,37.752899,37.751195,37.750717,37.750986,37.750939,37.749075,37.748903,37.748869,37.748643,37.750509,37.752376,37.756108]}]],[[{&#34;lng&#34;:[-122.405174,-122.403495,-122.401787,-122.399638840473,-122.398139,-122.394407919349,-122.389708268207,-122.391487,-122.393181,-122.394748,-122.396315,-122.396586,-122.400149,-122.400681,-122.402041,-122.402421,-122.40279,-122.40316,-122.404798,-122.405174],&#34;lat&#34;:[37.804763,37.805089,37.805314,37.8065658502084,37.80563,37.8012904644048,37.7958244063587,37.793859,37.79323,37.79448,37.793337,37.794588,37.794134,37.796777,37.797505,37.799382,37.801242,37.803098,37.802901,37.804763]}]],[[{&#34;lng&#34;:[-122.412908,-122.411376,-122.409014,-122.407869,-122.407083,-122.406668,-122.405513,-122.408431,-122.411719,-122.411911,-122.412283,-122.412471,-122.412849,-122.412908],&#34;lat&#34;:[37.802184,37.801125,37.79949,37.798697,37.798152,37.797865,37.797065,37.796704,37.796291,37.797245,37.799096,37.800032,37.801893,37.802184]}]],[[{&#34;lng&#34;:[-122.416292,-122.415218,-122.411364,-122.409722,-122.409541,-122.409171,-122.412461,-122.41575,-122.415936,-122.416113,-122.416292],&#34;lat&#34;:[37.793892,37.794029,37.794527,37.794737,37.793852,37.792043,37.791627,37.791206,37.792133,37.793014,37.793892]}]],[[{&#34;lng&#34;:[-122.408786,-122.407149,-122.40734,-122.407536,-122.404418,-122.404613,-122.400149,-122.396586,-122.396315,-122.399195,-122.406062,-122.407451,-122.407853,-122.408227,-122.408402,-122.408786],&#34;lat&#34;:[37.79016,37.790366,37.791296,37.792249,37.792639,37.793565,37.794134,37.794588,37.793337,37.791058,37.785757,37.784604,37.785492,37.787359,37.788293,37.79016]}]],[[{&#34;lng&#34;:[-122.412461,-122.409171,-122.407536,-122.40734,-122.407149,-122.408786,-122.412076,-122.412266,-122.412461],&#34;lat&#34;:[37.791627,37.792043,37.792249,37.791296,37.790366,37.79016,37.78974,37.790673,37.791627]}]],[[{&#34;lng&#34;:[-122.414617,-122.41297,-122.411326,-122.411516,-122.408227,-122.407853,-122.41114,-122.412782,-122.41443,-122.414617],&#34;lat&#34;:[37.785582,37.78579,37.785996,37.78694,37.787359,37.785492,37.785075,37.784866,37.784657,37.785582]}]],[[{&#34;lng&#34;:[-122.417707,-122.416072,-122.41443,-122.414242,-122.414054,-122.413866,-122.415505,-122.417146,-122.417339,-122.417707],&#34;lat&#34;:[37.784236,37.784449,37.784657,37.783724,37.782794,37.781863,37.781654,37.781447,37.782379,37.784236]}]],[[{&#34;lng&#34;:[-122.441883,-122.439690935158,-122.425942,-122.425041162157,-122.424876,-122.426671,-122.426613,-122.426257,-122.425628,-122.425209,-122.430045,-122.431679,-122.436799,-122.437138,-122.441905,-122.441883],&#34;lat&#34;:[37.80731,37.8086811541357,37.810979,37.811001508766,37.810799,37.809639,37.808152,37.807185,37.806456,37.804133,37.803518,37.803267,37.802715,37.804352,37.803744,37.80731]}]],[[{&#34;lng&#34;:[-122.441488,-122.438244,-122.433219,-122.432467,-122.435751,-122.437469,-122.440735,-122.440922,-122.441488],&#34;lat&#34;:[37.776435,37.776848,37.77749,37.773757,37.773346,37.773127,37.772713,37.773635,37.776435]}]],[[{&#34;lng&#34;:[-122.443866,-122.443356,-122.442765,-122.443312,-122.440358,-122.440735,-122.437469,-122.435751,-122.432467,-122.431904,-122.43157,-122.433572,-122.435794,-122.436588,-122.438338,-122.439301,-122.442009,-122.442575,-122.443533,-122.443866],&#34;lat&#34;:[37.767536,37.768944,37.769661,37.77047,37.770847,37.772713,37.773127,37.773346,37.773757,37.770964,37.769307,37.769178,37.769058,37.769001,37.769007,37.768166,37.765907,37.765928,37.766691,37.767536]}]],[[{&#34;lng&#34;:[-122.417615,-122.416292,-122.413365,-122.412548,-122.410336,-122.408952,-122.408104,-122.407451,-122.406062,-122.402708,-122.404933,-122.405719,-122.407159,-122.409387,-122.411606,-122.413161,-122.415561,-122.418704,-122.417615],&#34;lat&#34;:[37.7766,37.777494,37.779862,37.780508,37.782245,37.783288,37.78402,37.784604,37.785757,37.783259,37.7815,37.780878,37.779739,37.777977,37.776221,37.774992,37.773101,37.775645,37.7766]}]],[[{&#34;lng&#34;:[-122.404933,-122.402708,-122.400468,-122.39893,-122.397386,-122.401843,-122.403387,-122.404933],&#34;lat&#34;:[37.7815,37.783259,37.78503,37.783785,37.782554,37.779032,37.780265,37.7815]}]],[[{&#34;lng&#34;:[-122.332045780727,-122.327561619753,-122.33227,-122.332045780727],&#34;lat&#34;:[37.7877603470098,37.7806441119025,37.781517,37.7877603470098]}],[{&#34;lng&#34;:[-122.374012,-122.367868,-122.369169789163,-122.36758,-122.368368114461,-122.372924,-122.374012],&#34;lat&#34;:[37.739499,37.739994,37.7417391770183,37.740214,37.73825770119,37.738348,37.739499]}],[{&#34;lng&#34;:[-122.375574,-122.374893,-122.37510014051,-122.372855894608,-122.373512,-122.37431,-122.375009,-122.375574],&#34;lat&#34;:[37.732716,37.734423,37.7346324340399,37.7336007043268,37.732949,37.732548,37.732774,37.732716]}],[{&#34;lng&#34;:[-122.376458,-122.373537,-122.373101279268,-122.37512685239,-122.375783,-122.374951,-122.376458],&#34;lat&#34;:[37.738252,37.737176,37.7361628507961,37.7352917212532,37.735099,37.737563,37.738252]}],[{&#34;lng&#34;:[-122.377679,-122.373453,-122.368752,-122.362911,-122.364289,-122.358779,-122.360775,-122.361107,-122.362294,-122.367138,-122.371649,-122.372836,-122.371269,-122.378962,-122.377679],&#34;lat&#34;:[37.83046,37.832298,37.831286,37.822696,37.81887,37.814278,37.812831,37.808029,37.807016,37.807429,37.809379,37.811068,37.814669,37.826897,37.83046]}],[{&#34;lng&#34;:[-122.420365736738,-122.419585,-122.41847,-122.419827349401,-122.420365736738],&#34;lat&#34;:[37.8633422971737,37.863282,37.861764,37.8600830082779,37.8633422971737]}]],[[{&#34;lng&#34;:[-122.447915,-122.446395,-122.446394,-122.446783,-122.446384,-122.444196,-122.440168,-122.439491,-122.439337,-122.439167,-122.442175,-122.442258,-122.44222,-122.445106,-122.444407,-122.444485,-122.445283,-122.44695,-122.447915],&#34;lat&#34;:[37.757589,37.758856,37.760907,37.761781,37.761816,37.761947,37.762187,37.762227,37.760627,37.759028,37.758939,37.757734,37.756639,37.756961,37.756568,37.756354,37.756573,37.75655,37.757589]}]],[[{&#34;lng&#34;:[-122.449447,-122.447794,-122.445697,-122.449423,-122.44709,-122.446734,-122.447943,-122.445348,-122.44695,-122.445283,-122.444485,-122.444407,-122.445106,-122.44222,-122.442258,-122.442175,-122.439167,-122.438864,-122.440345,-122.442525,-122.443255,-122.443421,-122.44462,-122.444556,-122.444857,-122.447399,-122.449363,-122.45084,-122.449447],&#34;lat&#34;:[37.74666,37.749408,37.748855,37.750412,37.751505,37.753573,37.755532,37.755619,37.75655,37.756573,37.756354,37.756568,37.756961,37.756639,37.757734,37.758939,37.759028,37.755434,37.755228,37.75238,37.750769,37.750099,37.747081,37.746687,37.747025,37.746596,37.746129,37.745904,37.74666]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.447399,-122.444857,-122.444556,-122.443961,-122.441408,-122.438246,-122.43809,-122.435916,-122.433697,-122.431477,-122.431256,-122.431022,-122.430888,-122.433259,-122.43332,-122.435559,-122.43556,-122.440049,-122.444148,-122.445679,-122.447399],&#34;lat&#34;:[37.746596,37.747025,37.746687,37.746765,37.74861,37.748665,37.74705,37.747179,37.747297,37.747434,37.745032,37.742631,37.741833,37.741903,37.743296,37.743161,37.74155,37.745299,37.743297,37.744139,37.746596]}]],[[{&#34;lng&#34;:[-122.434401,-122.434697,-122.432443,-122.431349,-122.428726,-122.431759,-122.430741,-122.430888,-122.423361,-122.424258,-122.42427,-122.424724,-122.425286,-122.426722,-122.428573,-122.429391,-122.432422,-122.433104,-122.435825,-122.434401],&#34;lat&#34;:[37.736199,37.737256,37.735879,37.736618,37.738326,37.739803,37.740424,37.741833,37.74229,37.739935,37.739866,37.738431,37.73764,37.736372,37.735082,37.734636,37.733176,37.733419,37.733924,37.736199]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.406853,-122.399705,-122.395945,-122.396515,-122.398488,-122.400022,-122.399423,-122.399867,-122.402105,-122.402752,-122.404657,-122.40506,-122.406295,-122.406551,-122.406853],&#34;lat&#34;:[37.73805,37.73992,37.737784,37.737156,37.733808,37.732311,37.731703,37.730292,37.72779,37.729357,37.732949,37.733652,37.735307,37.735721,37.73805]}]],[[{&#34;lng&#34;:[-122.418206,-122.41587,-122.413673,-122.411488,-122.410551,-122.410619,-122.410441,-122.409378,-122.410662,-122.410365,-122.410507,-122.411992,-122.413938,-122.415184,-122.415899,-122.418584,-122.41777,-122.418752,-122.416103,-122.41764,-122.417819,-122.419121,-122.418206],&#34;lat&#34;:[37.748203,37.748267,37.748297,37.748344,37.748367,37.746941,37.744918,37.742923,37.742424,37.741362,37.739779,37.73971,37.73893,37.738933,37.739002,37.739297,37.740564,37.74118,37.745253,37.745882,37.745965,37.746775,37.748203]}]],[[{&#34;lng&#34;:[-122.425286,-122.424724,-122.42427,-122.422841,-122.421742,-122.418584,-122.415899,-122.415955,-122.41676,-122.420163,-122.420224,-122.422121,-122.422688,-122.426722,-122.425286],&#34;lat&#34;:[37.73764,37.738431,37.739866,37.741027,37.740581,37.739297,37.739002,37.737211,37.735605,37.735777,37.735058,37.735153,37.735205,37.736372,37.73764]}]],[[{&#34;lng&#34;:[-122.476131,-122.470595,-122.469549,-122.466336,-122.466469,-122.465403,-122.464186,-122.463442,-122.464462,-122.463433,-122.463361,-122.462108,-122.46147,-122.458635,-122.459174,-122.461381,-122.463686,-122.465807,-122.466043,-122.470235,-122.475726,-122.475854,-122.476007,-122.476131],&#34;lat&#34;:[37.748643,37.748869,37.748903,37.749075,37.750939,37.750986,37.750717,37.751195,37.752899,37.753028,37.753039,37.751972,37.751522,37.747769,37.747286,37.745569,37.743749,37.743574,37.743555,37.743289,37.743049,37.744915,37.746778,37.748643]}]],[[{&#34;lng&#34;:[-122.461381,-122.459174,-122.458743,-122.454637,-122.454275,-122.453959,-122.453897,-122.451692,-122.449861,-122.452306,-122.453773,-122.454238,-122.45464,-122.454932,-122.458082,-122.458849,-122.459184,-122.459386,-122.45984,-122.461234,-122.463686,-122.461381],&#34;lat&#34;:[37.745569,37.747286,37.746876,37.746142,37.745908,37.745765,37.745742,37.745629,37.743265,37.742819,37.743219,37.742835,37.742326,37.741777,37.739099,37.739158,37.739871,37.740571,37.741061,37.742584,37.743749,37.745569]}]],[[{&#34;lng&#34;:[-122.459198,-122.452506,-122.451646,-122.449546,-122.447878,-122.448582,-122.450378,-122.451895,-122.453378,-122.454189,-122.456108,-122.459187,-122.459198],&#34;lat&#34;:[37.720024,37.72003,37.719686,37.723011,37.723005,37.718411,37.716169,37.716978,37.717768,37.718239,37.718235,37.71823,37.720024]}]],[[{&#34;lng&#34;:[-122.472547,-122.47244,-122.471983,-122.46974,-122.466278,-122.462265,-122.462245,-122.46264,-122.466235,-122.468036,-122.468017,-122.471457,-122.472672,-122.472547],&#34;lat&#34;:[37.71964,37.721603,37.721661,37.721644,37.721668,37.721748,37.71822,37.717936,37.717912,37.7179,37.716103,37.716085,37.717248,37.71964]}]],[[{&#34;lng&#34;:[-122.46264,-122.462245,-122.459187,-122.456108,-122.454189,-122.453378,-122.451895,-122.450378,-122.448582,-122.448727,-122.449627,-122.452884,-122.455028,-122.455619,-122.456376,-122.457312,-122.460858,-122.462556,-122.462569,-122.462602,-122.462615,-122.46264],&#34;lat&#34;:[37.717936,37.71822,37.71823,37.718235,37.718239,37.717768,37.716978,37.716169,37.718411,37.71745,37.715739,37.712829,37.711317,37.711066,37.710825,37.710648,37.710586,37.711254,37.71315,37.714308,37.715619,37.717936]}]],[[{&#34;lng&#34;:[-122.486248,-122.484104,-122.479817,-122.4766,-122.476355,-122.476221,-122.479426,-122.479296,-122.485727,-122.485857,-122.485988,-122.486118,-122.486248],&#34;lat&#34;:[37.755683,37.755777,37.755966,37.756108,37.752376,37.750509,37.750368,37.748504,37.74822,37.750085,37.751951,37.753815,37.755683]}]],[[{&#34;lng&#34;:[-122.495433,-122.492675,-122.486248,-122.486118,-122.485988,-122.485857,-122.495046,-122.495165,-122.495307,-122.495433],&#34;lat&#34;:[37.755273,37.755398,37.755683,37.753815,37.751951,37.750085,37.749681,37.751543,37.75341,37.755273]}]],[[{&#34;lng&#34;:[-122.499059,-122.496564,-122.493784,-122.491294,-122.485855,-122.485518,-122.480254,-122.479547,-122.475189,-122.475173,-122.474972,-122.480015,-122.48193,-122.485302,-122.486179,-122.486606,-122.48889,-122.492161,-122.493529,-122.497017,-122.499059],&#34;lat&#34;:[37.731576,37.733856,37.733989,37.734096,37.734182,37.734356,37.73459,37.734619,37.734762,37.734562,37.731072,37.731148,37.731172,37.730946,37.730879,37.729555,37.730257,37.728836,37.729594,37.729303,37.731576]}]],[[{&#34;lng&#34;:[-122.484909,-122.479631,-122.478988,-122.475347,-122.47537,-122.474528,-122.474383,-122.475818,-122.47724,-122.478674,-122.479981,-122.483205,-122.483143,-122.484832,-122.485124,-122.484909],&#34;lat&#34;:[37.724032,37.719663,37.720844,37.720914,37.720275,37.719065,37.717507,37.716891,37.717709,37.717903,37.716942,37.717585,37.718517,37.718571,37.718575,37.724032]}]],[[{&#34;lng&#34;:[-122.509993368973,-122.507942,-122.505148,-122.50193,-122.501536,-122.501404,-122.501142,-122.501041,-122.502031,-122.505262,-122.506796,-122.508121548602,-122.509993368973],&#34;lat&#34;:[37.7488149503326,37.749111,37.749235,37.749377,37.743785,37.74191,37.738192,37.7353,37.735515,37.735565,37.735559,37.7354395332202,37.7488149503326]}]],[[{&#34;lng&#34;:[-122.472583,-122.472352,-122.470912,-122.466924,-122.464781,-122.464551,-122.464284,-122.466432,-122.467492,-122.472264,-122.472379,-122.472516,-122.472583],&#34;lat&#34;:[37.786317,37.787234,37.787221,37.788132,37.788564,37.784792,37.781089,37.78099,37.780938,37.780747,37.782581,37.784453,37.786317]}]],[[{&#34;lng&#34;:[-122.493491,-122.490364,-122.487151,-122.487286,-122.48408,-122.483936,-122.483804,-122.483667,-122.489019,-122.491172,-122.493305,-122.493446,-122.493491],&#34;lat&#34;:[37.783498,37.78364,37.783785,37.785649,37.785795,37.783931,37.782077,37.780207,37.779977,37.779869,37.779752,37.781627,37.783498]}]],[[{&#34;lng&#34;:[-122.472264,-122.467492,-122.466432,-122.464284,-122.464011,-122.463749,-122.465332,-122.466952,-122.471706,-122.471842,-122.471967,-122.472103,-122.472264],&#34;lat&#34;:[37.780747,37.780938,37.78099,37.781089,37.777235,37.773624,37.773423,37.773351,37.773141,37.775006,37.776871,37.778731,37.780747]}]],[[{&#34;lng&#34;:[-122.498801,-122.493446,-122.493305,-122.493168,-122.497455,-122.497319,-122.495175,-122.495039,-122.494903,-122.498114,-122.49825,-122.498386,-122.498521,-122.498669,-122.498801],&#34;lat&#34;:[37.781377,37.781627,37.779752,37.777774,37.777579,37.775715,37.775813,37.773947,37.772081,37.771934,37.7738,37.775667,37.777531,37.779507,37.781377]}]],[[{&#34;lng&#34;:[-122.509879,-122.506509,-122.505226,-122.502012,-122.498801,-122.498669,-122.498521,-122.501733,-122.506022,-122.509228,-122.509593,-122.509879],&#34;lat&#34;:[37.780627,37.780857,37.781077,37.781227,37.781377,37.779507,37.777531,37.777385,37.77719,37.777044,37.780173,37.780627]}]],[[{&#34;lng&#34;:[-122.506796,-122.505262,-122.502031,-122.501041,-122.499139,-122.497487,-122.496564,-122.499059,-122.497017,-122.493529,-122.492161,-122.48889,-122.486606,-122.483811,-122.484909,-122.485124,-122.485275,-122.477016,-122.472269,-122.471355,-122.471215,-122.471319,-122.486042,-122.502426880724,-122.506483,-122.508121548602,-122.506796],&#34;lat&#34;:[37.735559,37.735565,37.735515,37.7353,37.734404,37.733925,37.733856,37.731576,37.729303,37.729594,37.728836,37.730257,37.729555,37.727029,37.724032,37.718575,37.714823,37.714593,37.712838,37.712802,37.708939,37.708305,37.708233,37.7081329861058,37.723731,37.7354395332202,37.735559]}]],[[{&#34;lng&#34;:[-122.395945,-122.394093,-122.391268,-122.389425,-122.387705,-122.387143,-122.385282,-122.388529,-122.38717,-122.387399,-122.389035,-122.39106,-122.391617,-122.393904,-122.392776,-122.396515,-122.395945],&#34;lat&#34;:[37.737784,37.736729,37.739853,37.738803,37.742697,37.741297,37.74024,37.736475,37.735698,37.734738,37.732919,37.734066,37.732487,37.733788,37.735038,37.737156,37.737784]}]],[[{&#34;lng&#34;:[-122.403784,-122.403007,-122.403127,-122.403396,-122.403689,-122.39797,-122.398216,-122.392494,-122.392447,-122.391232,-122.390683,-122.394587,-122.396408,-122.402605,-122.403569,-122.403784],&#34;lat&#34;:[37.749433,37.752446,37.754478,37.756471,37.757015,37.757359,37.759909,37.760245,37.756436,37.752657,37.752663,37.752427,37.751179,37.750558,37.749475,37.749433]}]],[[{&#34;lng&#34;:[-123.013916,-123.007786,-123.007548,-123.003507,-123.000893,-122.998754,-123.002794,-123.005884,-123.007548,-123.012777,-123.013916],&#34;lat&#34;:[37.700355,37.698943,37.70214,37.704396,37.701011,37.697438,37.692736,37.693489,37.695934,37.696498,37.700355]}]],[[{&#34;lng&#34;:[-122.42667,-122.42639,-122.424555,-122.424848,-122.423807,-122.421847,-122.419875,-122.416632,-122.412969,-122.412468,-122.41099,-122.407036,-122.406665,-122.407272,-122.408171,-122.410433,-122.411316,-122.415396,-122.415291,-122.415635,-122.420908,-122.421475,-122.422241,-122.422811,-122.424478,-122.426737,-122.425548,-122.427189,-122.42667],&#34;lat&#34;:[37.717078,37.718561,37.721648,37.722053,37.723691,37.724199,37.72471,37.722157,37.723858,37.722658,37.719074,37.720085,37.719215,37.71782,37.716157,37.716414,37.714367,37.715474,37.713504,37.712733,37.713774,37.712201,37.711317,37.710511,37.71011,37.711225,37.714517,37.715748,37.717078]}]],[[{&#34;lng&#34;:[-122.426671,-122.424876,-122.425041162157,-122.420612315115,-122.419224,-122.418718,-122.418281,-122.417799,-122.417587,-122.420769,-122.420883,-122.424275,-122.424464,-122.424835,-122.425209,-122.425628,-122.426257,-122.426613,-122.426671],&#34;lat&#34;:[37.809639,37.810799,37.811001508766,37.8111121700604,37.808453,37.805932,37.804023,37.801267,37.800341,37.799939,37.799902,37.799472,37.800402,37.802268,37.804133,37.806456,37.807185,37.808152,37.809639]}]],[[{&#34;lng&#34;:[-122.418271,-122.414995,-122.414617,-122.41443,-122.416072,-122.417707,-122.417901,-122.418271],&#34;lat&#34;:[37.787038,37.787454,37.785582,37.784657,37.784449,37.784236,37.785167,37.787038]}]],[[{&#34;lng&#34;:[-122.426893,-122.423705,-122.423481,-122.423283,-122.423143,-122.422801,-122.422611,-122.425802,-122.426335,-122.426893],&#34;lat&#34;:[37.796272,37.796678,37.795739,37.79481,37.793923,37.792161,37.791277,37.790874,37.793517,37.796272]}]],[[{&#34;lng&#34;:[-122.436204,-122.434558,-122.429625,-122.42927,-122.428905,-122.43384,-122.435486,-122.435849,-122.436204],&#34;lat&#34;:[37.792263,37.792472,37.793098,37.791339,37.789534,37.788906,37.788697,37.790502,37.792263]}]],[[{&#34;lng&#34;:[-122.434562,-122.431291,-122.424704,-122.424678,-122.424108,-122.427396,-122.430696,-122.433973,-122.434562],&#34;lat&#34;:[37.784152,37.784621,37.785418,37.785291,37.782477,37.782057,37.781635,37.781219,37.784152]}]],[[{&#34;lng&#34;:[-122.432467,-122.430825,-122.42899,-122.428802,-122.425512,-122.424929,-122.426402,-122.42822,-122.43157,-122.431904,-122.432467],&#34;lat&#34;:[37.773757,37.773973,37.773248,37.772316,37.772734,37.770778,37.769596,37.769441,37.769307,37.770964,37.773757]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.424258,-122.423361,-122.422709,-122.422174,-122.421704,-122.421393,-122.420282,-122.418206,-122.419121,-122.417819,-122.41764,-122.416103,-122.418752,-122.41777,-122.418584,-122.421742,-122.422841,-122.42427,-122.424258],&#34;lat&#34;:[37.739935,37.74229,37.74394,37.745614,37.746412,37.748124,37.748161,37.748203,37.746775,37.745965,37.745882,37.745253,37.74118,37.740564,37.739297,37.740581,37.741027,37.739866,37.739935]}]],[[{&#34;lng&#34;:[-122.446308,-122.443664,-122.443075,-122.439483,-122.439558,-122.435892,-122.432422,-122.429391,-122.428573,-122.426158,-122.428038,-122.429304,-122.430165,-122.430199,-122.431294,-122.431801,-122.432447,-122.433982,-122.434798,-122.435385,-122.43702,-122.438787,-122.444486,-122.44473,-122.447878,-122.446308],&#34;lat&#34;:[37.725978,37.728304,37.728634,37.730112,37.730191,37.731577,37.733176,37.734636,37.735082,37.733971,37.732016,37.730742,37.729851,37.729872,37.72873,37.728249,37.727637,37.725762,37.724683,37.723909,37.723723,37.723637,37.722974,37.722987,37.723005,37.725978]}]],[[{&#34;lng&#34;:[-122.435385,-122.434798,-122.433982,-122.432447,-122.426912,-122.428074,-122.428837,-122.430026,-122.431205,-122.432385,-122.433985,-122.43715,-122.435385],&#34;lat&#34;:[37.723909,37.724683,37.725762,37.727637,37.725151,37.723591,37.723955,37.722398,37.720856,37.719298,37.720063,37.721575,37.723909]}]],[[{&#34;lng&#34;:[-122.420083,-122.419348,-122.416648,-122.416,-122.412887,-122.411349,-122.41238,-122.415182,-122.416222,-122.420082,-122.420083],&#34;lat&#34;:[37.708311,37.709947,37.710473,37.711915,37.711036,37.710605,37.708303,37.708231,37.708328,37.708231,37.708311]}]],[[{&#34;lng&#34;:[-122.464297,-122.46109,-122.458405,-122.45779,-122.456913,-122.452947,-122.452758,-122.452569,-122.455466,-122.457536,-122.460843,-122.462972,-122.463226,-122.463782,-122.463912,-122.464042,-122.464172,-122.464297],&#34;lat&#34;:[37.765963,37.766132,37.76616,37.766015,37.765874,37.766374,37.765443,37.764509,37.764161,37.763566,37.762628,37.762316,37.758562,37.758538,37.760405,37.762269,37.764131,37.765963]}]],[[{&#34;lng&#34;:[-122.462888,-122.460816,-122.461727,-122.461748,-122.459064,-122.459505,-122.456156,-122.453425,-122.453411,-122.453394,-122.454002,-122.452382,-122.452335,-122.453105,-122.458191,-122.459213,-122.461202,-122.462199,-122.462786,-122.462888],&#34;lat&#34;:[37.728056,37.728681,37.729993,37.730064,37.733645,37.734291,37.734212,37.73304,37.731568,37.729837,37.729166,37.727868,37.72318,37.723206,37.724344,37.724563,37.724974,37.725293,37.725549,37.728056]}]],[[{&#34;lng&#34;:[-122.50193,-122.499784,-122.499914,-122.495165,-122.495046,-122.494911,-122.494774,-122.494646,-122.494504,-122.494404,-122.494253,-122.494126,-122.493784,-122.496564,-122.497487,-122.499139,-122.501041,-122.501142,-122.501404,-122.501536,-122.50193],&#34;lat&#34;:[37.749377,37.749472,37.751334,37.751543,37.749681,37.747814,37.745949,37.744089,37.742222,37.740352,37.738493,37.736631,37.733989,37.733856,37.733925,37.734404,37.7353,37.738192,37.74191,37.743785,37.749377]}]],[[{&#34;lng&#34;:[-122.406062,-122.399195,-122.396315,-122.394748,-122.393181,-122.391487,-122.389708268207,-122.388570312096,-122.385683559326,-122.385323,-122.383505768697,-122.387421,-122.388423,-122.389168,-122.390527,-122.39275,-122.397386,-122.39893,-122.400468,-122.402708,-122.406062],&#34;lat&#34;:[37.785757,37.791058,37.793337,37.79448,37.79323,37.793859,37.7958244063587,37.7945008753206,37.7911433584052,37.790724,37.7830792797042,37.78279,37.781798,37.781649,37.780607,37.778852,37.782554,37.783785,37.78503,37.783259,37.785757]}]],[[{&#34;lng&#34;:[-122.414054,-122.410765,-122.410952,-122.41114,-122.407853,-122.407451,-122.408104,-122.408952,-122.410336,-122.412548,-122.413365,-122.413866,-122.414054],&#34;lat&#34;:[37.782794,37.783214,37.784141,37.785075,37.785492,37.784604,37.78402,37.783288,37.782245,37.780508,37.779862,37.781863,37.782794]}]],[[{&#34;lng&#34;:[-122.384047,-122.381558,-122.380887,-122.37911,-122.376847,-122.373364367386,-122.369169789163,-122.367868,-122.374012,-122.372924,-122.368368114461,-122.368397,-122.369459113871,-122.373101279268,-122.373537,-122.376458,-122.374951,-122.375783,-122.37512685239,-122.375854,-122.37510014051,-122.374893,-122.375574,-122.375009,-122.37431,-122.373512,-122.372855894608,-122.370094,-122.369891892504,-122.37364,-122.377681,-122.378278,-122.379697,-122.383431,-122.38209,-122.381439,-122.379779,-122.379932,-122.381284,-122.380186,-122.382919,-122.384047],&#34;lat&#34;:[37.737514,37.738123,37.738828,37.73768,37.740192,37.7457632793685,37.7417391770183,37.739994,37.739499,37.738348,37.73825770119,37.738186,37.7377292212437,37.7361628507961,37.737176,37.738252,37.737563,37.735099,37.7352917212532,37.734979,37.7346324340399,37.734423,37.732716,37.732774,37.732548,37.732949,37.7336007043268,37.732331,37.7325512356194,37.728106,37.72863,37.728964,37.727618,37.729738,37.730635,37.730582,37.730799,37.733381,37.734034,37.73266,37.734511,37.737514]}]],[[{&#34;lng&#34;:[-122.464284,-122.461068,-122.458878,-122.458505,-122.458371,-122.460553,-122.463749,-122.464011,-122.464284],&#34;lat&#34;:[37.781089,37.781227,37.781326,37.77619,37.774309,37.774031,37.773624,37.777235,37.781089]}]],[[{&#34;lng&#34;:[-122.429178,-122.425888,-122.424246,-122.419359,-122.419334,-122.421989,-122.42262,-122.423641,-122.424929,-122.425512,-122.428802,-122.42899,-122.430825,-122.429178],&#34;lat&#34;:[37.774181,37.774599,37.774811,37.775432,37.77521,37.773074,37.772503,37.771828,37.770778,37.772734,37.772316,37.773248,37.773973,37.774181]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]],[[{&#34;lng&#34;:[-122.393968,-122.388164,-122.387917,-122.385823,-122.380462169293,-122.379568,-122.380757,-122.378560351096,-122.380985,-122.381711,-122.390683,-122.391232,-122.392447,-122.392494,-122.392319,-122.39252,-122.392714,-122.393848,-122.393968],&#34;lat&#34;:[37.766614,37.766958,37.764379,37.767255,37.7675163525902,37.76055,37.755288,37.7535517721854,37.753228,37.753186,37.752663,37.752657,37.756436,37.760245,37.76106,37.762819,37.764084,37.765307,37.766614]}]],[[{&#34;lng&#34;:[-122.405572,-122.4026,-122.402061,-122.400785,-122.398863,-122.399615,-122.400435,-122.402103,-122.403078,-122.404059,-122.405572],&#34;lat&#34;:[37.724461,37.725235,37.72392,37.723616,37.718683,37.716278,37.719162,37.718721,37.72113,37.720864,37.724461]}]],[[{&#34;lng&#34;:[-122.415635,-122.415291,-122.415396,-122.411316,-122.410433,-122.406189,-122.407927,-122.410641,-122.411349,-122.412887,-122.416,-122.415635],&#34;lat&#34;:[37.712733,37.713504,37.715474,37.714367,37.716414,37.715244,37.711448,37.712206,37.710605,37.711036,37.711915,37.712733]}]],[[{&#34;lng&#34;:[-122.414877,-122.411581,-122.408332,-122.406625,-122.405174,-122.404798,-122.40316,-122.40279,-122.402421,-122.405645,-122.406024,-122.406199,-122.40777,-122.411019,-122.411376,-122.412908,-122.414877],&#34;lat&#34;:[37.803542,37.803963,37.804367,37.804535,37.804763,37.802901,37.803098,37.801242,37.799382,37.798969,37.800835,37.80172,37.801571,37.80117,37.801125,37.802184,37.803542]}]],[[{&#34;lng&#34;:[-122.475347,-122.474745,-122.474634,-122.474972,-122.475173,-122.471569,-122.466248,-122.462879,-122.463136,-122.461242,-122.459505,-122.459064,-122.461748,-122.461727,-122.460816,-122.462888,-122.462786,-122.462199,-122.462265,-122.466278,-122.46974,-122.471983,-122.47244,-122.472547,-122.472672,-122.474528,-122.47537,-122.475347],&#34;lat&#34;:[37.720914,37.726936,37.728992,37.731072,37.734562,37.734707,37.734937,37.736212,37.737023,37.735782,37.734291,37.733645,37.730064,37.729993,37.728681,37.728056,37.725549,37.725293,37.721748,37.721668,37.721644,37.721661,37.721603,37.71964,37.717248,37.719065,37.720275,37.720914]}]],[[{&#34;lng&#34;:[-122.512070759187,-122.50987,-122.509622,-122.509382,-122.509111,-122.502582,-122.50245,-122.502318,-122.502191,-122.50206,-122.50193,-122.505148,-122.507942,-122.509993368973,-122.512070759187],&#34;lat&#34;:[37.7636593024008,37.76409,37.762127,37.760272,37.758414,37.7587,37.756835,37.754953,37.753106,37.75124,37.749377,37.749235,37.749111,37.7488149503326,37.7636593024008]}]],[[{&#34;lng&#34;:[-122.405582,-122.404821,-122.404348,-122.403109,-122.402546,-122.400455,-122.398328,-122.398515,-122.397025,-122.396682,-122.393545,-122.391141,-122.387912,-122.386265,-122.38282,-122.383251,-122.374980911262,-122.377251,-122.379568,-122.37529,-122.390674,-122.39137503156,-122.393635,-122.395107,-122.404682,-122.405292,-122.405582],&#34;lat&#34;:[37.708231,37.709747,37.710443,37.711998,37.712266,37.713118,37.716259,37.717664,37.719515,37.720266,37.718487,37.71804,37.716328,37.717179,37.721872,37.720151,37.7155576312199,37.714557,37.711491,37.708482,37.70864,37.708331,37.708244,37.708339,37.708331,37.708262,37.708231]}]],[[{&#34;lng&#34;:[-122.406952,-122.40459,-122.403561,-122.403569,-122.402605,-122.396408,-122.394587,-122.390683,-122.381711,-122.380985,-122.378560351096,-122.376716,-122.376241,-122.373364367386,-122.376847,-122.37911,-122.380887,-122.381558,-122.385282,-122.387143,-122.387705,-122.389425,-122.391268,-122.394093,-122.395945,-122.399705,-122.406853,-122.406952],&#34;lat&#34;:[37.740495,37.744126,37.747762,37.749475,37.750558,37.751179,37.752427,37.752663,37.753186,37.753228,37.7535517721854,37.752094,37.748523,37.7457632793685,37.740192,37.73768,37.738828,37.738123,37.74024,37.741297,37.742697,37.738803,37.739853,37.736729,37.737784,37.73992,37.73805,37.740495]}]],[[{&#34;lng&#34;:[-122.425616,-122.42242,-122.422237,-122.422077,-122.421858,-122.421461,-122.421256,-122.422838,-122.424678,-122.424704,-122.425047,-122.425616],&#34;lat&#34;:[37.789952,37.790358,37.789399,37.788474,37.787535,37.786632,37.784745,37.784547,37.785291,37.785418,37.78713,37.789952]}]],[[{&#34;lng&#34;:[-122.454683,-122.451358,-122.448068,-122.446471,-122.44478,-122.441488,-122.440922,-122.444216,-122.447497,-122.450787,-122.454083,-122.454683],&#34;lat&#34;:[37.774755,37.775175,37.775599,37.775802,37.776017,37.776435,37.773635,37.773212,37.77279,37.772368,37.77193,37.774755]}]],[[{&#34;lng&#34;:[-122.439491,-122.435837,-122.435193,-122.435188,-122.435001,-122.434854,-122.434698,-122.434546,-122.437769,-122.438864,-122.439167,-122.439337,-122.439491],&#34;lat&#34;:[37.762227,37.762485,37.762727,37.762671,37.760889,37.759289,37.757687,37.756093,37.755541,37.755434,37.759028,37.760627,37.762227]}]],[[{&#34;lng&#34;:[-122.41676,-122.415955,-122.415899,-122.415184,-122.413938,-122.411992,-122.410507,-122.410377,-122.408217,-122.408779,-122.409798,-122.410513,-122.412447,-122.414313,-122.416824,-122.41676],&#34;lat&#34;:[37.735605,37.737211,37.739002,37.738933,37.73893,37.73971,37.739779,37.736894,37.737612,37.735817,37.734828,37.734592,37.73467,37.734763,37.734888,37.735605]}]],[[{&#34;lng&#34;:[-122.437811,-122.436642,-122.436203,-122.436144,-122.433682,-122.432777,-122.431154,-122.427189,-122.425548,-122.426737,-122.424478,-122.423377,-122.42389,-122.428082,-122.430027,-122.431341,-122.433382,-122.433982,-122.435382,-122.436771,-122.436993,-122.43837,-122.437811],&#34;lat&#34;:[37.71192,37.71355,37.714171,37.714254,37.713181,37.714437,37.716594,37.715748,37.714517,37.711225,37.71011,37.709203,37.708236,37.708431,37.708281,37.708231,37.708232,37.708132,37.708132,37.710156,37.710746,37.711133,37.71192]}]],[[{&#34;lng&#34;:[-122.462265,-122.462199,-122.461202,-122.459213,-122.458191,-122.453105,-122.452335,-122.451596,-122.449546,-122.451646,-122.452506,-122.459198,-122.459187,-122.462245,-122.462265],&#34;lat&#34;:[37.721748,37.725293,37.724974,37.724563,37.724344,37.723206,37.72318,37.723033,37.723011,37.719686,37.72003,37.720024,37.71823,37.71822,37.721748]}]],[[{&#34;lng&#34;:[-122.494504,-122.485335,-122.475726,-122.475621,-122.475485,-122.47539,-122.475189,-122.479547,-122.480254,-122.485518,-122.485855,-122.491294,-122.493784,-122.494126,-122.494253,-122.494404,-122.494504],&#34;lat&#34;:[37.742222,37.742626,37.743049,37.741183,37.739321,37.73745,37.734762,37.734619,37.73459,37.734356,37.734182,37.734096,37.733989,37.736631,37.738493,37.740352,37.742222]}]],[[{&#34;lng&#34;:[-122.448241,-122.448241130218,-122.439690935158,-122.441883,-122.441905,-122.441754,-122.441368,-122.441177,-122.447466,-122.44782,-122.44946,-122.448241],&#34;lat&#34;:[37.804478,37.8072521685356,37.8086811541357,37.80731,37.803744,37.802985,37.801126,37.80019,37.79939,37.801633,37.802454,37.804478]}]],[[{&#34;lng&#34;:[-122.453352,-122.450522,-122.44839,-122.448013,-122.447635,-122.448274,-122.449433,-122.44912,-122.451958,-122.4524,-122.452569,-122.452758,-122.452947,-122.453352],&#34;lat&#34;:[37.768246,37.768593,37.768865,37.767007,37.765139,37.764196,37.7632,37.761648,37.761478,37.763669,37.764509,37.765443,37.766374,37.768246]}]],[[{&#34;lng&#34;:[-122.475726,-122.470235,-122.466043,-122.465807,-122.463686,-122.461234,-122.45984,-122.459386,-122.459184,-122.458849,-122.459592,-122.460282,-122.459505,-122.461242,-122.463136,-122.462879,-122.466248,-122.471569,-122.475173,-122.475189,-122.47539,-122.475485,-122.475621,-122.475726],&#34;lat&#34;:[37.743049,37.743289,37.743555,37.743574,37.743749,37.742584,37.741061,37.740571,37.739871,37.739158,37.738533,37.737855,37.734291,37.735782,37.737023,37.736212,37.734937,37.734707,37.734562,37.734762,37.73745,37.739321,37.741183,37.743049]}]],[[{&#34;lng&#34;:[-122.41248,-122.408516,-122.40407,-122.401843,-122.397386,-122.39275,-122.397206,-122.398755,-122.400982,-122.399433,-122.401657,-122.404391,-122.40499,-122.408007,-122.409736,-122.410931,-122.41248],&#34;lat&#34;:[37.77063,37.77376,37.777273,37.779032,37.782554,37.778852,37.775332,37.776568,37.774809,37.773573,37.771817,37.76975,37.769715,37.769244,37.770349,37.769411,37.77063]}]],[[{&#34;lng&#34;:[-122.411349,-122.410641,-122.407927,-122.406189,-122.403693,-122.402188,-122.402154,-122.402546,-122.403109,-122.404348,-122.404821,-122.405582,-122.406481,-122.41238,-122.411349],&#34;lat&#34;:[37.710605,37.712206,37.711448,37.715244,37.714341,37.714731,37.712656,37.712266,37.711998,37.710443,37.709747,37.708231,37.708323,37.708303,37.710605]}]],[[{&#34;lng&#34;:[-122.442053,-122.440958,-122.437072,-122.436697,-122.433407,-122.433219,-122.438244,-122.441488,-122.441866,-122.442053],&#34;lat&#34;:[37.779244,37.779364,37.779867,37.778006,37.778422,37.77749,37.776848,37.776435,37.7783,37.779244]}]],[[{&#34;lng&#34;:[-122.477261,-122.47073,-122.470595,-122.470468,-122.470335,-122.476762,-122.476974,-122.477116,-122.477261],&#34;lat&#34;:[37.765486,37.765779,37.763849,37.762027,37.760124,37.759837,37.761701,37.763562,37.765486]}]],[[{&#34;lng&#34;:[-122.385024,-122.382176,-122.377833,-122.379697,-122.378278,-122.377681,-122.37364,-122.369891892504,-122.367697,-122.365478,-122.356784,-122.360238,-122.364084,-122.370411,-122.374980911262,-122.383251,-122.38282,-122.384517,-122.386899,-122.385024],&#34;lat&#34;:[37.724098,37.722246,37.72656,37.727618,37.728964,37.72863,37.728106,37.7325512356194,37.734943,37.734621,37.729505,37.719421,37.715701,37.717572,37.7155576312199,37.720151,37.721872,37.722862,37.724289,37.724098]}]],[[{&#34;lng&#34;:[-122.396682,-122.393284,-122.389801,-122.38793,-122.38521,-122.384517,-122.38282,-122.386265,-122.387912,-122.391141,-122.393545,-122.396682],&#34;lat&#34;:[37.720266,37.727764,37.725791,37.724729,37.723088,37.722862,37.721872,37.717179,37.716328,37.71804,37.718487,37.720266]}]],[[{&#34;lng&#34;:[-122.446247,-122.443538,-122.443347,-122.442575,-122.442009,-122.439301,-122.438338,-122.436588,-122.435794,-122.435472,-122.435193,-122.435837,-122.439491,-122.440168,-122.444196,-122.446384,-122.446247],&#34;lat&#34;:[37.762247,37.765323,37.765333,37.765928,37.765907,37.768166,37.769007,37.769001,37.769058,37.765727,37.762727,37.762485,37.762227,37.762187,37.761947,37.761816,37.762247]}]],[[{&#34;lng&#34;:[-122.410662,-122.409378,-122.410441,-122.410619,-122.410551,-122.40871,-122.406788,-122.405239,-122.403784,-122.403569,-122.403561,-122.40459,-122.406952,-122.406853,-122.408217,-122.410377,-122.410507,-122.410365,-122.410662],&#34;lat&#34;:[37.742424,37.742923,37.744918,37.746941,37.748367,37.748397,37.748805,37.749125,37.749433,37.749475,37.747762,37.744126,37.740495,37.73805,37.737612,37.736894,37.739779,37.741362,37.742424]}]],[[{&#34;lng&#34;:[-122.424275,-122.420883,-122.420769,-122.417587,-122.417385,-122.417218,-122.416839,-122.421768,-122.423481,-122.423705,-122.424275],&#34;lat&#34;:[37.799472,37.799902,37.799939,37.800341,37.799414,37.798472,37.796596,37.795945,37.795739,37.796678,37.799472]}]],[[{&#34;lng&#34;:[-122.435486,-122.43384,-122.428905,-122.425616,-122.425047,-122.428336,-122.431635,-122.434918,-122.435486],&#34;lat&#34;:[37.788697,37.788906,37.789534,37.789952,37.78713,37.786713,37.786293,37.785876,37.788697]}]],[[{&#34;lng&#34;:[-122.495046,-122.485857,-122.485727,-122.485597,-122.485466,-122.485335,-122.494504,-122.494646,-122.494774,-122.494911,-122.495046],&#34;lat&#34;:[37.749681,37.750085,37.74822,37.746354,37.744493,37.742626,37.742222,37.744089,37.745949,37.747814,37.749681]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#F2FABD&#34;,&#34;#F2FABD&#34;,&#34;#F2FABB&#34;,&#34;#65C2BF&#34;,&#34;#225EA8&#34;,&#34;#A5DBB8&#34;,&#34;#F1FABA&#34;,&#34;#B5E1B6&#34;,&#34;#C7E9B4&#34;,&#34;#6EC5BE&#34;,&#34;#61C0C0&#34;,&#34;#E5F5B2&#34;,&#34;#F5FBC3&#34;,&#34;#86D0BA&#34;,&#34;#F9FDCC&#34;,&#34;#D0ECB3&#34;,&#34;#FBFED1&#34;,&#34;#FAFDCD&#34;,&#34;#FFFFD9&#34;,&#34;#58BDC1&#34;,&#34;#CDEBB4&#34;,&#34;#2282B9&#34;,&#34;#E3F4B2&#34;,&#34;#2395C1&#34;,&#34;#26449B&#34;,&#34;#2370B1&#34;,&#34;#D6EFB3&#34;,&#34;#C7E9B4&#34;,&#34;#D8EFB3&#34;,&#34;#81CEBB&#34;,&#34;#CEECB3&#34;,&#34;#FFFFD9&#34;,&#34;#ECF8B1&#34;,&#34;#BFE5B5&#34;,&#34;#F3FABF&#34;,&#34;#FFFFD9&#34;,&#34;#4FBAC2&#34;,&#34;#BDE5B5&#34;,&#34;#E0F3B2&#34;,&#34;#2260A9&#34;,&#34;#F0F9B7&#34;,&#34;#E2F3B2&#34;,&#34;#E1F3B2&#34;,&#34;#F0F9B6&#34;,&#34;#DCF1B2&#34;,&#34;#94D4B9&#34;,&#34;#A8DCB7&#34;,&#34;#2364AB&#34;,&#34;#95D5B9&#34;,&#34;#B9E3B6&#34;,&#34;#F3FABF&#34;,&#34;#2D9EC2&#34;,&#34;#D4EEB3&#34;,&#34;#C0E6B5&#34;,&#34;#ECF8B1&#34;,&#34;#72C7BD&#34;,&#34;#F1F9B9&#34;,&#34;#F5FBC2&#34;,&#34;#FFFFD9&#34;,&#34;#FAFDCD&#34;,&#34;#7CCBBC&#34;,&#34;#2FA0C2&#34;,&#34;#1D2D82&#34;,&#34;#2193C0&#34;,&#34;#41B6C4&#34;,&#34;#7DCCBB&#34;,&#34;#E4F5B2&#34;,&#34;#F5FBC3&#34;,&#34;#F6FBC4&#34;,&#34;#D1EDB3&#34;,&#34;#BAE3B6&#34;,&#34;#8BD1BA&#34;,&#34;#30A0C2&#34;,&#34;#6EC5BE&#34;,&#34;#68C3BF&#34;,&#34;#F5FBC2&#34;,&#34;#F7FCC7&#34;,&#34;#FBFED0&#34;,&#34;#F4FBBF&#34;,&#34;#CFECB3&#34;,&#34;#F3FABD&#34;,&#34;#DEF2B2&#34;,&#34;#77C9BD&#34;,&#34;#6FC6BE&#34;,&#34;#227DB7&#34;,&#34;#E0F3B2&#34;,&#34;#F8FCC9&#34;,&#34;#74C8BD&#34;,&#34;#081D58&#34;,&#34;#C5E8B4&#34;,&#34;#2088BC&#34;,&#34;#61C0C0&#34;,&#34;#FFFFD9&#34;,&#34;#C2E7B5&#34;,&#34;#B4E1B6&#34;,&#34;#E2F4B2&#34;,&#34;#E9F6B1&#34;,&#34;#EBF7B1&#34;,&#34;#5CBEC1&#34;,&#34;#F4FBBF&#34;,&#34;#3CAFC3&#34;,&#34;#3FB3C4&#34;,&#34;#F3FABF&#34;,&#34;#5DBEC1&#34;,&#34;#DBF1B2&#34;,&#34;#F6FCC5&#34;,&#34;#FDFED5&#34;,&#34;#FFFFD9&#34;,&#34;#1D91C0&#34;,&#34;#CCEBB4&#34;,&#34;#94D5B9&#34;,&#34;#F7FCC6&#34;,&#34;#C9EAB4&#34;,&#34;#DDF2B2&#34;,&#34;#2D9DC1&#34;,&#34;#AADDB7&#34;,&#34;#CCEBB4&#34;,&#34;#2367AC&#34;,&#34;#2087BB&#34;,&#34;#53BBC2&#34;,&#34;#38AAC3&#34;,&#34;#EDF8B2&#34;,&#34;#EFF9B4&#34;,&#34;#F3FABE&#34;,&#34;#40B4C4&#34;,&#34;#D8F0B3&#34;,&#34;#235BA7&#34;,&#34;#235CA7&#34;,&#34;#E7F6B2&#34;,&#34;#9DD8B8&#34;,&#34;#EDF8B1&#34;,&#34;#ECF8B1&#34;,&#34;#87D0BA&#34;,&#34;#59BDC1&#34;,&#34;#FDFED4&#34;,&#34;#ECF8B1&#34;,&#34;#90D3BA&#34;,&#34;#D8F0B3&#34;,&#34;#99D6B9&#34;,&#34;#ACDEB7&#34;,&#34;#CFECB3&#34;,&#34;#EBF7B1&#34;,&#34;#FFFFD9&#34;,&#34;#38AAC3&#34;,&#34;#808080&#34;,&#34;#EDF8B2&#34;,&#34;#43B6C4&#34;,&#34;#FDFED5&#34;,&#34;#2D9DC1&#34;,&#34;#56BCC2&#34;,&#34;#E7F6B2&#34;,&#34;#A8DCB7&#34;,&#34;#DBF1B3&#34;,&#34;#DDF1B2&#34;,&#34;#53BBC2&#34;,&#34;#C8E9B4&#34;,&#34;#DBF1B3&#34;,&#34;#FDFED3&#34;,&#34;#A8DCB7&#34;,&#34;#B7E2B6&#34;,&#34;#E7F5B2&#34;,&#34;#1F8ABD&#34;,&#34;#FFFFD9&#34;,&#34;#FFFFD9&#34;,&#34;#CBEBB4&#34;,&#34;#82CEBB&#34;,&#34;#EFF9B6&#34;,&#34;#2369AD&#34;,&#34;#FCFED3&#34;,&#34;#F6FBC5&#34;,&#34;#A8DCB7&#34;,&#34;#2281B9&#34;,&#34;#F1F9B9&#34;,&#34;#DDF1B2&#34;,&#34;#236BAE&#34;,&#34;#ADDEB7&#34;,&#34;#86CFBA&#34;,&#34;#3CAEC3&#34;,&#34;#8FD3BA&#34;,&#34;#F0F9B6&#34;,&#34;#E3F4B2&#34;,&#34;#ADDEB7&#34;,&#34;#2375B3&#34;,&#34;#5EBFC1&#34;,&#34;#227FB8&#34;,&#34;#2184BA&#34;,&#34;#FCFED2&#34;,&#34;#DAF0B3&#34;,&#34;#E2F3B2&#34;,&#34;#FFFFD9&#34;,&#34;#F9FDCC&#34;,&#34;#3EB2C4&#34;,&#34;#CBEBB4&#34;,&#34;#3DB0C3&#34;,&#34;#A7DCB7&#34;,&#34;#EDF8B2&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075010100&lt;br&gt;Percent of Households above $200k: 3.89&#34;,&#34;GEOID: 06075010600&lt;br&gt;Percent of Households above $200k: 3.87&#34;,&#34;GEOID: 06075012000&lt;br&gt;Percent of Households above $200k: 4.04&#34;,&#34;GEOID: 06075012601&lt;br&gt;Percent of Households above $200k: 19.04&#34;,&#34;GEOID: 06075013300&lt;br&gt;Percent of Households above $200k: 32.76&#34;,&#34;GEOID: 06075015600&lt;br&gt;Percent of Households above $200k: 13.66&#34;,&#34;GEOID: 06075016000&lt;br&gt;Percent of Households above $200k: 4.25&#34;,&#34;GEOID: 06075016300&lt;br&gt;Percent of Households above $200k: 12.43&#34;,&#34;GEOID: 06075017700&lt;br&gt;Percent of Households above $200k: 10.87&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent of Households above $200k: 18.24&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent of Households above $200k: 19.41&#34;,&#34;GEOID: 06075026402&lt;br&gt;Percent of Households above $200k: 6.71&#34;,&#34;GEOID: 06075031302&lt;br&gt;Percent of Households above $200k: 2.98&#34;,&#34;GEOID: 06075042602&lt;br&gt;Percent of Households above $200k: 15.9&#34;,&#34;GEOID: 06075061100&lt;br&gt;Percent of Households above $200k: 1.79&#34;,&#34;GEOID: 06075011000&lt;br&gt;Percent of Households above $200k: 9.68&#34;,&#34;GEOID: 06075011800&lt;br&gt;Percent of Households above $200k: 1.15&#34;,&#34;GEOID: 06075012202&lt;br&gt;Percent of Households above $200k: 1.6&#34;,&#34;GEOID: 06075012502&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075012901&lt;br&gt;Percent of Households above $200k: 20.17&#34;,&#34;GEOID: 06075016600&lt;br&gt;Percent of Households above $200k: 10.06&#34;,&#34;GEOID: 06075017101&lt;br&gt;Percent of Households above $200k: 28.86&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent of Households above $200k: 6.88&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent of Households above $200k: 26.68&#34;,&#34;GEOID: 06075021300&lt;br&gt;Percent of Households above $200k: 36.22&#34;,&#34;GEOID: 06075022704&lt;br&gt;Percent of Households above $200k: 30.84&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent of Households above $200k: 8.75&#34;,&#34;GEOID: 06075025701&lt;br&gt;Percent of Households above $200k: 10.86&#34;,&#34;GEOID: 06075026004&lt;br&gt;Percent of Households above $200k: 8.56&#34;,&#34;GEOID: 06075031100&lt;br&gt;Percent of Households above $200k: 16.24&#34;,&#34;GEOID: 06075032602&lt;br&gt;Percent of Households above $200k: 9.9&#34;,&#34;GEOID: 06075033201&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075035202&lt;br&gt;Percent of Households above $200k: 5.6&#34;,&#34;GEOID: 06075042601&lt;br&gt;Percent of Households above $200k: 11.62&#34;,&#34;GEOID: 06075047801&lt;br&gt;Percent of Households above $200k: 3.6&#34;,&#34;GEOID: 06075060502&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075980200&lt;br&gt;Percent of Households above $200k: 20.88&#34;,&#34;GEOID: 06075010800&lt;br&gt;Percent of Households above $200k: 11.76&#34;,&#34;GEOID: 06075015500&lt;br&gt;Percent of Households above $200k: 7.37&#34;,&#34;GEOID: 06075021200&lt;br&gt;Percent of Households above $200k: 32.58&#34;,&#34;GEOID: 06075023300&lt;br&gt;Percent of Households above $200k: 4.7&#34;,&#34;GEOID: 06075026100&lt;br&gt;Percent of Households above $200k: 7.14&#34;,&#34;GEOID: 06075032801&lt;br&gt;Percent of Households above $200k: 7.23&#34;,&#34;GEOID: 06075047702&lt;br&gt;Percent of Households above $200k: 4.72&#34;,&#34;GEOID: 06075011901&lt;br&gt;Percent of Households above $200k: 7.95&#34;,&#34;GEOID: 06075013000&lt;br&gt;Percent of Households above $200k: 14.95&#34;,&#34;GEOID: 06075015700&lt;br&gt;Percent of Households above $200k: 13.41&#34;,&#34;GEOID: 06075022702&lt;br&gt;Percent of Households above $200k: 32.09&#34;,&#34;GEOID: 06075035100&lt;br&gt;Percent of Households above $200k: 14.86&#34;,&#34;GEOID: 06075047901&lt;br&gt;Percent of Households above $200k: 12.1&#34;,&#34;GEOID: 06075012100&lt;br&gt;Percent of Households above $200k: 3.56&#34;,&#34;GEOID: 06075012800&lt;br&gt;Percent of Households above $200k: 25.41&#34;,&#34;GEOID: 06075023003&lt;br&gt;Percent of Households above $200k: 9.1&#34;,&#34;GEOID: 06075032700&lt;br&gt;Percent of Households above $200k: 11.49&#34;,&#34;GEOID: 06075047701&lt;br&gt;Percent of Households above $200k: 5.58&#34;,&#34;GEOID: 06075010300&lt;br&gt;Percent of Households above $200k: 17.83&#34;,&#34;GEOID: 06075011100&lt;br&gt;Percent of Households above $200k: 4.37&#34;,&#34;GEOID: 06075011300&lt;br&gt;Percent of Households above $200k: 3.09&#34;,&#34;GEOID: 06075012302&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075012402&lt;br&gt;Percent of Households above $200k: 1.59&#34;,&#34;GEOID: 06075012902&lt;br&gt;Percent of Households above $200k: 16.76&#34;,&#34;GEOID: 06075013102&lt;br&gt;Percent of Households above $200k: 25.09&#34;,&#34;GEOID: 06075013200&lt;br&gt;Percent of Households above $200k: 39.83&#34;,&#34;GEOID: 06075013400&lt;br&gt;Percent of Households above $200k: 26.97&#34;,&#34;GEOID: 06075015300&lt;br&gt;Percent of Households above $200k: 21.84&#34;,&#34;GEOID: 06075015400&lt;br&gt;Percent of Households above $200k: 16.58&#34;,&#34;GEOID: 06075015801&lt;br&gt;Percent of Households above $200k: 6.71&#34;,&#34;GEOID: 06075016100&lt;br&gt;Percent of Households above $200k: 2.95&#34;,&#34;GEOID: 06075016200&lt;br&gt;Percent of Households above $200k: 2.9&#34;,&#34;GEOID: 06075016900&lt;br&gt;Percent of Households above $200k: 9.48&#34;,&#34;GEOID: 06075017802&lt;br&gt;Percent of Households above $200k: 12.03&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent of Households above $200k: 15.57&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent of Households above $200k: 25.02&#34;,&#34;GEOID: 06075021500&lt;br&gt;Percent of Households above $200k: 18.16&#34;,&#34;GEOID: 06075021700&lt;br&gt;Percent of Households above $200k: 18.79&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent of Households above $200k: 3.11&#34;,&#34;GEOID: 06075023102&lt;br&gt;Percent of Households above $200k: 2.41&#34;,&#34;GEOID: 06075023200&lt;br&gt;Percent of Households above $200k: 1.16&#34;,&#34;GEOID: 06075025600&lt;br&gt;Percent of Households above $200k: 3.5&#34;,&#34;GEOID: 06075025702&lt;br&gt;Percent of Households above $200k: 9.86&#34;,&#34;GEOID: 06075026003&lt;br&gt;Percent of Households above $200k: 3.79&#34;,&#34;GEOID: 06075026200&lt;br&gt;Percent of Households above $200k: 7.67&#34;,&#34;GEOID: 06075030301&lt;br&gt;Percent of Households above $200k: 17.29&#34;,&#34;GEOID: 06075030500&lt;br&gt;Percent of Households above $200k: 18.07&#34;,&#34;GEOID: 06075030700&lt;br&gt;Percent of Households above $200k: 29.42&#34;,&#34;GEOID: 06075032601&lt;br&gt;Percent of Households above $200k: 7.36&#34;,&#34;GEOID: 06075033203&lt;br&gt;Percent of Households above $200k: 2.23&#34;,&#34;GEOID: 06075040100&lt;br&gt;Percent of Households above $200k: 17.59&#34;,&#34;GEOID: 06075042800&lt;br&gt;Percent of Households above $200k: 43.71&#34;,&#34;GEOID: 06075047600&lt;br&gt;Percent of Households above $200k: 11.13&#34;,&#34;GEOID: 06075060100&lt;br&gt;Percent of Households above $200k: 28.3&#34;,&#34;GEOID: 06075060700&lt;br&gt;Percent of Households above $200k: 19.44&#34;,&#34;GEOID: 06075980300&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075025403&lt;br&gt;Percent of Households above $200k: 11.33&#34;,&#34;GEOID: 06075025900&lt;br&gt;Percent of Households above $200k: 12.46&#34;,&#34;GEOID: 06075026002&lt;br&gt;Percent of Households above $200k: 7.08&#34;,&#34;GEOID: 06075026301&lt;br&gt;Percent of Households above $200k: 6.09&#34;,&#34;GEOID: 06075026303&lt;br&gt;Percent of Households above $200k: 5.78&#34;,&#34;GEOID: 06075030102&lt;br&gt;Percent of Households above $200k: 19.83&#34;,&#34;GEOID: 06075030202&lt;br&gt;Percent of Households above $200k: 3.51&#34;,&#34;GEOID: 06075030302&lt;br&gt;Percent of Households above $200k: 22.94&#34;,&#34;GEOID: 06075010500&lt;br&gt;Percent of Households above $200k: 22.24&#34;,&#34;GEOID: 06075010700&lt;br&gt;Percent of Households above $200k: 3.6&#34;,&#34;GEOID: 06075011200&lt;br&gt;Percent of Households above $200k: 19.83&#34;,&#34;GEOID: 06075011700&lt;br&gt;Percent of Households above $200k: 8.03&#34;,&#34;GEOID: 06075011902&lt;br&gt;Percent of Households above $200k: 2.72&#34;,&#34;GEOID: 06075012301&lt;br&gt;Percent of Households above $200k: 0.52&#34;,&#34;GEOID: 06075012401&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075012602&lt;br&gt;Percent of Households above $200k: 27.33&#34;,&#34;GEOID: 06075016400&lt;br&gt;Percent of Households above $200k: 10.2&#34;,&#34;GEOID: 06075016700&lt;br&gt;Percent of Households above $200k: 14.91&#34;,&#34;GEOID: 06075017601&lt;br&gt;Percent of Households above $200k: 2.62&#34;,&#34;GEOID: 06075017801&lt;br&gt;Percent of Households above $200k: 10.69&#34;,&#34;GEOID: 06075017902&lt;br&gt;Percent of Households above $200k: 7.79&#34;,&#34;GEOID: 06075020401&lt;br&gt;Percent of Households above $200k: 25.47&#34;,&#34;GEOID: 06075020402&lt;br&gt;Percent of Households above $200k: 13.28&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent of Households above $200k: 10.24&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent of Households above $200k: 31.81&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent of Households above $200k: 28.36&#34;,&#34;GEOID: 06075021600&lt;br&gt;Percent of Households above $200k: 20.59&#34;,&#34;GEOID: 06075021800&lt;br&gt;Percent of Households above $200k: 23.67&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent of Households above $200k: 5.35&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent of Households above $200k: 5.02&#34;,&#34;GEOID: 06075023001&lt;br&gt;Percent of Households above $200k: 3.66&#34;,&#34;GEOID: 06075025200&lt;br&gt;Percent of Households above $200k: 22.11&#34;,&#34;GEOID: 06075025401&lt;br&gt;Percent of Households above $200k: 8.5&#34;,&#34;GEOID: 06075030400&lt;br&gt;Percent of Households above $200k: 33.12&#34;,&#34;GEOID: 06075030600&lt;br&gt;Percent of Households above $200k: 33.01&#34;,&#34;GEOID: 06075031202&lt;br&gt;Percent of Households above $200k: 6.37&#34;,&#34;GEOID: 06075031301&lt;br&gt;Percent of Households above $200k: 14.27&#34;,&#34;GEOID: 06075031400&lt;br&gt;Percent of Households above $200k: 5.49&#34;,&#34;GEOID: 06075032802&lt;br&gt;Percent of Households above $200k: 5.65&#34;,&#34;GEOID: 06075032902&lt;br&gt;Percent of Households above $200k: 15.85&#34;,&#34;GEOID: 06075033100&lt;br&gt;Percent of Households above $200k: 20.14&#34;,&#34;GEOID: 06075033204&lt;br&gt;Percent of Households above $200k: 0.75&#34;,&#34;GEOID: 06075035400&lt;br&gt;Percent of Households above $200k: 5.58&#34;,&#34;GEOID: 06075040200&lt;br&gt;Percent of Households above $200k: 15.25&#34;,&#34;GEOID: 06075042700&lt;br&gt;Percent of Households above $200k: 8.55&#34;,&#34;GEOID: 06075045200&lt;br&gt;Percent of Households above $200k: 14.57&#34;,&#34;GEOID: 06075047802&lt;br&gt;Percent of Households above $200k: 13.13&#34;,&#34;GEOID: 06075047902&lt;br&gt;Percent of Households above $200k: 9.84&#34;,&#34;GEOID: 06075060400&lt;br&gt;Percent of Households above $200k: 5.72&#34;,&#34;GEOID: 06075061200&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075061400&lt;br&gt;Percent of Households above $200k: 23.61&#34;,&#34;GEOID: 06075980401&lt;br&gt;Percent of Households above $200k: NaN&#34;,&#34;GEOID: 06075980501&lt;br&gt;Percent of Households above $200k: 5.33&#34;,&#34;GEOID: 06075010200&lt;br&gt;Percent of Households above $200k: 21.76&#34;,&#34;GEOID: 06075012201&lt;br&gt;Percent of Households above $200k: 0.54&#34;,&#34;GEOID: 06075013101&lt;br&gt;Percent of Households above $200k: 25.5&#34;,&#34;GEOID: 06075013500&lt;br&gt;Percent of Households above $200k: 20.37&#34;,&#34;GEOID: 06075015900&lt;br&gt;Percent of Households above $200k: 6.35&#34;,&#34;GEOID: 06075016801&lt;br&gt;Percent of Households above $200k: 13.41&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent of Households above $200k: 8.11&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent of Households above $200k: 7.85&#34;,&#34;GEOID: 06075025300&lt;br&gt;Percent of Households above $200k: 20.57&#34;,&#34;GEOID: 06075025500&lt;br&gt;Percent of Households above $200k: 10.77&#34;,&#34;GEOID: 06075026001&lt;br&gt;Percent of Households above $200k: 8.13&#34;,&#34;GEOID: 06075026404&lt;br&gt;Percent of Households above $200k: 0.78&#34;,&#34;GEOID: 06075030101&lt;br&gt;Percent of Households above $200k: 13.42&#34;,&#34;GEOID: 06075031000&lt;br&gt;Percent of Households above $200k: 12.23&#34;,&#34;GEOID: 06075035300&lt;br&gt;Percent of Households above $200k: 6.41&#34;,&#34;GEOID: 06075061500&lt;br&gt;Percent of Households above $200k: 28.02&#34;,&#34;GEOID: 06075012501&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075023103&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075045100&lt;br&gt;Percent of Households above $200k: 10.3&#34;,&#34;GEOID: 06075016802&lt;br&gt;Percent of Households above $200k: 16.19&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent of Households above $200k: 4.84&#34;,&#34;GEOID: 06075022600&lt;br&gt;Percent of Households above $200k: 31.6&#34;,&#34;GEOID: 06075025800&lt;br&gt;Percent of Households above $200k: 0.83&#34;,&#34;GEOID: 06075026401&lt;br&gt;Percent of Households above $200k: 2.77&#34;,&#34;GEOID: 06075010400&lt;br&gt;Percent of Households above $200k: 13.45&#34;,&#34;GEOID: 06075030900&lt;br&gt;Percent of Households above $200k: 28.98&#34;,&#34;GEOID: 06075035201&lt;br&gt;Percent of Households above $200k: 4.31&#34;,&#34;GEOID: 06075061000&lt;br&gt;Percent of Households above $200k: 7.84&#34;,&#34;GEOID: 06075980900&lt;br&gt;Percent of Households above $200k: 31.32&#34;,&#34;GEOID: 06075015100&lt;br&gt;Percent of Households above $200k: 13.06&#34;,&#34;GEOID: 06075016500&lt;br&gt;Percent of Households above $200k: 15.95&#34;,&#34;GEOID: 06075020500&lt;br&gt;Percent of Households above $200k: 22.96&#34;,&#34;GEOID: 06075025402&lt;br&gt;Percent of Households above $200k: 15.29&#34;,&#34;GEOID: 06075026302&lt;br&gt;Percent of Households above $200k: 4.72&#34;,&#34;GEOID: 06075031201&lt;br&gt;Percent of Households above $200k: 6.98&#34;,&#34;GEOID: 06075033000&lt;br&gt;Percent of Households above $200k: 13.04&#34;,&#34;GEOID: 06075012700&lt;br&gt;Percent of Households above $200k: 30.27&#34;,&#34;GEOID: 06075017102&lt;br&gt;Percent of Households above $200k: 19.69&#34;,&#34;GEOID: 06075030800&lt;br&gt;Percent of Households above $200k: 29.24&#34;,&#34;GEOID: 06075018000&lt;br&gt;Percent of Households above $200k: 28.65&#34;,&#34;GEOID: 06075026403&lt;br&gt;Percent of Households above $200k: 0.98&#34;,&#34;GEOID: 06075015802&lt;br&gt;Percent of Households above $200k: 8.26&#34;,&#34;GEOID: 06075030201&lt;br&gt;Percent of Households above $200k: 7.12&#34;,&#34;GEOID: 06075980600&lt;br&gt;Percent of Households above $200k: 0&#34;,&#34;GEOID: 06075023400&lt;br&gt;Percent of Households above $200k: 1.74&#34;,&#34;GEOID: 06075017000&lt;br&gt;Percent of Households above $200k: 22.47&#34;,&#34;GEOID: 06075025100&lt;br&gt;Percent of Households above $200k: 10.32&#34;,&#34;GEOID: 06075010900&lt;br&gt;Percent of Households above $200k: 22.73&#34;,&#34;GEOID: 06075015200&lt;br&gt;Percent of Households above $200k: 13.5&#34;,&#34;GEOID: 06075032901&lt;br&gt;Percent of Households above $200k: 5.37&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #FFFFD9 0%, #EFF9B4 11.4378238341969%, #CEECB4 22.8756476683938%, #93D4B9 34.3134715025907%, #5ABEC1 45.7512953367876%, #30A1C2 57.1891191709845%, #2378B4 68.6269430051813%, #264DA0 80.0647668393782%, #1C2C80 91.5025906735751%, #081D58 &#34;],&#34;labels&#34;:[&#34;0%&#34;,&#34;5%&#34;,&#34;10%&#34;,&#34;15%&#34;,&#34;20%&#34;,&#34;25%&#34;,&#34;30%&#34;,&#34;35%&#34;,&#34;40%&#34;],&#34;na_color&#34;:&#34;#808080&#34;,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent of Households&lt;br&gt;above $200k&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0,&#34;p_n&#34;:0.915025906735751},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.692736,37.8633422971737],&#34;lng&#34;:[-123.013916,-122.327561619753]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;p&gt;See? You can play around with the map! Zoom in, click on specific tracts to see demographic information, etc!&lt;/p&gt;
&lt;p&gt;If you’d like to save your map, follow these steps:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# library(htmlwidgets)
# saveWidget(map1, file=&amp;quot;map1.html&amp;quot;, selfcontained=FALSE)
# saveWidget(map2, file=&amp;quot;map2.html&amp;quot;, selfcontained=FALSE)
# saveWidget(map3, file=&amp;quot;map3.html&amp;quot;, selfcontained=FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now let’s actually try to get some census data of interest to our own study! I’m just looking at one neighborhood, so I’ll be subsetting at the very end by census tract.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# How about median values for specified owner-occupied housing units?
homevalue &amp;lt;- acs.fetch(endyear = 2013, geography = geo,
                table.number = &amp;quot;B25077&amp;quot;, col.names = &amp;quot;pretty&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## Warning: NAs introduced by coercion&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# homevalue &amp;lt;- acs.fetch(endyear=2010, span = 0, geography=geo,
# keyword=&amp;quot;HOUSING&amp;quot;, dataset = &amp;quot;sf3&amp;quot;,col.names = &amp;quot;pretty&amp;quot;)

# homevalue&amp;lt;-acs.fetch(endyear = 2000, dataset = &amp;quot;sf3&amp;quot;, geography = geo,
#                table.number = &amp;quot;H076&amp;quot;, col.names = &amp;quot;pretty&amp;quot;)


# the resulting &amp;quot;income&amp;quot; object is not a data.frame it&amp;#39;s a list
# to see what&amp;#39;s available

# names(attributes(homevalue))
# attr(homevalue, &amp;quot;acs.colnames&amp;quot;)


# convert to a data.frame for merging
homevalue_df &amp;lt;- data.frame(paste0(str_pad(homevalue@geography$state, 2, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(homevalue@geography$county, 3, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(homevalue@geography$tract, 6, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;)), 
                        homevalue@estimate[,c(&amp;quot;Median Value (Dollars) for Owner-Occupied Housing Units: Median value (dollars)&amp;quot;)], 
                        stringsAsFactors = FALSE)

homevalue_df &amp;lt;- select(homevalue_df, 1:2)
rownames(homevalue_df)&amp;lt;-1:nrow(homevalue_df)
names(homevalue_df)&amp;lt;-c(&amp;quot;GEOID&amp;quot;, &amp;quot;Median Value for Owner-Occupied Housing Unit&amp;quot;)
# income_df$percent &amp;lt;- 100*(income_df$over_200/income_df$total)
homevalue_merged&amp;lt;- geo_join(tracts, homevalue_df, &amp;quot;GEOID&amp;quot;, &amp;quot;GEOID&amp;quot;)
# get it to just those tracts that we want (help from http://rprogramming.net/subset-data-in-r/)
homevalue_mission&amp;lt;-subset(homevalue_merged, NAME==&amp;quot;177&amp;quot; | NAME==&amp;quot;201&amp;quot;| NAME==&amp;quot;202&amp;quot;| NAME==&amp;quot;203&amp;quot;| NAME==&amp;quot;206&amp;quot;| NAME==&amp;quot;207&amp;quot;| NAME==&amp;quot;208&amp;quot;| NAME==&amp;quot;209&amp;quot;| NAME==&amp;quot;210&amp;quot;| NAME==&amp;quot;211&amp;quot;| NAME==&amp;quot;214&amp;quot;| NAME==&amp;quot;228.01&amp;quot;| NAME==&amp;quot;228.02&amp;quot;| NAME==&amp;quot;228.03&amp;quot;| NAME==&amp;quot;229.01&amp;quot;| NAME==&amp;quot;229.02&amp;quot;| NAME==&amp;quot;229.03&amp;quot;)

# there are some tracts with no land that we should exclude
homevalue_mission &amp;lt;- homevalue_mission[homevalue_mission$ALAND&amp;gt;0,]&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s try mapping this subset…&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# library(leaflet)
popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, homevalue_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Median Value for Owner-Occupied Housing: &amp;quot;, round(homevalue_mission$Median.Value.for.Owner.Occupied.Housing.Unit,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = homevalue_mission$Median.Value.for.Owner.Occupied.Housing.Unit
)

Mission_Home_Value&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = homevalue_mission, 
              fillColor = ~pal(Median.Value.for.Owner.Occupied.Housing.Unit), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = homevalue_mission$Median.Value.for.Owner.Occupied.Housing.Unit, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;2013 Median Property Value&amp;quot;,
            labFormat = labelFormat(prefix = &amp;quot;$&amp;quot;)) 
Mission_Home_Value&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-a62530af4d124692566e&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-a62530af4d124692566e&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#ACDEB7&#34;,&#34;#237CB6&#34;,&#34;#EEF8B3&#34;,&#34;#FDFED3&#34;,&#34;#081D58&#34;,&#34;#CFECB3&#34;,&#34;#46B7C4&#34;,&#34;#237BB6&#34;,&#34;#E1F3B2&#34;,&#34;#2373B2&#34;,&#34;#081D58&#34;,&#34;#081D58&#34;,&#34;#39ABC3&#34;,&#34;#C9EAB4&#34;,&#34;#FFFFD9&#34;,&#34;#84CFBB&#34;,&#34;#F0F9B7&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Median Value for Owner-Occupied Housing: 732800&#34;,&#34;GEOID: 06075021000&lt;br&gt;Median Value for Owner-Occupied Housing: 876400&#34;,&#34;GEOID: 06075022801&lt;br&gt;Median Value for Owner-Occupied Housing: 664000&#34;,&#34;GEOID: 06075020200&lt;br&gt;Median Value for Owner-Occupied Housing: 625000&#34;,&#34;GEOID: 06075020600&lt;br&gt;Median Value for Owner-Occupied Housing: 1000001&#34;,&#34;GEOID: 06075022803&lt;br&gt;Median Value for Owner-Occupied Housing: 703900&#34;,&#34;GEOID: 06075020300&lt;br&gt;Median Value for Owner-Occupied Housing: 806600&#34;,&#34;GEOID: 06075020700&lt;br&gt;Median Value for Owner-Occupied Housing: 876800&#34;,&#34;GEOID: 06075022903&lt;br&gt;Median Value for Owner-Occupied Housing: 681400&#34;,&#34;GEOID: 06075020800&lt;br&gt;Median Value for Owner-Occupied Housing: 884800&#34;,&#34;GEOID: 06075021100&lt;br&gt;Median Value for Owner-Occupied Housing: 1000001&#34;,&#34;GEOID: 06075021400&lt;br&gt;Median Value for Owner-Occupied Housing: 1000001&#34;,&#34;GEOID: 06075022802&lt;br&gt;Median Value for Owner-Occupied Housing: 823700&#34;,&#34;GEOID: 06075022902&lt;br&gt;Median Value for Owner-Occupied Housing: 711400&#34;,&#34;GEOID: 06075020100&lt;br&gt;Median Value for Owner-Occupied Housing: 618200&#34;,&#34;GEOID: 06075022901&lt;br&gt;Median Value for Owner-Occupied Housing: 758200&#34;,&#34;GEOID: 06075020900&lt;br&gt;Median Value for Owner-Occupied Housing: 659200&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #F3FABE 8.32894623115183%, #D2EDB3 21.4247736386233%, #92D4B9 34.5206010460947%, #50BAC2 47.6164284535661%, #2596C1 60.7122558610376%, #2363AA 73.808083268509%, #253695 86.9039106759804%, #081D58 99.9997380834518%, #081D58 &#34;],&#34;labels&#34;:[&#34;$650,000&#34;,&#34;$700,000&#34;,&#34;$750,000&#34;,&#34;$800,000&#34;,&#34;$850,000&#34;,&#34;$900,000&#34;,&#34;$950,000&#34;,&#34;$1,000,000&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;2013 Median Property Value&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.0832894623115183,&#34;p_n&#34;:0.999997380834519},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;p&gt;If you want, save your map:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# library(htmlwidgets)
# saveWidget(Mission_Home_Value, file=&amp;quot;Mission_Home_Value.html&amp;quot;, selfcontained=FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;exploring-other-demographics&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Exploring other demographics&lt;/h4&gt;
&lt;p&gt;Let’s get additional information – what about ethnicity / race? For consistancy, using the “Hispanic or Latino, and not Hispanic or Latino by Race” in the census year 2013.&lt;/p&gt;
&lt;p&gt;FYI: The ‘names’ and ‘attr’ commands are useful when you are creating a data frame, subsetting it, etc. as they tell you the things present in the census table you’ve just downloaded. You don’t have to do the commands everytime but the first time it is useful so you can go further with your data frame and maps.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;hispanicorlation &amp;lt;- acs.fetch(endyear = 2013, geography = geo,
                table.number = &amp;quot;B03002&amp;quot;, col.names = &amp;quot;pretty&amp;quot;)

# names(attributes(hispanicorlation))

# attr(hispanicorlation, &amp;quot;acs.colnames&amp;quot;)

# convert to a data.frame for merging
hispanicorlation_df &amp;lt;- data.frame(paste0(str_pad(hispanicorlation@geography$state, 2, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(hispanicorlation@geography$county, 3, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(hispanicorlation@geography$tract, 6, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;)), 
                        hispanicorlation@estimate[,c(&amp;quot;Hispanic or Latino by Race: Total:&amp;quot;,
&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino:&amp;quot;,&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino: White alone&amp;quot;,&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino: Black or African American alone&amp;quot;,&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino: Asian alone&amp;quot;,&amp;quot;Hispanic or Latino by Race: Hispanic or Latino:&amp;quot;)], 
                        stringsAsFactors = FALSE)

hispanicorlation_df &amp;lt;- select(hispanicorlation_df, 1:7)
rownames(hispanicorlation_df)&amp;lt;-1:nrow(hispanicorlation_df)
names(hispanicorlation_df)&amp;lt;-c(&amp;quot;GEOID&amp;quot;, &amp;quot;Hispanic_or_Latino_by_Race_Total&amp;quot;,
&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Black_or_African_American_alone&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Asian_alone&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Hispanic_or_Latino&amp;quot;)
# Get percentages for each group
hispanicorlation_df$percenthispanic &amp;lt;- 100*(hispanicorlation_df$Hispanic_or_Latino_by_Race_Hispanic_or_Latino/hispanicorlation_df$Hispanic_or_Latino_by_Race_Total)

hispanicorlation_df$percent_white &amp;lt;- 100*(hispanicorlation_df$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone/hispanicorlation_df$Hispanic_or_Latino_by_Race_Total)

hispanicorlation_df$percent_black_AA &amp;lt;- 100*(hispanicorlation_df$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Black_or_African_American_alone/hispanicorlation_df$Hispanic_or_Latino_by_Race_Total)

hispanicorlation_df$percent_asian &amp;lt;- 100*(hispanicorlation_df$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Asian_alone/hispanicorlation_df$Hispanic_or_Latino_by_Race_Total)

demographics&amp;lt;- geo_join(tracts, hispanicorlation_df, &amp;quot;GEOID&amp;quot;, &amp;quot;GEOID&amp;quot;)
# get it to just those tracts that we want (help from http://rprogramming.net/subset-data-in-r/)
demographics_mission&amp;lt;-subset(demographics, NAME==&amp;quot;177&amp;quot; | NAME==&amp;quot;201&amp;quot;| NAME==&amp;quot;202&amp;quot;| NAME==&amp;quot;203&amp;quot;| NAME==&amp;quot;206&amp;quot;| NAME==&amp;quot;207&amp;quot;| NAME==&amp;quot;208&amp;quot;| NAME==&amp;quot;209&amp;quot;| NAME==&amp;quot;210&amp;quot;| NAME==&amp;quot;211&amp;quot;| NAME==&amp;quot;214&amp;quot;| NAME==&amp;quot;228.01&amp;quot;| NAME==&amp;quot;228.02&amp;quot;| NAME==&amp;quot;228.03&amp;quot;| NAME==&amp;quot;229.01&amp;quot;| NAME==&amp;quot;229.02&amp;quot;| NAME==&amp;quot;229.03&amp;quot;)

# there are some tracts with no land that we should exclude
demographics_mission &amp;lt;- demographics_mission[demographics_mission$ALAND&amp;gt;0,]
# Save this
# write.csv(demographics_mission,file=paste(&amp;quot;2013 Mission Demographics.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;mapping-other-types-of-census-data&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Mapping other types of census data&lt;/h4&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, demographics_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent Hispanic or Latino&amp;quot;, round(demographics_mission$percenthispanic,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = demographics_mission$percenthispanic
)

hispanicorlatinopop&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = demographics_mission, 
              fillColor = ~pal(percenthispanic), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = demographics_mission$percenthispanic, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent Hispanic&amp;lt;br&amp;gt;or Latino&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
hispanicorlatinopop&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-ced570f557ea9baa8adb&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-ced570f557ea9baa8adb&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#57BDC1&#34;,&#34;#FDFED5&#34;,&#34;#69C3BF&#34;,&#34;#BEE5B5&#34;,&#34;#FFFFD8&#34;,&#34;#102369&#34;,&#34;#F6FCC5&#34;,&#34;#FFFFD9&#34;,&#34;#2370B0&#34;,&#34;#1F2F86&#34;,&#34;#E7F5B2&#34;,&#34;#FEFFD7&#34;,&#34;#3AACC3&#34;,&#34;#2371B1&#34;,&#34;#2367AC&#34;,&#34;#081D58&#34;,&#34;#26449C&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent Hispanic or Latino33.25&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent Hispanic or Latino12.35&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent Hispanic or Latino31.6&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent Hispanic or Latino24.19&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent Hispanic or Latino11.89&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent Hispanic or Latino56.41&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent Hispanic or Latino14.69&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent Hispanic or Latino11.81&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent Hispanic or Latino44.48&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent Hispanic or Latino53.57&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent Hispanic or Latino18.59&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent Hispanic or Latino12.12&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent Hispanic or Latino36.52&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent Hispanic or Latino44.31&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent Hispanic or Latino45.5&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent Hispanic or Latino58.05&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent Hispanic or Latino50.05&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #F5FBC3 6.89247882140931%, #DDF2B2 17.7057271448412%, #B4E1B6 28.5189754682731%, #78CABC 39.332223791705%, #41B6C4 50.1454721151369%, #2495C1 60.9587204385689%, #236BAE 71.7719687620008%, #26459C 82.5852170854327%, #182977 93.3984654088646%, #081D58 &#34;],&#34;labels&#34;:[&#34;15%&#34;,&#34;20%&#34;,&#34;25%&#34;,&#34;30%&#34;,&#34;35%&#34;,&#34;40%&#34;,&#34;45%&#34;,&#34;50%&#34;,&#34;55%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent Hispanic&lt;br&gt;or Latino&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.0689247882140931,&#34;p_n&#34;:0.933984654088646},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;/div&gt;
&lt;div id=&#34;age-by-language-spoken-at-home-for-the-population-5-years-and-over&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Age by Language Spoken at Home for the Population 5 Years and Over&lt;/h4&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;language &amp;lt;- acs.fetch(endyear = 2013, geography = geo,
                table.number = &amp;quot;B16007&amp;quot;, col.names = &amp;quot;pretty&amp;quot;)

# names(attributes(language))

# attr(language, &amp;quot;acs.colnames&amp;quot;)

# convert to a data.frame for merging
language_df &amp;lt;- data.frame(paste0(str_pad(language@geography$state, 2, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(language@geography$county, 3, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(language@geography$tract, 6, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;)), 
                        language@estimate[,c(&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: Total:&amp;quot;,&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: 5 to 17 years: Speak only English&amp;quot;,&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: 5 to 17 years: Speak Spanish&amp;quot;,&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: 18 to 64 years: Speak only English&amp;quot;,&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: 18 to 64 years: Speak Spanish&amp;quot;,&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: 65 years and over: Speak only English&amp;quot;,&amp;quot;Age by Language Spoken at Home for the Population 5+ Yrs: 65 years and over: Speak Spanish&amp;quot;)], 
                        stringsAsFactors = FALSE)

language_df &amp;lt;- select(language_df, 1:8)
rownames(language_df)&amp;lt;-1:nrow(language_df)
names(language_df)&amp;lt;-c(&amp;quot;GEOID&amp;quot;, &amp;quot;Age_by_Language_Spoken_at_Home_Total&amp;quot;,
&amp;quot;5_to_17_years_Speak_only_English&amp;quot;,&amp;quot;5_to_17_years_Speak_Spanish&amp;quot;,&amp;quot;18_to_64_years_Speak_only_English&amp;quot;,&amp;quot;18_to_64_years_Speak_Spanish&amp;quot;,&amp;quot;65_years_and_over_Speak_only_English&amp;quot;,&amp;#39;65_years_and_over_Speak_Spanish&amp;#39;)
# Get percentages for each group
language_df$englishtotal &amp;lt;- language_df$`5_to_17_years_Speak_only_English`+language_df$`18_to_64_years_Speak_only_English`+language_df$`65_years_and_over_Speak_only_English`

language_df$spanishtotal &amp;lt;- language_df$`5_to_17_years_Speak_Spanish`+language_df$`18_to_64_years_Speak_Spanish`+language_df$`65_years_and_over_Speak_Spanish`

language_df$percentenglish &amp;lt;- 100*(language_df$englishtotal/language_df$Age_by_Language_Spoken_at_Home_Total)

language_df$percentspanish &amp;lt;- 100*(language_df$spanishtotal/language_df$Age_by_Language_Spoken_at_Home_Total)

language&amp;lt;- geo_join(tracts, language_df, &amp;quot;GEOID&amp;quot;, &amp;quot;GEOID&amp;quot;)
# get it to just those tracts that we want (help from http://rprogramming.net/subset-data-in-r/)
language_mission&amp;lt;-subset(language, NAME==&amp;quot;177&amp;quot; | NAME==&amp;quot;201&amp;quot;| NAME==&amp;quot;202&amp;quot;| NAME==&amp;quot;203&amp;quot;| NAME==&amp;quot;206&amp;quot;| NAME==&amp;quot;207&amp;quot;| NAME==&amp;quot;208&amp;quot;| NAME==&amp;quot;209&amp;quot;| NAME==&amp;quot;210&amp;quot;| NAME==&amp;quot;211&amp;quot;| NAME==&amp;quot;214&amp;quot;| NAME==&amp;quot;228.01&amp;quot;| NAME==&amp;quot;228.02&amp;quot;| NAME==&amp;quot;228.03&amp;quot;| NAME==&amp;quot;229.01&amp;quot;| NAME==&amp;quot;229.02&amp;quot;| NAME==&amp;quot;229.03&amp;quot;)

# there are some tracts with no land that we should exclude
language_mission &amp;lt;- language_mission[language_mission$ALAND&amp;gt;0,]
# Save this
# write.csv(language_mission,file=paste(&amp;quot;2013 Mission Languages.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s try to map percentages of Spanish spoken at home&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, language_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent Spanish Spoken at Home&amp;quot;, round(language_mission$percentspanish,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = language_mission$percentspanish
)

spanishathome&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = language_mission, 
              fillColor = ~pal(percentspanish), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = language_mission$percentspanish, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent Spanish&amp;lt;br&amp;gt;Spoken at Home&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
spanishathome&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-4aa702100a015ec6c6cb&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-4aa702100a015ec6c6cb&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#41B6C4&#34;,&#34;#F9FDCC&#34;,&#34;#51BBC2&#34;,&#34;#CEECB4&#34;,&#34;#FDFED5&#34;,&#34;#081D58&#34;,&#34;#F5FBC3&#34;,&#34;#FFFFD9&#34;,&#34;#2374B2&#34;,&#34;#2456A4&#34;,&#34;#EBF7B1&#34;,&#34;#F7FCC6&#34;,&#34;#51BAC2&#34;,&#34;#227FB7&#34;,&#34;#254FA1&#34;,&#34;#1A2B7B&#34;,&#34;#2359A6&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent Spanish Spoken at Home29.95&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent Spanish Spoken at Home9.31&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent Spanish Spoken at Home28.82&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent Spanish Spoken at Home17.71&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent Spanish Spoken at Home8.07&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent Spanish Spoken at Home52.5&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent Spanish Spoken at Home10.53&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent Spanish Spoken at Home7.44&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent Spanish Spoken at Home38.83&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent Spanish Spoken at Home42.38&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent Spanish Spoken at Home13.41&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent Spanish Spoken at Home10.06&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent Spanish Spoken at Home28.87&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent Spanish Spoken at Home37.61&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent Spanish Spoken at Home43.28&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent Spanish Spoken at Home49.12&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent Spanish Spoken at Home41.89&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #F7FCC7 5.68451929398675%, #E0F3B2 16.7804582005765%, #B8E3B6 27.8763971071663%, #79CABC 38.9723360137561%, #41B6C4 50.0682749203459%, #2395C0 61.1642138269357%, #2369AD 72.2601527335255%, #26429B 83.3560916401153%, #152772 94.4520305467051%, #081D58 &#34;],&#34;labels&#34;:[&#34;10%&#34;,&#34;15%&#34;,&#34;20%&#34;,&#34;25%&#34;,&#34;30%&#34;,&#34;35%&#34;,&#34;40%&#34;,&#34;45%&#34;,&#34;50%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent Spanish&lt;br&gt;Spoken at Home&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.0568451929398675,&#34;p_n&#34;:0.944520305467051},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;/div&gt;
&lt;div id=&#34;comparing-demographics-over-time&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Comparing demographics over time&lt;/h4&gt;
&lt;p&gt;Comparison time – how much have these things changed in two years? I’ll just look at percent Hispanic or Latino for now.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;hispanicorlation2 &amp;lt;- acs.fetch(endyear = 2015, geography = geo,
                table.number = &amp;quot;B03002&amp;quot;, col.names = &amp;quot;pretty&amp;quot;)

# Not really necessary, but if you want to check it out you can
# names(attributes(hispanicorlation2))

# attr(hispanicorlation2, &amp;quot;acs.colnames&amp;quot;)

# convert to a data.frame for merging
hispanicorlation2_df &amp;lt;- data.frame(paste0(str_pad(hispanicorlation2@geography$state, 2, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(hispanicorlation2@geography$county, 3, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;), 
                             str_pad(hispanicorlation2@geography$tract, 6, &amp;quot;left&amp;quot;, pad=&amp;quot;0&amp;quot;)), 
                        hispanicorlation2@estimate[,c(&amp;quot;Hispanic or Latino by Race: Total:&amp;quot;,
&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino:&amp;quot;,&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino: White alone&amp;quot;,&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino: Black or African American alone&amp;quot;,&amp;quot;Hispanic or Latino by Race: Not Hispanic or Latino: Asian alone&amp;quot;,&amp;quot;Hispanic or Latino by Race: Hispanic or Latino:&amp;quot;)], 
                        stringsAsFactors = FALSE)

hispanicorlation2_df &amp;lt;- select(hispanicorlation2_df, 1:7)
rownames(hispanicorlation2_df)&amp;lt;-1:nrow(hispanicorlation2_df)
names(hispanicorlation2_df)&amp;lt;-c(&amp;quot;GEOID&amp;quot;, &amp;quot;Hispanic_or_Latino_by_Race_Total&amp;quot;,
&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Black_or_African_American_alone&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Asian_alone&amp;quot;,&amp;quot;Hispanic_or_Latino_by_Race_Hispanic_or_Latino&amp;quot;)
# Get percentages for each group
hispanicorlation2_df$percenthispanic &amp;lt;- 100*(hispanicorlation2_df$Hispanic_or_Latino_by_Race_Hispanic_or_Latino/hispanicorlation2_df$Hispanic_or_Latino_by_Race_Total)

hispanicorlation2_df$percent_white &amp;lt;- 100*(hispanicorlation2_df$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone/hispanicorlation2_df$Hispanic_or_Latino_by_Race_Total)

hispanicorlation2_df$percent_black_AA &amp;lt;- 100*(hispanicorlation2_df$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Black_or_African_American_alone/hispanicorlation2_df$Hispanic_or_Latino_by_Race_Total)

hispanicorlation2_df$percent_asian &amp;lt;- 100*(hispanicorlation2_df$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_Asian_alone/hispanicorlation2_df$Hispanic_or_Latino_by_Race_Total)

demographics2&amp;lt;- geo_join(tracts, hispanicorlation2_df, &amp;quot;GEOID&amp;quot;, &amp;quot;GEOID&amp;quot;)
# get it to just those tracts that we want (help from http://rprogramming.net/subset-data-in-r/)
demographics2_mission&amp;lt;-subset(demographics2, NAME==&amp;quot;177&amp;quot; | NAME==&amp;quot;201&amp;quot;| NAME==&amp;quot;202&amp;quot;| NAME==&amp;quot;203&amp;quot;| NAME==&amp;quot;206&amp;quot;| NAME==&amp;quot;207&amp;quot;| NAME==&amp;quot;208&amp;quot;| NAME==&amp;quot;209&amp;quot;| NAME==&amp;quot;210&amp;quot;| NAME==&amp;quot;211&amp;quot;| NAME==&amp;quot;214&amp;quot;| NAME==&amp;quot;228.01&amp;quot;| NAME==&amp;quot;228.02&amp;quot;| NAME==&amp;quot;228.03&amp;quot;| NAME==&amp;quot;229.01&amp;quot;| NAME==&amp;quot;229.02&amp;quot;| NAME==&amp;quot;229.03&amp;quot;)

# there are some tracts with no land that we should exclude
demographics2_mission &amp;lt;- demographics2_mission[demographics2_mission$ALAND&amp;gt;0,]
# Save this
# write.csv(demographics2_mission,file=paste(&amp;quot;2015 Mission Demographics.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, demographics2_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent Hispanic or Latino&amp;quot;, round(demographics2_mission$percenthispanic,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = demographics2_mission$percenthispanic
)

hispanicorlatinopop2&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = demographics2_mission, 
              fillColor = ~pal(percenthispanic), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = demographics2_mission$percenthispanic, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent Hispanic&amp;lt;br&amp;gt;or Latino&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
hispanicorlatinopop2&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-07e604aed6350d7e0866&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-07e604aed6350d7e0866&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#5BBEC1&#34;,&#34;#D5EEB3&#34;,&#34;#56BCC2&#34;,&#34;#D6EFB3&#34;,&#34;#FFFFD9&#34;,&#34;#26439B&#34;,&#34;#F1FABA&#34;,&#34;#F5FBC3&#34;,&#34;#2366AC&#34;,&#34;#236EB0&#34;,&#34;#FCFED2&#34;,&#34;#F8FCC9&#34;,&#34;#2187BB&#34;,&#34;#227DB7&#34;,&#34;#2088BC&#34;,&#34;#081D58&#34;,&#34;#2456A4&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent Hispanic or Latino33.81&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent Hispanic or Latino20.64&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent Hispanic or Latino34.35&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent Hispanic or Latino20.47&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent Hispanic or Latino9.93&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent Hispanic or Latino53.37&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent Hispanic or Latino14.95&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent Hispanic or Latino13.52&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent Hispanic or Latino48.15&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent Hispanic or Latino47.13&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent Hispanic or Latino11.12&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent Hispanic or Latino12.52&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent Hispanic or Latino43.95&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent Hispanic or Latino45.15&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent Hispanic or Latino43.78&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent Hispanic or Latino62.28&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent Hispanic or Latino50.48&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #FFFFD9 0.127060974197249%, #D9F0B3 19.231293255919%, #7CCBBC 38.3355255376407%, #2FA0C2 57.4397578193624%, #2359A6 76.5439901010841%, #12256C 95.6482223828059%, #081D58 &#34;],&#34;labels&#34;:[&#34;10%&#34;,&#34;20%&#34;,&#34;30%&#34;,&#34;40%&#34;,&#34;50%&#34;,&#34;60%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent Hispanic&lt;br&gt;or Latino&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.00127060974197249,&#34;p_n&#34;:0.956482223828059},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;p&gt;Do appear to be some changes! What about percent White?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Start with 2013 data
popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, demographics_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent White&amp;quot;, round(demographics_mission$percent_white,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = demographics_mission$percent_white
)

whitepop&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = demographics_mission, 
              fillColor = ~pal(percent_white), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = demographics_mission$percent_white, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent White&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
whitepop&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-80f326e50b7dd4582836&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-80f326e50b7dd4582836&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#63C1C0&#34;,&#34;#243391&#34;,&#34;#6EC5BE&#34;,&#34;#52BBC2&#34;,&#34;#081D58&#34;,&#34;#F8FCC8&#34;,&#34;#26459C&#34;,&#34;#0C205F&#34;,&#34;#ADDEB7&#34;,&#34;#FCFED3&#34;,&#34;#203089&#34;,&#34;#1A2B7D&#34;,&#34;#D7EFB3&#34;,&#34;#D7EFB3&#34;,&#34;#FEFFD8&#34;,&#34;#FFFFD9&#34;,&#34;#F0F9B6&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent White48.57&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent White68.78&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent White47.39&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent White50.02&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent White74.24&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent White30.7&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent White66.22&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent White73.55&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent White42&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent White29.15&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent White69.51&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent White70.65&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent White37.36&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent White37.43&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent White28.5&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent White28.31&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent White33.27&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #FAFDCD 3.68789306470377%, #E7F6B2 14.5719835018919%, #C5E8B4 25.4560739390801%, #87D0BA 36.3401643762683%, #53BBC2 47.2242548134565%, #2D9EC2 58.1083452506446%, #2376B4 68.9924356878328%, #254EA0 79.876526125021%, #1E2E84 90.7606165622092%, #081D58 &#34;],&#34;labels&#34;:[&#34;30%&#34;,&#34;35%&#34;,&#34;40%&#34;,&#34;45%&#34;,&#34;50%&#34;,&#34;55%&#34;,&#34;60%&#34;,&#34;65%&#34;,&#34;70%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent White&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.0368789306470377,&#34;p_n&#34;:0.907606165622092},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, demographics2_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent White&amp;quot;, round(demographics2_mission$percent_white,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = demographics2_mission$percent_white
)

whitepop2&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = demographics2_mission, 
              fillColor = ~pal(percent_white), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = demographics2_mission$percent_white, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent White&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
whitepop2&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-da54f952096d6ee79aa8&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-da54f952096d6ee79aa8&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#A3DAB8&#34;,&#34;#225EA8&#34;,&#34;#A4DAB8&#34;,&#34;#32A2C2&#34;,&#34;#081D58&#34;,&#34;#EEF8B2&#34;,&#34;#2553A3&#34;,&#34;#26489E&#34;,&#34;#E8F6B1&#34;,&#34;#E9F6B1&#34;,&#34;#192A7A&#34;,&#34;#192A7B&#34;,&#34;#E0F3B2&#34;,&#34;#C2E7B5&#34;,&#34;#F7FCC8&#34;,&#34;#FFFFD9&#34;,&#34;#D4EEB3&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent White41.18&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent White62.36&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent White41.17&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent White53.38&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent White74.62&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent White31.66&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent White64.06&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent White65.59&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent White32.75&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent White32.55&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent White71.06&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent White71.03&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent White34.01&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent White38.48&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent White28.41&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent White25.77&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent White35.88&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #F3FABD 8.65657472871711%, #DAF0B3 18.8924670494713%, #B1E0B6 29.1283593702255%, #78CABC 39.3642516909797%, #44B7C4 49.6001440117339%, #2899C1 59.836036332488%, #2372B1 70.0719286532422%, #264CA0 80.3078209739964%, #1E2E85 90.5437132947506%, #081D58 &#34;],&#34;labels&#34;:[&#34;30%&#34;,&#34;35%&#34;,&#34;40%&#34;,&#34;45%&#34;,&#34;50%&#34;,&#34;55%&#34;,&#34;60%&#34;,&#34;65%&#34;,&#34;70%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent White&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.0865657472871711,&#34;p_n&#34;:0.905437132947506},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;/div&gt;
&lt;div id=&#34;future-directions&#34; class=&#34;section level4&#34;&gt;
&lt;h4&gt;Future directions&lt;/h4&gt;
&lt;p&gt;The next step is of course actually mapping population changes, which is very easy with our data frames in hand.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;demographics2_mission$HLpercentchange &amp;lt;- 100*((demographics2_mission$Hispanic_or_Latino_by_Race_Hispanic_or_Latino - demographics_mission$Hispanic_or_Latino_by_Race_Hispanic_or_Latino)/demographics_mission$Hispanic_or_Latino_by_Race_Hispanic_or_Latino)

demographics2_mission$Wpercentchange &amp;lt;- 100*((demographics2_mission$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone - demographics_mission$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone)/demographics_mission$Hispanic_or_Latino_by_Race_Not_Hispanic_or_Latino_White_alone)

popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, demographics2_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent Hispanic Latino Change 2013-2015&amp;quot;, round(demographics2_mission$HLpercentchange,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = demographics2_mission$HLpercentchange
)

HLD&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = demographics2_mission, 
              fillColor = ~pal(HLpercentchange), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = demographics2_mission$HLpercentchange, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent Hispanic Latino&amp;lt;br&amp;gt;Change 2013-2015&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
HLD&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-a8b9d5d138ed3ae3a10e&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-a8b9d5d138ed3ae3a10e&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#3AADC3&#34;,&#34;#081D58&#34;,&#34;#33A4C2&#34;,&#34;#D2EDB3&#34;,&#34;#CAEAB4&#34;,&#34;#BCE4B5&#34;,&#34;#8DD2BA&#34;,&#34;#51BBC2&#34;,&#34;#51BAC2&#34;,&#34;#D4EEB3&#34;,&#34;#FFFFD9&#34;,&#34;#41B6C4&#34;,&#34;#2395C0&#34;,&#34;#65C2C0&#34;,&#34;#93D4B9&#34;,&#34;#2377B4&#34;,&#34;#95D5B9&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent Hispanic Latino Change 2013-201516.86&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent Hispanic Latino Change 2013-201565.85&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent Hispanic Latino Change 2013-201519.95&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent Hispanic Latino Change 2013-2015-16.34&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent Hispanic Latino Change 2013-2015-13.46&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent Hispanic Latino Change 2013-2015-10.3&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent Hispanic Latino Change 2013-2015-1.74&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent Hispanic Latino Change 2013-201510.93&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent Hispanic Latino Change 2013-201511.05&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent Hispanic Latino Change 2013-2015-16.92&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent Hispanic Latino Change 2013-2015-38.64&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent Hispanic Latino Change 2013-201513.76&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent Hispanic Latino Change 2013-201525.3&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent Hispanic Latino Change 2013-20156.98&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent Hispanic Latino Change 2013-2015-2.8&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent Hispanic Latino Change 2013-201533.31&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent Hispanic Latino Change 2013-2015-3.1&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #DDF2B2 17.8419603267211%, #82CEBB 36.9829638273046%, #33A4C2 56.123967327888%, #225DA8 75.2649708284714%, #152772 94.4059743290549%, #081D58 &#34;],&#34;labels&#34;:[&#34;-20%&#34;,&#34;0%&#34;,&#34;20%&#34;,&#34;40%&#34;,&#34;60%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent Hispanic Latino&lt;br&gt;Change 2013-2015&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.178419603267211,&#34;p_n&#34;:0.944059743290549},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;popup &amp;lt;- paste0(&amp;quot;GEOID: &amp;quot;, demographics2_mission$GEOID, &amp;quot;&amp;lt;br&amp;gt;&amp;quot;, &amp;quot;Percent White Change 2013-2015&amp;quot;, round(demographics2_mission$Wpercentchange,2))
pal &amp;lt;- colorNumeric(
  palette = &amp;quot;YlGnBu&amp;quot;,
  domain = demographics2_mission$Wpercentchange
)

whitepopD&amp;lt;-leaflet() %&amp;gt;%
  addProviderTiles(&amp;quot;CartoDB.Positron&amp;quot;) %&amp;gt;%
  addPolygons(data = demographics2_mission, 
              fillColor = ~pal(Wpercentchange), 
              color = &amp;quot;#b2aeae&amp;quot;, # you need to use hex colors
              fillOpacity = 0.7, 
              weight = 1, 
              smoothFactor = 0.2,
              popup = popup) %&amp;gt;%
  addLegend(pal = pal, 
            values = demographics2_mission$Wpercentchange, 
            position = &amp;quot;bottomright&amp;quot;, 
            title = &amp;quot;Percent White Change&amp;lt;br&amp;gt;2013-2015&amp;quot;,
            labFormat = labelFormat(suffix = &amp;quot;%&amp;quot;)) 
whitepopD&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;htmlwidget-20668383783f7075488b&#34; style=&#34;width:672px;height:480px;&#34; class=&#34;leaflet html-widget&#34;&gt;&lt;/div&gt;
&lt;script type=&#34;application/json&#34; data-for=&#34;htmlwidget-20668383783f7075488b&#34;&gt;{&#34;x&#34;:{&#34;options&#34;:{&#34;crs&#34;:{&#34;crsClass&#34;:&#34;L.CRS.EPSG3857&#34;,&#34;code&#34;:null,&#34;proj4def&#34;:null,&#34;projectedBounds&#34;:null,&#34;options&#34;:{}}},&#34;calls&#34;:[{&#34;method&#34;:&#34;addProviderTiles&#34;,&#34;args&#34;:[&#34;CartoDB.Positron&#34;,null,null,{&#34;errorTileUrl&#34;:&#34;&#34;,&#34;noWrap&#34;:false,&#34;zIndex&#34;:null,&#34;unloadInvisibleTiles&#34;:null,&#34;updateWhenIdle&#34;:null,&#34;detectRetina&#34;:false,&#34;reuseTiles&#34;:false}]},{&#34;method&#34;:&#34;addPolygons&#34;,&#34;args&#34;:[[[[{&#34;lng&#34;:[-122.418704,-122.415561,-122.41248,-122.410931,-122.409736,-122.408007,-122.40499,-122.404497,-122.40509,-122.408424,-122.413003,-122.413588,-122.417332,-122.417487,-122.417802,-122.417794,-122.419219,-122.418704],&#34;lat&#34;:[37.775645,37.773101,37.77063,37.769411,37.770349,37.769244,37.769715,37.764664,37.764628,37.764427,37.764091,37.763823,37.763572,37.765183,37.768405,37.770435,37.775316,37.775645]}]],[[{&#34;lng&#34;:[-122.425384,-122.420964,-122.420812,-122.420665,-122.420507,-122.420282,-122.421393,-122.42485,-122.424777,-122.424934,-122.425097,-122.42524,-122.425384],&#34;lat&#34;:[37.755035,37.755295,37.753703,37.752104,37.750506,37.748161,37.748124,37.747825,37.748644,37.750237,37.751845,37.753436,37.755035]}]],[[{&#34;lng&#34;:[-122.417332,-122.413588,-122.413003,-122.408424,-122.408064,-122.407754,-122.412348,-122.41672,-122.416872,-122.417026,-122.417179,-122.417332],&#34;lat&#34;:[37.763572,37.763823,37.764091,37.764427,37.760601,37.757705,37.75743,37.757167,37.758764,37.760367,37.761968,37.763572]}]],[[{&#34;lng&#34;:[-122.426948,-122.426402,-122.424929,-122.423641,-122.42262,-122.422365,-122.422308,-122.422044,-122.421732,-122.426137,-122.426293,-122.426462,-122.426713,-122.426948],&#34;lat&#34;:[37.769175,37.769596,37.770778,37.771828,37.772503,37.769868,37.769278,37.76654,37.7633,37.763036,37.764651,37.766272,37.768982,37.769175]}]],[[{&#34;lng&#34;:[-122.435188,-122.432124,-122.430727,-122.426137,-122.425836,-122.425685,-122.425532,-122.430115,-122.432331,-122.434546,-122.434698,-122.434854,-122.435001,-122.435188],&#34;lat&#34;:[37.762671,37.762668,37.762753,37.763036,37.759835,37.758232,37.756636,37.75636,37.756227,37.756093,37.757687,37.759289,37.760889,37.762671]}]],[[{&#34;lng&#34;:[-122.41672,-122.412348,-122.407754,-122.407602,-122.407449,-122.409253,-122.412047,-122.416413,-122.416567,-122.41672],&#34;lat&#34;:[37.757167,37.75743,37.757705,37.756109,37.754506,37.754398,37.75423,37.753968,37.755568,37.757167]}]],[[{&#34;lng&#34;:[-122.435193,-122.430824,-122.428949,-122.428635,-122.426948,-122.426713,-122.426462,-122.426293,-122.426137,-122.430727,-122.432124,-122.435188,-122.435193],&#34;lat&#34;:[37.762727,37.766014,37.767504,37.767846,37.769175,37.768982,37.766272,37.764651,37.763036,37.762753,37.762668,37.762671,37.762727]}]],[[{&#34;lng&#34;:[-122.426137,-122.421732,-122.421578,-122.421425,-122.421118,-122.420964,-122.425384,-122.425532,-122.425685,-122.425836,-122.426137],&#34;lat&#34;:[37.763036,37.7633,37.761701,37.760101,37.756902,37.755295,37.755035,37.756636,37.758232,37.759835,37.763036]}]],[[{&#34;lng&#34;:[-122.409253,-122.407449,-122.406453,-122.403127,-122.403007,-122.403784,-122.405239,-122.406788,-122.40871,-122.408978,-122.409253],&#34;lat&#34;:[37.754398,37.754506,37.754277,37.754478,37.752446,37.749433,37.749125,37.748805,37.748397,37.751201,37.754398]}]],[[{&#34;lng&#34;:[-122.421732,-122.417332,-122.417179,-122.417026,-122.416872,-122.41672,-122.416567,-122.418748,-122.420964,-122.421118,-122.421425,-122.421578,-122.421732],&#34;lat&#34;:[37.7633,37.763572,37.761968,37.760367,37.758764,37.757167,37.755568,37.755437,37.755295,37.756902,37.760101,37.761701,37.7633]}]],[[{&#34;lng&#34;:[-122.434546,-122.432331,-122.430115,-122.425532,-122.425384,-122.42524,-122.425097,-122.429659,-122.43409,-122.43424,-122.434317,-122.434546],&#34;lat&#34;:[37.756093,37.756227,37.75636,37.756636,37.755035,37.753436,37.751845,37.751571,37.751304,37.752891,37.75369,37.756093]}]],[[{&#34;lng&#34;:[-122.43409,-122.429659,-122.425097,-122.424934,-122.424777,-122.42485,-122.427045,-122.431477,-122.433697,-122.433769,-122.43409],&#34;lat&#34;:[37.751304,37.751571,37.751845,37.750237,37.748644,37.747825,37.747696,37.747434,37.747297,37.748097,37.751304]}]],[[{&#34;lng&#34;:[-122.408424,-122.40509,-122.405105,-122.40648,-122.406389,-122.403689,-122.403396,-122.403127,-122.406453,-122.407449,-122.407602,-122.407754,-122.408064,-122.408424],&#34;lat&#34;:[37.764427,37.764628,37.763852,37.760731,37.759804,37.757015,37.756471,37.754478,37.754277,37.754506,37.756109,37.757705,37.760601,37.764427]}]],[[{&#34;lng&#34;:[-122.412047,-122.409253,-122.408978,-122.40871,-122.410551,-122.411488,-122.411746,-122.4119,-122.412047],&#34;lat&#34;:[37.75423,37.754398,37.751201,37.748397,37.748367,37.748344,37.751034,37.752629,37.75423]}]],[[{&#34;lng&#34;:[-122.42262,-122.421989,-122.419334,-122.419219,-122.417794,-122.417802,-122.417487,-122.417332,-122.421732,-122.422044,-122.422308,-122.422365,-122.42262],&#34;lat&#34;:[37.772503,37.773074,37.77521,37.775316,37.770435,37.768405,37.765183,37.763572,37.7633,37.76654,37.769278,37.769868,37.772503]}]],[[{&#34;lng&#34;:[-122.416413,-122.412047,-122.4119,-122.411746,-122.411488,-122.413673,-122.41587,-122.416108,-122.416413],&#34;lat&#34;:[37.753968,37.75423,37.752629,37.751034,37.748344,37.748297,37.748267,37.750771,37.753968]}]],[[{&#34;lng&#34;:[-122.420964,-122.418748,-122.416567,-122.416413,-122.416108,-122.41587,-122.418206,-122.420282,-122.420507,-122.420665,-122.420812,-122.420964],&#34;lat&#34;:[37.755295,37.755437,37.755568,37.753968,37.750771,37.748267,37.748203,37.748161,37.750506,37.752104,37.753703,37.755295]}]]],null,null,{&#34;lineCap&#34;:null,&#34;lineJoin&#34;:null,&#34;clickable&#34;:true,&#34;pointerEvents&#34;:null,&#34;className&#34;:&#34;&#34;,&#34;stroke&#34;:true,&#34;color&#34;:&#34;#b2aeae&#34;,&#34;weight&#34;:1,&#34;opacity&#34;:0.5,&#34;fill&#34;:true,&#34;fillColor&#34;:[&#34;#3BAEC3&#34;,&#34;#ABDDB7&#34;,&#34;#4FBAC2&#34;,&#34;#2457A5&#34;,&#34;#2367AC&#34;,&#34;#39ABC3&#34;,&#34;#73C7BD&#34;,&#34;#D8F0B3&#34;,&#34;#FFFFD9&#34;,&#34;#2458A5&#34;,&#34;#225EA8&#34;,&#34;#192B7B&#34;,&#34;#60C0C0&#34;,&#34;#263F99&#34;,&#34;#1D91C0&#34;,&#34;#081D58&#34;,&#34;#236DAF&#34;],&#34;fillOpacity&#34;:0.7,&#34;dashArray&#34;:null,&#34;smoothFactor&#34;:0.2,&#34;noClip&#34;:false},[&#34;GEOID: 06075017700&lt;br&gt;Percent White Change 2013-2015-2.55&#34;,&#34;GEOID: 06075021000&lt;br&gt;Percent White Change 2013-2015-10.02&#34;,&#34;GEOID: 06075022801&lt;br&gt;Percent White Change 2013-2015-4.17&#34;,&#34;GEOID: 06075020200&lt;br&gt;Percent White Change 2013-20155.5&#34;,&#34;GEOID: 06075020600&lt;br&gt;Percent White Change 2013-20154.1&#34;,&#34;GEOID: 06075022803&lt;br&gt;Percent White Change 2013-2015-2.24&#34;,&#34;GEOID: 06075020300&lt;br&gt;Percent White Change 2013-2015-6.59&#34;,&#34;GEOID: 06075020700&lt;br&gt;Percent White Change 2013-2015-13.59&#34;,&#34;GEOID: 06075022903&lt;br&gt;Percent White Change 2013-2015-20.02&#34;,&#34;GEOID: 06075020800&lt;br&gt;Percent White Change 2013-20155.45&#34;,&#34;GEOID: 06075021100&lt;br&gt;Percent White Change 2013-20154.89&#34;,&#34;GEOID: 06075021400&lt;br&gt;Percent White Change 2013-201510.66&#34;,&#34;GEOID: 06075022802&lt;br&gt;Percent White Change 2013-2015-5.21&#34;,&#34;GEOID: 06075022902&lt;br&gt;Percent White Change 2013-20157.93&#34;,&#34;GEOID: 06075020100&lt;br&gt;Percent White Change 2013-20150.7&#34;,&#34;GEOID: 06075022901&lt;br&gt;Percent White Change 2013-201513.14&#34;,&#34;GEOID: 06075020900&lt;br&gt;Percent White Change 2013-20153.63&#34;],null,null,null,null]},{&#34;method&#34;:&#34;addLegend&#34;,&#34;args&#34;:[{&#34;colors&#34;:[&#34;#FFFFD9 , #FFFFD9 0.0473533814124354%, #E5F5B2 15.1294053612736%, #ABDDB7 30.2114573411348%, #5DBFC1 45.2935093209961%, #2697C1 60.3755613008573%, #225CA7 75.4576132807185%, #1E2E85 90.5396652605797%, #081D58 &#34;],&#34;labels&#34;:[&#34;-20%&#34;,&#34;-15%&#34;,&#34;-10%&#34;,&#34;-5%&#34;,&#34;0%&#34;,&#34;5%&#34;,&#34;10%&#34;],&#34;na_color&#34;:null,&#34;na_label&#34;:&#34;NA&#34;,&#34;opacity&#34;:0.5,&#34;position&#34;:&#34;bottomright&#34;,&#34;type&#34;:&#34;numeric&#34;,&#34;title&#34;:&#34;Percent White Change&lt;br&gt;2013-2015&#34;,&#34;extra&#34;:{&#34;p_1&#34;:0.000473533814124354,&#34;p_n&#34;:0.905396652605797},&#34;layerId&#34;:null,&#34;className&#34;:&#34;info legend&#34;}]}],&#34;limits&#34;:{&#34;lat&#34;:[37.747297,37.775645],&#34;lng&#34;:[-122.435193,-122.403007]}},&#34;evals&#34;:[],&#34;jsHooks&#34;:[]}&lt;/script&gt;
&lt;/div&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Intro to R and R Studio</title>
      <link>/portfolio/2017-10-07-intro-to-r/</link>
      <pubDate>Wed, 07 Oct 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-10-07-intro-to-r/</guid>
      <description>&lt;p&gt;The term ‘R’ refers to both a programming language and the environment in which you execute commands and manipulate data. R is a command-line interface, which means you tell R what you want it to do via lines of text. Most commands you will be using will come from different packages people have developed for R – this means that when you issue a command, you are often actually invoking a series of commands someone has encoded in the command you are using. There are other more basic commands in R you can use without downloading and loading packages, so we will start there to get a sense of how the command-line works.&lt;/p&gt;
&lt;p&gt;R is a language – it has syntactic rules, preferred word order and both required and optional arguments in each command you issue. Let’s start with setting our current working directory. A directory is the ‘location’ of where your data is currently being drawn from and/or saved to. Check the location of your current working directory with the following command:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;getwd()&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;/Users/katelyons/katelyons/content/portfolio&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If this is where your data is, or where you’d like to save things, great! If not, you can change your working directory. The argument you will put for this command is the path of the directory you want to use. You can see what most of your path will look like from the results of getwd() – just adjust it to the specific folder you want to work in. The one I’ve put as an example is for my computer – yours (especially if you are not using a Mac!) will look different:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# setwd(&amp;quot;/Users/katelyons/Documents/Workshop&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is a fundamental lesson about R – you have to be careful to be specific and when you get an error, make sure to think about what you are telling R to do in each command. Often you’ll find an error is just leaving a parentheses out or forgetting that something needs to be in quotes! (For example, when writing this I forgot to add the “” around my path and got an error!)&lt;/p&gt;
&lt;p&gt;A great resource to look at command syntax and arguments is R help. You can either use the ‘Help’ tab in R Studio and type in the command in the search field, or use the command-line:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# ?getwd()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Another important thing to think about and remember about R are the different data types and objects that you can work with in this environment. ‘Data type’ refers to, in a loose way, the format or form of your data. What type your data is represented by influences what kind of commands you can use to manipulate the data. You’ll mainly encounter three types of types in our work on text and social media mining: character, factor and logical. &lt;a href=&#34;https://www.stat.berkeley.edu/~nolan/stat133/Fall05/lectures/DataTypes4.pdf&#34;&gt;This collection of slides&lt;/a&gt; provides a nice summary of definitions and examples that I will reproduce here. They label these types in terms of ‘vectors’. Vectors are “sequence[s] of data elements of the same basic type” (see &lt;a href=&#34;http://www.r-tutor.com/r-introduction/vector&#34;&gt;here&lt;/a&gt;) – so just a way of describing more than one instance of these representations of your data.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Character or string vector: “each element in the vector is a string of one or more characters”&lt;/li&gt;
&lt;/ul&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s make a character vector
char_vector &amp;lt;- c(&amp;quot;this&amp;quot;, &amp;quot;is&amp;quot;, &amp;quot;a&amp;quot;, &amp;quot;character&amp;quot;, &amp;quot;vector&amp;quot;)
# This command says concatenate (the little c) the following series of strings of characters that are encapsulated in the parentheses and put that concatenated result into (the &amp;#39;&amp;lt;-&amp;#39;) a value that I will call &amp;#39;char_vector&amp;#39;. 

# You can use R to evaluate different aspects of character vectors like number of strings, numbers of characters in each string or whether or not a string is present in your vector

# How many strings does my vector have?
length(char_vector)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] 5&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# How many characters are in each string?
nchar(char_vector)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] 4 2 1 9 6&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Which string matches &amp;#39;a&amp;#39;?
char_vector == &amp;quot;a&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] FALSE FALSE  TRUE FALSE FALSE&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Note use of double equals signs. We do this because &amp;#39;=&amp;#39; is another way to perform &amp;#39;&amp;lt;-&amp;#39;. So we had run char_vector = &amp;quot;a&amp;quot; the vector would have changed to the character &amp;quot;a&amp;quot;. This wouldn&amp;#39;t be the end of the world, because we could go back and re-create the original character vector again. R does not have an &amp;#39;un-do&amp;#39; however, so be careful about accidentally overwriting something you do not want to overwrite. A good practice to avoid this is if you are trying a command out, just create a new output value. We&amp;#39;ll see more examples of this practice as we perform more complex commands.
char_vector2 &amp;lt;- c(&amp;quot;this&amp;quot;, &amp;quot;is&amp;quot;, &amp;quot;a&amp;quot;, &amp;quot;test&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;Logical: “binary, two values represented by TRUE and FALSE”&lt;/li&gt;
&lt;/ul&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We&amp;#39;ve already been introduced to this value in the previous example. We can store the output of our char_value == &amp;#39;a&amp;#39; command in a new value, which will be a vector of logical types!
logical_vector &amp;lt;- char_vector == &amp;quot;a&amp;quot;

#Check it out
logical_vector&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] FALSE FALSE  TRUE FALSE FALSE&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;Factor: “set of numeric codes with character-valued levels”&lt;/li&gt;
&lt;/ul&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# I&amp;#39;ve adapted this example from the statistics slides. Here we have a subset of sharks we are studying, consisting of four great white sharks and two mako sharks. 
sharks = factor(c(1,0,1,0,0,0), levels = c(0, 1), labels = c(&amp;quot;great_white&amp;quot;, &amp;quot;mako&amp;quot;))
# Note how I&amp;#39;ve used &amp;#39;=&amp;#39; instead of &amp;#39;&amp;lt;-&amp;#39;. It really doesn&amp;#39;t matter which one you use. The same holds for &amp;#39; vs. &amp;quot;. &amp;quot; can be easier because R Studio will automatically provide the accompanying quotation mark (helping you not forgetting it and avoiding a syntax error). 
# This command says concatenate a factor of two occurances of 1 and four occurances of 0, the two different levels of this factor are 0 and 1 and these values will be named respectively &amp;#39;great_white&amp;#39; and &amp;#39;mako&amp;#39;. Note that the label has to be one string -- so a two word phrase like &amp;#39;great white&amp;#39; has to be &amp;#39;great_white&amp;#39;.&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can check what kind of vector you are working with with the class command.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;class(char_vector)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;character&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;class(logical_vector)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;logical&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;class(sharks)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;factor&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we understand what a vector is, let’s more to more complicated representations of data. From stats slides:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Vector: “a collection of ordered homogeneous elements”&lt;/li&gt;
&lt;li&gt;“We can think of matrices, arrays, lists and data frames as deviations from a vector”&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;We won’t go into matrices, arrays or lists because we won’t be working with any of those things, but we WILL be working with data frames:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Data Frame: “a list with possible heterogeneous vector elements of the same length. The elements of a data frame can be numeric vectors, factor vectors, and logical vectors, but &lt;em&gt;they must all be of the same length&lt;/em&gt;” (emphasis added)&lt;/li&gt;
&lt;/ul&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;shark_id = c(1, 7, 4, 2) 
species = c(&amp;quot;great_white&amp;quot;, &amp;quot;whale&amp;quot;, &amp;quot;mako&amp;quot;, &amp;quot;hammerhead&amp;quot;) 
sex = c(&amp;quot;f&amp;quot;, &amp;quot;m&amp;quot;, &amp;quot;f&amp;quot;, &amp;quot;f&amp;quot;) 
shark_df = data.frame(shark_id, species, sex)
# You can also see if something is a data frame using class
class(shark_df)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;data.frame&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# For data frames we can either use the &amp;#39;View&amp;#39; command to open it in a different window (useful for larger data frames) or &amp;#39;head&amp;#39; to see the first few entries in the data frame (also useful for larger data frames). Because our data frame is so small, we can also just type in the name to see the entire thing
shark_df&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##   shark_id     species sex
## 1        1 great_white   f
## 2        7       whale   m
## 3        4        mako   f
## 4        2  hammerhead   f&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We can learn a lot about our data frame with some other commands:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# What are all the vectors in our data frame? We could also think of this as &amp;#39;what categories are in my data frame&amp;#39;? 
names(shark_df)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;shark_id&amp;quot; &amp;quot;species&amp;quot;  &amp;quot;sex&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# How many categories? (Helpful for large data frames)
length(shark_df)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] 3&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# What levels are present in a category?
levels(shark_df$species)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;great_white&amp;quot; &amp;quot;hammerhead&amp;quot;  &amp;quot;mako&amp;quot;        &amp;quot;whale&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# This last command introduces the &amp;#39;$&amp;#39; -- this just singles out a particular column or category. For some commands you need to be specific as to which column in the data frame you are referring to, as in this case. We can&amp;#39;t ask R to tell us the levels of the entire data frame because that wouldn&amp;#39;t make sense to R -- where would it look? How would it know that you meant &amp;#39;for each column tell me the levels&amp;#39; -- the &amp;#39;levels&amp;#39; command just tells R &amp;#39;tell me the different levels in this category&amp;#39; so you have to specify which category. &amp;lt;- Hopefully this is insightful into the fundamental &amp;#39;sense&amp;#39; of how to use R: be specific and try to think about what the command is telling R to do. This is really helpful when you run into errors, which you most certainly WILL do.

# We can get a more general sense of our data frame too
summary(shark_df)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##     shark_id           species  sex  
##  Min.   :1.00   great_white:1   f:3  
##  1st Qu.:1.75   hammerhead :1   m:1  
##  Median :3.00   mako       :1        
##  Mean   :3.50   whale      :1        
##  3rd Qu.:4.75                        
##  Max.   :7.00&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Interesting! Do you see how it treated ‘shark_id’ like numeric values? That isn’t what we meant! But of course R would think that because it makes the most sense to R – numbers are treated as numbers, the first assumption is not to treat these numbers as symbols for something else! Luckily, however, most of the time it is easy to straighten this out by telling R you want to treat a category ‘as’ something – in this case, ‘as.factor’:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Again, make sure you specify the column
# You also have to &amp;#39;save&amp;#39; this transformation by &amp;#39;putting&amp;#39; the result into the column of the data frame you are transforming. Otherwise, R just thinks &amp;#39;oh, you want to see what this would look like if this column was a factor&amp;#39;
shark_df$shark_id &amp;lt;- as.factor(shark_df$shark_id)

# If we made a mistake and want to turn it into a number again, just use
# as.numeric(shark_df$shark_id)

# Let&amp;#39;s try again
summary(shark_df)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##  shark_id        species  sex  
##  1:1      great_white:1   f:3  
##  2:1      hammerhead :1   m:1  
##  4:1      mako       :1        
##  7:1      whale      :1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can think of data frames like an Excel or CSV (comma separated value) file. In fact, when you load in an excel or CSV, it automatically becomes a data frame. Let’s try this with some actual data, a &lt;a href=&#34;https://www.dropbox.com/s/4a23bi4bg6zqo69/subtweets.csv?dl=0&#34;&gt;CSV file&lt;/a&gt; of 50 subsetted tweets from my dissertation data.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Make sure you are in the same directory as the location of the file you are trying to load in or R cannot find it
data2 &amp;lt;- read.csv(&amp;quot;subtweets.csv&amp;quot;, header=T)
# This is telling R, find a CSV file with this exact name and load it into my work space. It is also telling R that it is true that this file has a header with category names. If this was marked &amp;#39;F&amp;#39; R would treat the first row as observations.
# CSV is basically the same as Excel, except CSV doesn&amp;#39;t require a separate package in R to load it in. There are packages that can do this, but I prefer if I am working with something I created in Excel to just &amp;#39;Save As&amp;#39; the file as CSV from the Excel side. This is just a personal preference.
# Check what class
class(data2)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] &amp;quot;data.frame&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# What does this data frame look like?
# View(data2)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We will routinely be using R to load in and export out data, usually in CSV. Sometimes this can mess up the encoding of your data however (e.g. if you just downloaded tweets and haven’t tagged emojis yet, saving to CSV will lose the encoding) so if you are just saving something you are working on and want to come back later, you can save it as R data. This is a really helpful thing you can do, because it doesn’t matter what the object is (it can be a vector, list, data frame, etc.) and when you load it back it it’s just how you left it.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# When you want to save something you have been working on (this will automatically save it to your current working directory)
save(data2,file=paste(&amp;quot;sampledata.Rda&amp;quot;))

# When you want to load it in (make sure you&amp;#39;re in the same directory as where you saved this)
load(&amp;#39;sampledata.Rda&amp;#39;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now let’s look at this data and use some packages to do different things with it! This is a file of mined tweets. We have ‘document’, which is the number id of the tweet in the corpus; ‘text’ which is the content of the tweet; ‘favorited’ which is the number of times the tweet has been favorited; ‘created’, when the tweet was tweeted; ‘truncated’, ‘replytoSID’ which I have never needed so never researched what they are; ‘id’ which is Twitter’s in-house id # for the tweet and ‘replyToUID’, another thing I’ve never used; ‘statusSource’ which is all Instagram bc I just wanted to work with Instagram data; ‘screenName’ which is name of the poster; ‘retweetsCount’, how many times the tweet was re-tweeted; ‘isRetweet’, whether or not a tweet is a re-tweet of someone else and finally, ‘longitude’ and ‘latitude’, the coordinates of where the tweet has been geotagged by the user.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;names(data2)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;##  [1] &amp;quot;X&amp;quot;             &amp;quot;document&amp;quot;      &amp;quot;text&amp;quot;          &amp;quot;favorited&amp;quot;    
##  [5] &amp;quot;favoriteCount&amp;quot; &amp;quot;replyToSN&amp;quot;     &amp;quot;created&amp;quot;       &amp;quot;truncated&amp;quot;    
##  [9] &amp;quot;replyToSID&amp;quot;    &amp;quot;id&amp;quot;            &amp;quot;replyToUID&amp;quot;    &amp;quot;statusSource&amp;quot; 
## [13] &amp;quot;screenName&amp;quot;    &amp;quot;retweetCount&amp;quot;  &amp;quot;isRetweet&amp;quot;     &amp;quot;retweeted&amp;quot;    
## [17] &amp;quot;longitude&amp;quot;     &amp;quot;latitude&amp;quot;&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At least for this workshop, you’ll be working primarily with data frames (or spending time turning something into a data frame so can you can work with it). Data frames have a lot of useful features. For example, they are really easy to subset. For example, in my data set I had a number of spam posts that were generated by bots that weren’t informative to my research question. How do I get rid of those posts?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Get rid of super frequent spam posters
# Note I&amp;#39;m creating a new data frame, data3 so I don&amp;#39;t &amp;#39;lose&amp;#39; my original data frame just yet
data3 &amp;lt;- data2[! data2$screenName %in% c(&amp;quot;4AMSOUNDS&amp;quot;,
      &amp;quot;BruciusTattoo&amp;quot;,&amp;quot;LionsHeartSF&amp;quot;,&amp;quot;hermesalchemist&amp;quot;,&amp;quot;Mrsourmash&amp;quot;,&amp;quot;AaronTheEra&amp;quot;,&amp;quot;AmnesiaBar&amp;quot;,&amp;quot;audreymose2&amp;quot;,&amp;quot;audreymosez&amp;quot;,&amp;quot;Bernalcutlery&amp;quot;,&amp;quot;blncdbrkfst&amp;quot;,&amp;quot;BrunosSF&amp;quot;,&amp;quot;chiddythekidd&amp;quot;,&amp;quot;ChurchChills&amp;quot;,&amp;quot;deeXiepoo&amp;quot;,&amp;quot;fabricoutletsf&amp;quot;,&amp;quot;gever&amp;quot;,&amp;quot;miramirasf&amp;quot;,&amp;quot;papalote415&amp;quot;,&amp;quot;HappyHoundsMasg&amp;quot;,&amp;quot;faern_me&amp;quot;),]

# Let&amp;#39;s use another useful command called &amp;#39;nrow&amp;#39; to check the number of rows and see how many posts got eliminated
nrow(data3)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## [1] 44&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The above command is an introduction to the wonderful world of regular expressions (regex. A lot of the packages we’ll be dealing with eliminate the need to use regular expressions, but we will still need them occasionally nonetheless. The most frustrating aspects of regex are the things that make them so powerful and useful. Regex use specific sets of commands made up of specific common symbols to describe ‘sets of strings’ (&lt;a href=&#34;https://stat.ethz.ch/R-manual/R-devel/library/base/html/regex.html&#34;&gt;R Documentation&lt;/a&gt;). So, in this example above, we’ve used regex to say subset (the ‘[]’) everything in the data2$screenName column that is NOT (the ‘!’) present (the ‘%in%’) this list of values (‘c’ concatenate, then the list of names of posters you don’t want). (Technically ‘%in%’ isn’t a regex – it’s part of base R and used to ‘match’ values. There is a way to match using regex but %in% is much more intuitive). We could also use a modified version of this command to isolate those spam posters like so:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Just remove the &amp;#39;!&amp;#39;
test &amp;lt;- data2[ data2$screenName %in% c(&amp;quot;4AMSOUNDS&amp;quot;,
      &amp;quot;BruciusTattoo&amp;quot;,&amp;quot;LionsHeartSF&amp;quot;,&amp;quot;hermesalchemist&amp;quot;,&amp;quot;Mrsourmash&amp;quot;,&amp;quot;AaronTheEra&amp;quot;,&amp;quot;AmnesiaBar&amp;quot;,&amp;quot;audreymose2&amp;quot;,&amp;quot;audreymosez&amp;quot;,&amp;quot;Bernalcutlery&amp;quot;,&amp;quot;blncdbrkfst&amp;quot;,&amp;quot;BrunosSF&amp;quot;,&amp;quot;chiddythekidd&amp;quot;,&amp;quot;ChurchChills&amp;quot;,&amp;quot;deeXiepoo&amp;quot;,&amp;quot;fabricoutletsf&amp;quot;,&amp;quot;gever&amp;quot;,&amp;quot;miramirasf&amp;quot;,&amp;quot;papalote415&amp;quot;,&amp;quot;HappyHoundsMasg&amp;quot;,&amp;quot;faern_me&amp;quot;),]

# View(test)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;We’ll return to regex in the coming weeks, especially when we want to identify values that match a certain set of characteristics.&lt;/p&gt;
&lt;p&gt;Now that we have some experience with the command line, let’s download and load some packages and use them to look at our data. I also want to introduce both the ggplot2 package itself as a visualization tool and introduce the logic of that package (which can often confuse and frustrate people working with it for the first time!).&lt;/p&gt;
&lt;p&gt;When you want to use a package in R, you have to download the package AND load the package into your library. Once you download a package once, you won’t have to download it again (unless you upgrade R, sometimes you have to re-download to do that) but you will have to re-load it in your library whenever you’ve opened a new session. Let’s start with getting the ggplot2 package and the ggmap package.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# install.packages(&amp;quot;tidyverse&amp;quot;)
# This will not only give us ggplot2, but a whole other slew of packages that will be really useful down the line like dplyr, tidyr, purrr, etc. 
# install.packages(&amp;quot;ggmap&amp;quot;)
library(tidyverse)
library(ggmap)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now what we are going to do is use the ggmap package to download a map from Google Map’s API (Application Programming Interface – more about this in week 3) and plot the location of tweets using ggplot2. Let’s start with ggmap:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s start with ggmap&amp;#39;s &amp;#39;get_map&amp;#39; function, which allows you to specify a location (depending on what you are looking at it this could be more vauge like &amp;#39;San Francisco, California&amp;#39; or &amp;#39;California&amp;#39;)
# The &amp;#39;zoom&amp;#39; parameter is just how close you want to be. For me, I was looking at a neighborhood so this is pretty close, but you can play around with it to get the map you want. 
map &amp;lt;- get_map(location = &amp;#39;Valencia St. and 20th, San Francisco, California&amp;#39;, zoom = 15)

#Check out the map (it&amp;#39;ll pop up in your &amp;#39;Plots&amp;#39; tab in R Studio -- you can click &amp;#39;Zoom&amp;#39; if you want a closer look)
ggmap(map)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-17-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Great! We have our map. Now let’s plot stuff on it. The inherent idea behind ggplot2 is layering. You build a visualization (graph, map, etc.) layer by layer. So here we will tell ggplot2 that we want our map as a ground layer, with the coordinates of our tweets put on top of that. Luckily, ggmap maintains latitude and longitude so when we tell ggplot2 what our x and y axes are, they’ll match up with the ones on the map. As always, however, we will have to do some preliminary steps to make sure everything is in the right format.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;data3$longitude&amp;lt;-as.numeric(data3$longitude)
data3$latitude&amp;lt;-as.numeric(data3$latitude)

mapPoints &amp;lt;- ggmap(map) + geom_point(aes(x = longitude, y = latitude), 
                                     data=data3, alpha=0.5, size = 3)
mapPoints&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-18-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Great! What if we want to look at this from a bit further away? We’d have to change the map layer. So, we get another map.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;map2 &amp;lt;- get_map(location = &amp;#39;Valencia St. and 20th, San Francisco, California&amp;#39;, zoom = 13)
ggmap(map2)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-19-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Map it again!
mapPoints2 &amp;lt;- ggmap(map2) + geom_point(aes(x = longitude, y = latitude), 
                                     data=data3, alpha=0.5, size = 3)
mapPoints2&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-19-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;As we will see, we can encode lots of different information in maps other than just location, but this necessitates identifying other features we might want to showcase on our map. We don’t have any of those yet, but we can pretend screen name is one of those things.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Note the only thing that changes is &amp;#39;color&amp;#39;. This tells ggplot2 I want to distinguish each dot by differences in category &amp;#39;screenName&amp;#39;
mapPoints3 &amp;lt;- ggmap(map) + geom_point(aes(x = longitude, y = latitude, color = screenName), 
                                     data=data3, alpha=0.5, size = 3)
mapPoints3&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-20-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Gross! Too many names. Let’s first use the dplyr package to subset the data so it’s not TOO messy. Let’s pick a random 10 tweets.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# dplyr package is already loaded as part of the &amp;#39;tidyverse&amp;#39;
submap &amp;lt;- sample_n(data3, 10)

# Note the only thing that changes is data to &amp;#39;submap&amp;#39;
mapPoints4 &amp;lt;- ggmap(map) + geom_point(aes(x = longitude, y = latitude, color = screenName), 
                                     data=submap, alpha=0.5, size = 3)
mapPoints4&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-21-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;That’s kind of faint. Because I don’t have too much overlap, I’ll make the dots more opaque by playing around with the ‘alpha’ argument.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;mapPoints5 &amp;lt;- ggmap(map) + geom_point(aes(x = longitude, y = latitude, color = screenName), 
                                     data=submap, alpha=1, size = 3)
mapPoints5&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-07-intro-to-r_files/figure-html/unnamed-chunk-22-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Next week: loading in, cleaning and playing around with text files!&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Intro to Text Mining</title>
      <link>/portfolio/2017-10-19-intro-to-text-mining/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-10-19-intro-to-text-mining/</guid>
      <description>&lt;p&gt;There are many examples of doing analyses of texts in English, but not as many in other languages. To address this somewhat I’ll be looking at librettos of operas on stage at the Metropolitan Opera this season (2017-2018) as these have a lot of variation linguistically and also offer some unique and interesting processing challenges.&lt;/p&gt;
&lt;p&gt;Let’s start with Mozart’s &lt;a href=&#34;http://opera.stanford.edu/iu/libretti/cosi.htm&#34;&gt;‘Cosi Fan Tutte’&lt;/a&gt; (libretto by Da Ponte).&lt;/p&gt;
&lt;p&gt;To prepare this data, I copy-pasted it from the above site into Word and then saved it as a ‘.txt’ file. &lt;em&gt;IMPORTANT&lt;/em&gt;: Because there are ‘special characters’ due to it’s being in Italian, I made sure to save it in UTF-8 encoding so those would be maintained in R. As a best practice, I’d recommend you save most text files you’ll be importing into R in this manner.&lt;/p&gt;
&lt;p&gt;You can access the Cosi Fan Tutte data &lt;a href=&#34;https://www.dropbox.com/sh/27turt2c7lxusb9/AACMuoGGLWa7H2tvpcX2XD2ra?dl=0&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;

&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Make sure you have all the right packages
packs = c(&amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;, &amp;quot;tm&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,&amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;, &amp;quot;readr&amp;quot;)
# If you need to re-install (which you might have to if you got a new R Studio version) here is a shortcut:
# lapply(packs, install.packages, character.only=T)
lapply(packs, library, character.only=T)
# Set your directory
# setwd(&amp;quot;/Users/katelyons/Documents/Workshop&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# library(devtools)
# This might be a nice way to work with data like this (or transcript data) but due to issues with rJava I&amp;#39;m not going to explore it now. If rJava works for you, this might be a package to check out!
# install_github(&amp;quot;qdap&amp;quot;, &amp;quot;trinker&amp;quot;, depend)
# library(qdap)
# library(trinker)

# Each scene is saved as a separate text file so I have to load in each one and then combine them in R. We will use the readr package to read in each file
library(readr)
# Depending on where your file is you might need to include the entire path
# a1s1 &amp;lt;- readLines(&amp;quot;cosi/a1s1.txt&amp;quot;)
a1s1 &amp;lt;- readLines(&amp;quot;cosi/a1s1.txt&amp;quot;)
a1s2 &amp;lt;- readLines(&amp;quot;cosi/a1s2.txt&amp;quot;)
a1s2 &amp;lt;- readLines(&amp;quot;cosi/a1s2.txt&amp;quot;)
a1s3 &amp;lt;- readLines(&amp;quot;cosi/a1s3.txt&amp;quot;)
a1s4 &amp;lt;- readLines(&amp;quot;cosi/a1s4.txt&amp;quot;)
a1s5 &amp;lt;- readLines(&amp;quot;cosi/a1s5.txt&amp;quot;)
a1s6 &amp;lt;- readLines(&amp;quot;cosi/a1s6.txt&amp;quot;)
a1s7 &amp;lt;- readLines(&amp;quot;cosi/a1s7.txt&amp;quot;)
a1s8 &amp;lt;- readLines(&amp;quot;cosi/a1s8.txt&amp;quot;)
a1s9 &amp;lt;- readLines(&amp;quot;cosi/a1s9.txt&amp;quot;)
a1s10 &amp;lt;- readLines(&amp;quot;cosi/a1s10.txt&amp;quot;)
a1s11 &amp;lt;- readLines(&amp;quot;cosi/a1s11.txt&amp;quot;)
a1s12 &amp;lt;- readLines(&amp;quot;cosi/a1s12.txt&amp;quot;)
a1s13 &amp;lt;- readLines(&amp;quot;cosi/a1s13.txt&amp;quot;)
a1s14 &amp;lt;- readLines(&amp;quot;cosi/a1s14.txt&amp;quot;)
a1s15 &amp;lt;- readLines(&amp;quot;cosi/a1s15.txt&amp;quot;)
a1s16 &amp;lt;- readLines(&amp;quot;cosi/a1s16.txt&amp;quot;)
a2s1 &amp;lt;- readLines(&amp;quot;cosi/a2s1.txt&amp;quot;)
a2s2 &amp;lt;- readLines(&amp;quot;cosi/a2s2.txt&amp;quot;)
a2s3 &amp;lt;- readLines(&amp;quot;cosi/a2s3.txt&amp;quot;)
a2s4 &amp;lt;- readLines(&amp;quot;cosi/a2s4.txt&amp;quot;)
a2s5 &amp;lt;- readLines(&amp;quot;cosi/a2s5.txt&amp;quot;)
a2s6 &amp;lt;- readLines(&amp;quot;cosi/a2s6.txt&amp;quot;)
a2s7 &amp;lt;- readLines(&amp;quot;cosi/a2s7.txt&amp;quot;)
a2s8 &amp;lt;- readLines(&amp;quot;cosi/a2s8.txt&amp;quot;)
a2s9 &amp;lt;- readLines(&amp;quot;cosi/a2s9.txt&amp;quot;)
a2s10 &amp;lt;- readLines(&amp;quot;cosi/a2s10.txt&amp;quot;)
a2s11 &amp;lt;- readLines(&amp;quot;cosi/a2s11.txt&amp;quot;)
a2s12 &amp;lt;- readLines(&amp;quot;cosi/a2s12.txt&amp;quot;)
a2s13 &amp;lt;- readLines(&amp;quot;cosi/a2s13.txt&amp;quot;)
a2s14 &amp;lt;- readLines(&amp;quot;cosi/a2s14.txt&amp;quot;)
a2s15 &amp;lt;- readLines(&amp;quot;cosi/a2s15.txt&amp;quot;)
a2s16 &amp;lt;- readLines(&amp;quot;cosi/a2s16.txt&amp;quot;)
a2s17 &amp;lt;- readLines(&amp;quot;cosi/a2s17.txt&amp;quot;)
a2s18 &amp;lt;- readLines(&amp;quot;cosi/a2s18.txt&amp;quot;)

# It&amp;#39;s a long opera!
# Turn them all into  data frames so it&amp;#39;s easier to work with
a1s1 &amp;lt;- data_frame(text = a1s1)
a1s2 &amp;lt;- data_frame(text = a1s2)
a1s3 &amp;lt;- data_frame(text = a1s3)
a1s4 &amp;lt;- data_frame(text = a1s4)
a1s5 &amp;lt;- data_frame(text = a1s5)
a1s6 &amp;lt;- data_frame(text = a1s6)
a1s7 &amp;lt;- data_frame(text = a1s7)
a1s8 &amp;lt;- data_frame(text = a1s8)
a1s9 &amp;lt;- data_frame(text = a1s9)
a1s10 &amp;lt;- data_frame(text = a1s10)
a1s11 &amp;lt;- data_frame(text = a1s11)
a1s12 &amp;lt;- data_frame(text = a1s12)
a1s13 &amp;lt;- data_frame(text = a1s13)
a1s14 &amp;lt;- data_frame(text = a1s14)
a1s15 &amp;lt;- data_frame(text = a1s15)
a1s16 &amp;lt;- data_frame(text = a1s16)
a2s1 &amp;lt;- data_frame(text = a2s1)
a2s2 &amp;lt;- data_frame(text = a2s2)
a2s3 &amp;lt;- data_frame(text = a2s3)
a2s4 &amp;lt;- data_frame(text = a2s4)
a2s5 &amp;lt;- data_frame(text = a2s5)
a2s6 &amp;lt;- data_frame(text = a2s6)
a2s7 &amp;lt;- data_frame(text = a2s7)
a2s8 &amp;lt;- data_frame(text = a2s8)
a2s9 &amp;lt;- data_frame(text = a2s9)
a2s10 &amp;lt;- data_frame(text = a2s10)
a2s11 &amp;lt;- data_frame(text = a2s11)
a2s12 &amp;lt;- data_frame(text = a2s12)
a2s13 &amp;lt;- data_frame(text = a2s13)
a2s14 &amp;lt;- data_frame(text = a2s14)
a2s15 &amp;lt;- data_frame(text = a2s15)
a2s16 &amp;lt;- data_frame(text = a2s16)
a2s17 &amp;lt;- data_frame(text = a2s17)
a2s18 &amp;lt;- data_frame(text = a2s18)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Take a second to look at your data. If you see ‘weird’ or ‘strange’ characters that don’t look like Italian characters, you should go back and re-load things and specify the encoding like so:&lt;/p&gt;
&lt;p&gt;I haven’t run into this issue on Mac, but I’ve seen some instances in Windows in which the UTF-8 encoding saved in these text files doesn’t survive once loaded into R.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# a1s1 &amp;lt;- readLines(&amp;quot;cosi/a1s1.txt&amp;quot;, encoding = &amp;quot;UTF-8&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we want to combine these all together into a larger data frame.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Combine these data frames (help from https://www.r-bloggers.com/concatenating-a-list-of-data-frames/)

cosiDF &amp;lt;- rbind.fill(a1s1, a1s2, a1s3, a1s4, a1s5, a1s6, a1s7, a1s8, a1s9, a1s10, a1s11, a1s12, a1s13, a1s14, a1s15, a1s16, a2s1, a2s2, a2s3, a2s4, a2s5, a2s6, a2s7, a2s8, a2s9, a2s10, a2s11, a2s12, a2s13, a2s14, a2s15, a2s16, a2s17, a2s18)

# Now let&amp;#39;s clean the content of the data!

# Let&amp;#39;s get rid of stage directions (help from https://stackoverflow.com/questions/13529360/replace-text-within-parenthesis-in-r)

# This says &amp;#39;substitute anything inbetween parentheses and the parentheses themselves with nothing&amp;#39;
cosiDF$text &amp;lt;- gsub( &amp;quot; *\\(.*?\\) *&amp;quot;, &amp;quot;&amp;quot;, cosiDF$text)

# Let&amp;#39;s get rid of descriptions of pieces in the opera 
# Same pattern as above -- &amp;#39;replace this string with nothing&amp;#39;
cosiDF$text &amp;lt;- gsub(&amp;quot;Recitativo&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;No.&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;Aria&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;Duetto&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;Terzetto&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;Quartetto&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;Quintetto&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;Sestetto&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)

# Also get rid of numbers (help from here: http://www.endmemo.com/program/R/gsub.php)
cosiDF$text &amp;lt;- gsub(&amp;quot;\\d+&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
# Get rid of &amp;quot;d&amp;#39;&amp;quot; a shortening of &amp;quot;di&amp;quot; (easier to do it this way than as a stop word)
cosiDF$text &amp;lt;- gsub(&amp;quot;d&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
# Get rid of &amp;quot;un&amp;#39;&amp;quot;, a shortening of &amp;quot;uno/a&amp;quot; in front of words that start with a vowel
cosiDF$text &amp;lt;- gsub(&amp;quot;un&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
cosiDF$text &amp;lt;- gsub(&amp;quot;d&amp;#39;un&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
# Get rid of &amp;quot;dell&amp;#39;&amp;quot;, a shortening of &amp;quot;dello/a/gli/i&amp;quot;
cosiDF$text &amp;lt;- gsub(&amp;quot;dell&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)
# Get rid of &amp;quot;l&amp;#39;&amp;quot; a shortening of &amp;quot;la&amp;quot; or &amp;quot;il&amp;quot; in front of words that start with a vowel
cosiDF$text &amp;lt;- gsub(&amp;quot;l&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, cosiDF$text)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have a specific issue related to the format of our data. The data is formatted like a script, with the speaker name at the top and content underneath. This isn’t how R thinks though – it it’s in the same column, R things it’s all the same factor. This will be an issue for us, especially if we want to look at things like how a character’s speech changes over time. To change this one column into two columns, one column with the character name and the other with the line of the character, takes MANY steps! I got help from &lt;a href=&#34;https://stackoverflow.com/questions/16596515/aggregating-by-unique-identifier-and-concatenating-related-values-into-a-string&#34;&gt;here&lt;/a&gt; and &lt;a href=&#34;https://stackoverflow.com/questions/29376178/count-changes-to-contents-of-a-character-vector&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# First label whether a row is a line or character name

# Create a data frame with the character names that exist in the text
characters = c(&amp;quot;FERRANDO&amp;quot;, &amp;quot;DORABELLA&amp;quot;, &amp;quot;FIORDILIGI&amp;quot;, &amp;quot;GUILELMO&amp;quot;, &amp;quot;DON ALFONSO&amp;quot;, &amp;quot;DESPINA&amp;quot;, &amp;quot;SOLDATI&amp;quot;,  &amp;quot;CORO&amp;quot;, &amp;quot;CORO DI SERVI E SUONATORI&amp;quot;) 
chardf &amp;lt;- data.frame(characters)

# Now use regular expression to identify when the row is a character name (TRUE) or not (FALSE)
# FALSE then means that we are looking at a line row
chargrepl &amp;lt;- grepl(paste(chardf$characters, collapse = &amp;quot;|&amp;quot;), cosiDF$text)
# This says go through each row and if something in that row is the same as something in our characters data frame (the &amp;quot;|&amp;quot; is working as an &amp;#39;or&amp;#39;, telling R that it could be Dorabella or Fiordiligi or Don Alfonso, etc.) then put &amp;#39;TRUE&amp;#39;

# The output is a list so we have to turn it into a data frame to merge it back with our data
chargreplDF &amp;lt;-as.data.frame(chargrepl)

# To merge something, you have to have a common key. Create an id row for each data frame you want to combine and that will be your key! 
cosiDF$id &amp;lt;- 1:nrow(cosiDF)
chargreplDF$id &amp;lt;- 1:nrow(chargreplDF)

# Now merge!
text_df &amp;lt;- merge(cosiDF,chargreplDF,by=&amp;quot;id&amp;quot;)

# Now things get complicated. We are going to &amp;#39;group&amp;#39; things according to names. We have name rows represented by &amp;#39;TRUE&amp;#39; and lines represented by &amp;#39;FALSE&amp;#39;. So we can create a group by telling R to make a new id row that starts anew each time it sees &amp;#39;TRUE&amp;#39;.

# Make groups of lines associated with names (help from here https://stackoverflow.com/questions/29376178/count-changes-to-contents-of-a-character-vector)
test_df &amp;lt;- text_df %&amp;gt;% mutate( 
   try_1 = cumsum(ifelse(chargrepl == TRUE, 1, 0)) 
   )

# Now we want to copy our text column so we can eventually have two columns in which one has all of the lines and the other has the characters speaking those lines
# Create a duplicate column of the text
test_df$text2 &amp;lt;- test_df$text

# Let&amp;#39;s get rid of superfluous info for our character column (so, in this case, lines)
# Get rid of lines in one column
test_df$text &amp;lt;- ifelse(test_df$chargrepl == TRUE, test_df$text, &amp;quot;&amp;quot;)
# This command says look through the text column and if the corresponding chargrepl column (our logical vector) is TRUE leave it alone but if it is else (&amp;#39;ifelse&amp;#39;) replace with &amp;quot;&amp;quot; (nothing).  

# Do the same for the other text column, but opposite
test_df$text2 &amp;lt;- ifelse(test_df$chargrepl == FALSE, test_df$text2, &amp;quot;&amp;quot;)

# Now let&amp;#39;s aggregate things
result &amp;lt;- aggregate(text ~ try_1, data = test_df, paste, collapse = &amp;quot;&amp;quot;)
result2 &amp;lt;- aggregate(text2 ~ try_1, data = test_df, paste, collapse = &amp;quot;&amp;quot;)
# These commands are creating two data frames. The first one is just character names, the second is the lines associated with those characters. The collapse is also helping us out in that it&amp;#39;s putting each series of phrases in one cell. That&amp;#39;s because we are collapsing by the group id we created a few steps back (&amp;#39;try_1&amp;#39;). 

# Now we merge these data frames together by their shared column of group id
new_data &amp;lt;- merge(result, result2, by=&amp;quot;try_1&amp;quot;)

# Let&amp;#39;s get rid of that id!
new_data &amp;lt;- new_data[,c(&amp;quot;text&amp;quot;,&amp;quot;text2&amp;quot;)]&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Transform this into something we can work with -- tidy text! Tidy text is a format of one row per term -- makes things easier in processing, visualizing things
tribble &amp;lt;- new_data %&amp;gt;%
  unnest_tokens(word, text2)

# You can see what tidy text looks like:
# View(tribble)

# Note how each line is still there (see character names), but each word is a new row&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s get rid of words that aren’t that informative to a larger scale exploratory analysis (stop words). We are going to use the “stopwords(‘italian’)”&amp;quot; command from the tm package to access their list of Italian stop words, but because we are dealing with a libretto, there are some ‘non-standard’ uses of words which we have to add to this list. I’ve also found another source with more Italian stop words so I’ve added this too (available &lt;a href=&#34;https://github.com/AndreaCirilloAC/TweetIT/blob/master/lexicon/IT/stopwords.csv&#34;&gt;here&lt;/a&gt;; I’ve transformed this into txt form which you can access &lt;a href=&#34;https://www.dropbox.com/s/pwkozjyjl94yb7w/extraitstopwords.txt?dl=0&#34;&gt;here&lt;/a&gt;).&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# You might need to modify this to make sure the path is correct
extrastopwords &amp;lt;- readLines(&amp;quot;extraitstopwords.txt&amp;quot;)

# Put these all together
mystopwords &amp;lt;- c(stopwords(&amp;#39;italian&amp;#39;), extrastopwords, &amp;quot;de&amp;quot;, &amp;quot;no&amp;quot;, &amp;quot;son&amp;quot;, &amp;quot;far&amp;quot;, &amp;quot;fa&amp;quot;, &amp;quot;fe&amp;quot;, &amp;quot;fo&amp;quot;, &amp;quot;or&amp;quot;, &amp;quot;vo&amp;quot;, &amp;quot;po&amp;quot;, &amp;quot;partono&amp;quot;, &amp;quot;quel&amp;quot;, &amp;quot;ah&amp;quot;, &amp;quot;oh&amp;quot;, &amp;quot;me&amp;quot;, &amp;quot;cosa&amp;quot;, &amp;quot;c&amp;#39;è&amp;quot;, &amp;quot;ch&amp;#39;io&amp;quot;, &amp;quot;ei&amp;quot;, &amp;quot;te&amp;quot;, &amp;quot;qua&amp;quot;, &amp;quot;là&amp;quot;, &amp;quot;può&amp;quot;, &amp;quot;ognuno&amp;quot;, &amp;quot;sè&amp;quot;, &amp;quot;mai&amp;quot;, &amp;quot;don&amp;quot;, &amp;quot;lor&amp;quot;, &amp;quot;già&amp;quot;, &amp;quot;fate&amp;quot;, &amp;quot;sì&amp;quot;, &amp;quot;ogni&amp;quot;, &amp;quot;poi&amp;quot;, &amp;quot;han&amp;quot;, &amp;quot;so&amp;quot;, &amp;quot;sa&amp;quot;, &amp;quot;cose&amp;quot;, &amp;quot;dunque&amp;quot;, &amp;quot;ancora&amp;quot;, &amp;quot;cos&amp;#39;è&amp;quot;, &amp;quot;ella&amp;quot;, &amp;quot;tal&amp;quot;, &amp;quot;anzi&amp;quot;, &amp;quot;eh&amp;quot;, &amp;quot;fan&amp;quot;, &amp;quot;esser&amp;quot;, &amp;quot;essi&amp;quot;, &amp;quot;nè&amp;quot;, &amp;quot;siam&amp;quot;, &amp;quot;alfin&amp;quot;, &amp;quot;dì&amp;quot;, &amp;quot;fra&amp;quot;, &amp;quot;lì&amp;quot;, &amp;quot;v&amp;#39;è&amp;quot;, &amp;quot;d&amp;#39;un&amp;quot;, &amp;quot;ve&amp;quot;, &amp;quot;ce&amp;quot;, &amp;quot;ch&amp;#39;è&amp;quot;, &amp;quot;ch&amp;#39;ei&amp;quot;, &amp;quot;ciò&amp;quot;, &amp;quot;qui&amp;quot;, &amp;quot;qual&amp;quot;, &amp;quot;perchè&amp;quot;, &amp;quot;così&amp;quot;, &amp;quot;dentro&amp;quot;, &amp;quot;par&amp;quot;, &amp;quot;pur&amp;quot;, &amp;quot;dir&amp;quot;, &amp;quot;dee&amp;quot;, &amp;quot;va&amp;quot;, &amp;quot;ch’io&amp;quot;, &amp;quot;c’è&amp;quot;, &amp;quot;ecc&amp;quot;, &amp;quot;don&amp;quot;, &amp;quot;donna&amp;quot;, &amp;quot;ecc&amp;quot;, &amp;quot;d&amp;quot;, &amp;quot;ta&amp;quot;, &amp;quot;via&amp;quot;, &amp;quot;orsù&amp;quot;, &amp;quot;qualche&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are mostly shortenings – “no” for “non”; “son” for “sono”; “or” for “ora”; “quel” for “quello/a/i/e”, “c’è” for “ci è” (‘there is’); “ch’io” for “che io”; “po” for “po’” (‘a little’) etc. We also have “partono” which is a common stage direction (‘they leave’) and “ognuno da sè” (‘each to themselves’); “ah” and “oh” as exclaimations of surprise / dismay; “me” as a form of the indirect object “mi”; “cosa” being used kind of as ‘what’ (“cosa” means ‘thing’ in modern Italian, but here we see it being used more in terms of “what’s happening?”, “what’s this I feel?” etc. – not very informative of the opera as a whole, except maybe perhaps that people are confused or surprised a good deal). &lt;em&gt;Depending on your document you will most likely be adding stop words of your own.&lt;/em&gt; This also tends to be a cyclical process, in which you realize common ‘meaningless’ words (in terms of analyzing the text for broader themes, etc.) and go back and add them to your list.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# this is a character class so we have to turn it into a data frame to work with the anti_join function
itstopwords &amp;lt;- data_frame(text=mystopwords)

# For anti_join to work you&amp;#39;ll want to have the columns named the same so it&amp;#39;ll have directions of what to put together
# When you use tidytext&amp;#39;s English stop words (introduced by the data(stop_words) function) the words are automatically under a &amp;#39;word&amp;#39; column heading
names(itstopwords)[names(itstopwords)==&amp;quot;text&amp;quot;] &amp;lt;- &amp;quot;word&amp;quot;

# anti_join is a really useful function from the dplyr package. It basically says &amp;#39;in comparing two data frames; produce a new data frame that is one of our data frames with stuff that is present in the other data frame removed&amp;#39;. This is how we are getting &amp;#39;rid&amp;#39; of the stop words, and why tidy text is helpful here. Because each individual term is in a row, we can use things like anti_join (instead of a complicated regular expression) to eliminate stop words we&amp;#39;ve put in another data frame. 
tribble2 &amp;lt;- tribble %&amp;gt;%
  anti_join(itstopwords)

# Now with our new data frame with stop words removed, let&amp;#39;s look at the most frequent terms
cosifreq &amp;lt;- tribble2 %&amp;gt;%
  count(word, sort = TRUE) &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If this command doesn’t work, it might be because R doesn’t know which package to draw the ‘count’ function from – the one we want is from dplyr, not plyr. So we can tell R specifically which package we want to use.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# cosifreq &amp;lt;- tribble2 %&amp;gt;%
#   dplyr::count(word, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have a count of words, ordered from most frequent to least frequent. You can look at this list or perform other visualizations such as bigrams. Code from &lt;a href=&#34;http://tidytextmining.com/ngrams.html#tokenizing-by-n-gram&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;cosi_bigrams &amp;lt;- new_data %&amp;gt;%
  unnest_tokens(bigram, text2, token = &amp;quot;ngrams&amp;quot;, n = 2)
# To see these do the following command:
# cosi_bigrams

# To see them ranked from most common to least common:
# cosi_bigrams %&amp;gt;%
#   count(bigram, sort = TRUE)

# As Silge and Robinson point out (and as you might have noticed) we have to get rid of the stop words again.
bigrams_separated &amp;lt;- cosi_bigrams %&amp;gt;%
  separate(bigram, c(&amp;quot;word1&amp;quot;, &amp;quot;word2&amp;quot;), sep = &amp;quot; &amp;quot;)

bigrams_filtered &amp;lt;- bigrams_separated %&amp;gt;%
  filter(!word1 %in% itstopwords$word) %&amp;gt;%
  filter(!word2 %in% itstopwords$word)

# new bigram counts:
bigram_counts &amp;lt;- bigrams_filtered %&amp;gt;% 
  count(word1, word2, sort = TRUE)

bigram_counts&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 1,687 x 3
##     word1    word2     n
##     &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
##  1  bella     vita     5
##  2   vita  militar     5
##  3   caro     bene     4
##  4  mille    volte     4
##  5 cangia     loco     3
##  6  cento zecchini     3
##  7  doman     poco     3
##  8   loco     oggi     3
##  9   oggi    molto     3
## 10   poco      ora     3
## # ... with 1,677 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Interestingly for an opera (which might suggest certain phrases being repeated multiple times) we have few bigrams. We see some repeats for the chorus “Bella Vita Militar”, for example.&lt;/p&gt;
&lt;p&gt;What about trigrams?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;new_data %&amp;gt;%
  unnest_tokens(trigram, text2, token = &amp;quot;ngrams&amp;quot;, n = 3) %&amp;gt;%
  separate(trigram, c(&amp;quot;word1&amp;quot;, &amp;quot;word2&amp;quot;, &amp;quot;word3&amp;quot;), sep = &amp;quot; &amp;quot;) %&amp;gt;%
  filter(!word1 %in% itstopwords$word,
         !word2 %in% itstopwords$word,
         !word3 %in% itstopwords$word) %&amp;gt;%
  count(word1, word2, word3, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 747 x 4
##      word1   word2    word3     n
##      &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
##  1   bella    vita  militar     5
##  2  cangia    loco     oggi     3
##  3   doman    poco      ora     3
##  4    loco    oggi    molto     3
##  5  abbiam  giusta  ragione     2
##  6 alanima    vaga      sol     2
##  7   bombe   forza accresce     2
##  8    caro padrone   pagate     2
##  9   molto   doman     poco     2
## 10    oggi   molto    doman     2
## # ... with 737 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This helps sort out some of the overlap of the bigrams – we see “bella vita militar” not counted as two separate bigrams, for example.&lt;/p&gt;
&lt;p&gt;We can also look at bigrams of specific words. What if we want to compare how the two sisters tend to be talked about?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;bigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;dorabella&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 8 x 2
##         word2     n
##         &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1      capace     1
## 2   conducono     1
## 3     infedel     1
## 4     prender     1
## 5 scioccherie     1
## 6       senza     1
## 7     signora     1
## 8      stessa     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;bigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;fiordiligi&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 3 x 2
##          word2     n
##          &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1       ditemi     1
## 2 scempiaggini     1
## 3     tradirmi     1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are lots of other things we can do with bigrams, but let’s move on to other representations of the text. Let’s try a word cloud.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;wordcloud(cosifreq$word,cosifreq$n, min.freq=5, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-13-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Unsurprisingly the main characters’ names are very common, but we also have “pieta” (pity); “amici” (friends); “amor” (love); “stelle” (stars). How do these compare to another Mozart and Da Ponte collaboration, Don Giovanni?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Load in all of the different scenes
# If you are having encoding issues try the encoding argument
# dona1s1 &amp;lt;- readLines(&amp;quot;giovanni/a1s1.txt&amp;quot;, encoding = &amp;quot;UTF-8&amp;quot;)
dona1s1 &amp;lt;- readLines(&amp;quot;giovanni/a1s1.txt&amp;quot;)
dona1s2 &amp;lt;- readLines(&amp;quot;giovanni/a1s2.txt&amp;quot;)
dona1s2 &amp;lt;- readLines(&amp;quot;giovanni/a1s2.txt&amp;quot;)
dona1s3 &amp;lt;- readLines(&amp;quot;giovanni/a1s3.txt&amp;quot;)
dona1s4 &amp;lt;- readLines(&amp;quot;giovanni/a1s4.txt&amp;quot;)
dona1s5 &amp;lt;- readLines(&amp;quot;giovanni/a1s5.txt&amp;quot;)
dona1s6 &amp;lt;- readLines(&amp;quot;giovanni/a1s6.txt&amp;quot;)
dona1s7 &amp;lt;- readLines(&amp;quot;giovanni/a1s7.txt&amp;quot;)
dona1s8 &amp;lt;- readLines(&amp;quot;giovanni/a1s8.txt&amp;quot;)
dona1s9 &amp;lt;- readLines(&amp;quot;giovanni/a1s9.txt&amp;quot;)
dona1s10 &amp;lt;- readLines(&amp;quot;giovanni/a1s10.txt&amp;quot;)
dona1s11 &amp;lt;- readLines(&amp;quot;giovanni/a1s11.txt&amp;quot;)
dona1s12 &amp;lt;- readLines(&amp;quot;giovanni/a1s12.txt&amp;quot;)
dona1s13 &amp;lt;- readLines(&amp;quot;giovanni/a1s13.txt&amp;quot;)
dona1s14 &amp;lt;- readLines(&amp;quot;giovanni/a1s14.txt&amp;quot;)
dona1s15 &amp;lt;- readLines(&amp;quot;giovanni/a1s15.txt&amp;quot;)
dona1s16 &amp;lt;- readLines(&amp;quot;giovanni/a1s16.txt&amp;quot;)
dona1s17 &amp;lt;- readLines(&amp;quot;giovanni/a1s17.txt&amp;quot;)
dona1s18 &amp;lt;- readLines(&amp;quot;giovanni/a1s18.txt&amp;quot;)
dona1s19 &amp;lt;- readLines(&amp;quot;giovanni/a1s19.txt&amp;quot;)
dona1s20 &amp;lt;- readLines(&amp;quot;giovanni/a1s20.txt&amp;quot;)
dona2s1 &amp;lt;- readLines(&amp;quot;giovanni/a2s1.txt&amp;quot;)
dona2s2 &amp;lt;- readLines(&amp;quot;giovanni/a2s2.txt&amp;quot;)
dona2s3 &amp;lt;- readLines(&amp;quot;giovanni/a2s3.txt&amp;quot;)
dona2s4 &amp;lt;- readLines(&amp;quot;giovanni/a2s4.txt&amp;quot;)
dona2s5 &amp;lt;- readLines(&amp;quot;giovanni/a2s5.txt&amp;quot;)
dona2s6 &amp;lt;- readLines(&amp;quot;giovanni/a2s6.txt&amp;quot;)
dona2s7 &amp;lt;- readLines(&amp;quot;giovanni/a2s7.txt&amp;quot;)
dona2s8 &amp;lt;- readLines(&amp;quot;giovanni/a2s8.txt&amp;quot;)
dona2s9 &amp;lt;- readLines(&amp;quot;giovanni/a2s9.txt&amp;quot;)
dona2s10 &amp;lt;- readLines(&amp;quot;giovanni/a2s10.txt&amp;quot;)
dona2s11 &amp;lt;- readLines(&amp;quot;giovanni/a2s11.txt&amp;quot;)
dona2s12 &amp;lt;- readLines(&amp;quot;giovanni/a2s12.txt&amp;quot;)
dona2s13 &amp;lt;- readLines(&amp;quot;giovanni/a2s13.txt&amp;quot;)
dona2s14 &amp;lt;- readLines(&amp;quot;giovanni/a2s14.txt&amp;quot;)
dona2s15 &amp;lt;- readLines(&amp;quot;giovanni/a2s15.txt&amp;quot;)
dona2s16 &amp;lt;- readLines(&amp;quot;giovanni/a2s16.txt&amp;quot;)


# Turn them all into  data frames so it&amp;#39;s easier to work with
dona1s1 &amp;lt;- data_frame(text = dona1s1)
dona1s2 &amp;lt;- data_frame(text = dona1s2)
dona1s3 &amp;lt;- data_frame(text = dona1s3)
dona1s4 &amp;lt;- data_frame(text = dona1s4)
dona1s5 &amp;lt;- data_frame(text = dona1s5)
dona1s6 &amp;lt;- data_frame(text = dona1s6)
dona1s7 &amp;lt;- data_frame(text = dona1s7)
dona1s8 &amp;lt;- data_frame(text = dona1s8)
dona1s9 &amp;lt;- data_frame(text = dona1s9)
dona1s10 &amp;lt;- data_frame(text = dona1s10)
dona1s11 &amp;lt;- data_frame(text = dona1s11)
dona1s12 &amp;lt;- data_frame(text = dona1s12)
dona1s13 &amp;lt;- data_frame(text = dona1s13)
dona1s14 &amp;lt;- data_frame(text = dona1s14)
dona1s15 &amp;lt;- data_frame(text = dona1s15)
dona1s16 &amp;lt;- data_frame(text = dona1s16)
dona1s17 &amp;lt;- data_frame(text = dona1s17)
dona1s18 &amp;lt;- data_frame(text = dona1s18)
dona1s19 &amp;lt;- data_frame(text = dona1s19)
dona1s20 &amp;lt;- data_frame(text = dona1s20)
dona2s1 &amp;lt;- data_frame(text = dona2s1)
dona2s2 &amp;lt;- data_frame(text = dona2s2)
dona2s3 &amp;lt;- data_frame(text = dona2s3)
dona2s4 &amp;lt;- data_frame(text = dona2s4)
dona2s5 &amp;lt;- data_frame(text = dona2s5)
dona2s6 &amp;lt;- data_frame(text = dona2s6)
dona2s7 &amp;lt;- data_frame(text = dona2s7)
dona2s8 &amp;lt;- data_frame(text = dona2s8)
dona2s9 &amp;lt;- data_frame(text = dona2s9)
dona2s10 &amp;lt;- data_frame(text = dona2s10)
dona2s11 &amp;lt;- data_frame(text = dona2s11)
dona2s12 &amp;lt;- data_frame(text = dona2s12)
dona2s13 &amp;lt;- data_frame(text = dona2s13)
dona2s14 &amp;lt;- data_frame(text = dona2s14)
dona2s15 &amp;lt;- data_frame(text = dona2s15)
dona2s16 &amp;lt;- data_frame(text = dona2s16)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Combine these data frames (help from https://www.r-bloggers.com/concatenating-a-list-of-data-frames/)

giovanniDF &amp;lt;- rbind.fill(dona1s1, dona1s2, dona1s3, dona1s4, dona1s5, dona1s6, dona1s7, dona1s8, dona1s9, dona1s10, dona1s11, dona1s12, dona1s13, dona1s14, dona1s15, dona1s16, dona2s1, dona2s2, dona2s3, dona2s4, dona2s5, dona2s6, dona2s7, dona2s8, dona2s9, dona2s10, dona2s11, dona2s12, dona2s13, dona2s14, dona2s15, dona2s16)

# NOTE TO SELF TRY DOING THESE BEFORE EVERYTHING ELSE
giovanniDF$text &amp;lt;- gsub(&amp;quot;NO.&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;RECITATIVO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;ACCOMPAGNATO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;ARIA&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;DUETTO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;TERZETTO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;QUARTETTO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;QUINTETTO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;SESTETTO&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)

# Let&amp;#39;s get rid of stage directions (help from https://stackoverflow.com/questions/13529360/replace-text-within-parenthesis-in-r)
giovanniDF$text &amp;lt;- gsub( &amp;quot; *\\(.*?\\) *&amp;quot;, &amp;quot;&amp;quot;, giovanniDF$text)

# Also get rid of numbers (help from here: http://www.endmemo.com/program/R/gsub.php)
giovanniDF$text &amp;lt;- gsub(&amp;quot;\\d+&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
# Get rid of &amp;quot;d&amp;#39;&amp;quot; a shortening of &amp;quot;di&amp;quot; (easier to do it this way than as a stop word)
giovanniDF$text &amp;lt;- gsub(&amp;quot;d&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
# Get rid of &amp;quot;dell&amp;#39;&amp;quot;, a shortening of &amp;quot;dello/a/gli/i&amp;quot;
giovanniDF$text &amp;lt;- gsub(&amp;quot;dell&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
# Get rid of &amp;quot;l&amp;#39;&amp;quot; a shortening of &amp;quot;la&amp;quot; or &amp;quot;il&amp;quot; in front of words that start with a vowel
giovanniDF$text &amp;lt;- gsub(&amp;quot;l&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)

# There are some issues with this libretto, I have to make sure it can pick up the names as separate things
giovanniDF$text &amp;lt;- gsub(&amp;quot;DONNA&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)
giovanniDF$text &amp;lt;- gsub(&amp;quot;DON&amp;quot;,&amp;quot;&amp;quot;, giovanniDF$text)

# Let&amp;#39;s transform this into transcript form
# First label whether a row is a line or character name

# Create a data frame with the character names that exist in the text
doncharacters = c(&amp;quot;GIOVANNI&amp;quot;, &amp;quot;ANNA&amp;quot;, &amp;quot;LEPORELLO&amp;quot;, &amp;quot;COMMENDATORE&amp;quot;, &amp;quot;OTTAVIO&amp;quot;, &amp;quot;ELVIRA&amp;quot;, &amp;quot;ZERLINA&amp;quot;,  &amp;quot;MASETTO&amp;quot;, &amp;quot;COMMENDATORE&amp;quot;, &amp;quot;CONTADINE&amp;quot;, &amp;quot;CONTADINI&amp;quot;, &amp;quot;CORO&amp;quot;) 
donchardf &amp;lt;- data.frame(doncharacters)

# Now use regular expression to identify when the row is a character name (TRUE) or not (FALSE)
# FALSE then means that we are looking at a line row
donchargrepl &amp;lt;- grepl(paste(donchardf$doncharacters, collapse = &amp;quot;|&amp;quot;), giovanniDF$text)
# This says go through each row and if something in that row is the same as something in our characters data frame (the &amp;quot;|&amp;quot; is working as an &amp;#39;or&amp;#39;, telling R that it could be Dorabella or Fiordiligi or Don Alfonso, etc.) then put &amp;#39;TRUE&amp;#39;

# The output is a list so we have to turn it into a data frame to merge it back with our data
donchargreplDF &amp;lt;-as.data.frame(donchargrepl)

# To merge something, you have to have a common key. Create an id row for each data frame you want to combine and that will be your key! 
giovanniDF$id &amp;lt;- 1:nrow(giovanniDF)
donchargreplDF$id &amp;lt;- 1:nrow(donchargreplDF)

# Now merge!
dontext_df &amp;lt;- merge(giovanniDF,donchargreplDF,by=&amp;quot;id&amp;quot;)

# Now things get complicated. We are going to &amp;#39;group&amp;#39; things according to names. We have name rows represented by &amp;#39;TRUE&amp;#39; and lines represented by &amp;#39;FALSE&amp;#39;. So we can create a group by telling R to make a new id row that starts anew each time it sees &amp;#39;TRUE&amp;#39;.

# Make groups of lines associated with names (help from here https://stackoverflow.com/questions/29376178/count-changes-to-contents-of-a-character-vector)
dontest_df &amp;lt;- dontext_df %&amp;gt;% mutate( 
   try_1 = cumsum(ifelse(donchargrepl == TRUE, 1, 0)) 
   )

# Now we want to copy our text column so we can eventually have two columns in which one has all of the lines and the other has the characters speaking those lines
# Create a duplicate column of the text
dontest_df$text2 &amp;lt;- dontest_df$text

# Let&amp;#39;s get rid of superfluous info for our character column (so, in this case, lines)
# Get rid of lines in one column
dontest_df$text &amp;lt;- ifelse(dontest_df$donchargrepl == TRUE, dontest_df$text, &amp;quot;&amp;quot;)
# This command says look through the text column and if the corresponding chargrepl column (our logical vector) is TRUE leave it alone but if it is else (&amp;#39;ifelse&amp;#39;) replace with &amp;quot;&amp;quot; (nothing).  

# Do the same for the other text column, but opposite
dontest_df$text2 &amp;lt;- ifelse(dontest_df$donchargrepl == FALSE, dontest_df$text2, &amp;quot;&amp;quot;)

# Now let&amp;#39;s aggregate things
donresult &amp;lt;- aggregate(text ~ try_1, data = dontest_df, paste, collapse = &amp;quot;&amp;quot;)
donresult2 &amp;lt;- aggregate(text2 ~ try_1, data = dontest_df, paste, collapse = &amp;quot;&amp;quot;)
# These commands are creating two data frames. The first one is just character names, the second is the lines associated with those characters. The collapse is also helping us out in that it&amp;#39;s putting each series of phrases in one cell. That&amp;#39;s because we are collapsing by the group id we created a few steps back (&amp;#39;try_1&amp;#39;). 

# Now we merge these data frames together by their shared column of group id
new_giovanni &amp;lt;- merge(donresult, donresult2, by=&amp;quot;try_1&amp;quot;)

# Let&amp;#39;s get rid of that id!
new_giovanni &amp;lt;- new_giovanni[,c(&amp;quot;text&amp;quot;,&amp;quot;text2&amp;quot;)]&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Transform this into something we can work with: tidy text!
tidy_giovanni &amp;lt;- new_giovanni %&amp;gt;%
  unnest_tokens(word, text2)

# Get rid of stop words

giovannitribble &amp;lt;- tidy_giovanni %&amp;gt;%
  anti_join(itstopwords)

giovannifreq &amp;lt;- giovannitribble %&amp;gt;%
  count(word, sort = TRUE) &lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;wordcloud(giovannifreq$word,giovannifreq$n, min.freq=5, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-17-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We do see some differences even here – Cosi has mention of friends, heart, love, etc. while Giovanni has some more negative words like “barbaro”. Both clearly are tied to the individual plots of the operas (you can tell which word cloud belongs where!)&lt;/p&gt;
&lt;p&gt;To round it out let’s do the last famous Da Ponte - Mozart colab, Le Nozze Di Figaro. This one is all in one document, so it’ll require less work to load it in.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;figaro &amp;lt;- readLines(&amp;quot;figaro/figaro.txt&amp;quot;)

figaroDF &amp;lt;- data_frame(text = figaro)

# Let&amp;#39;s get rid of stage directions (help from https://stackoverflow.com/questions/13529360/replace-text-within-parenthesis-in-r)
figaroDF$text &amp;lt;- gsub( &amp;quot; *\\(.*?\\) *&amp;quot;, &amp;quot;&amp;quot;, figaroDF$text)

# Get rid of everything between &amp;#39;&amp;lt; &amp;gt;&amp;#39; -- this is how arias and recits have been marked in this libretto
figaroDF$text &amp;lt;- gsub(&amp;quot;&amp;lt;[^&amp;gt;]+&amp;gt;&amp;quot;, &amp;quot;&amp;quot;, figaroDF$text)

# There are also some instances in which quotations are used, let&amp;#39;s get rid of those too
figaroDF$text &amp;lt;- gsub( &amp;#39;&amp;quot;&amp;#39;, &amp;#39;&amp;#39;, figaroDF$text)
# Note we chanage &amp;quot; to &amp;#39; in the gsub function to not confuse R -- if we had &amp;quot;&amp;quot;&amp;quot; R wouldn&amp;#39;t know what to do with that because it would read it as one quotation ending and another incomplete one.

# Also get rid of numbers (help from here: http://www.endmemo.com/program/R/gsub.php)
figaroDF$text &amp;lt;- gsub(&amp;quot;\\d+&amp;quot;,&amp;quot;&amp;quot;, figaroDF$text)
# Get rid of &amp;quot;d&amp;#39;&amp;quot; a shortening of &amp;quot;di&amp;quot; (easier to do it this way than as a stop word)
figaroDF$text &amp;lt;- gsub(&amp;quot;d&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, figaroDF$text)
# Get rid of &amp;quot;dell&amp;#39;&amp;quot;, a shortening of &amp;quot;dello/a/gli/i&amp;quot;
figaroDF$text &amp;lt;- gsub(&amp;quot;dell&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, figaroDF$text)
# You have to do this before l&amp;#39; because R would get rid of the &amp;quot;l&amp;#39;&amp;quot; of &amp;quot;dell&amp;#39;&amp;quot;, leaving you with &amp;quot;del&amp;quot; smooshed onto the next word, which is not what you want!
# Get rid of &amp;quot;l&amp;#39;&amp;quot; a shortening of &amp;quot;la&amp;quot; or &amp;quot;il&amp;quot; in front of words that start with a vowel
figaroDF$text &amp;lt;- gsub(&amp;quot;l&amp;#39;&amp;quot;,&amp;quot;&amp;quot;, figaroDF$text)

# There are some issues with this libretto, I have to make sure it can pick up the names as separate things
# figaroDF$text &amp;lt;- gsub(&amp;quot;DONNA&amp;quot;,&amp;quot;&amp;quot;, figaroDF$text)
# figaroDF$text &amp;lt;- gsub(&amp;quot;DON&amp;quot;,&amp;quot;&amp;quot;, figaroDF$text)

# Let&amp;#39;s transform this into transcript form
# First label whether a row is a line or character name

# Create a data frame with the character names that exist in the text
figcharacters = c(&amp;quot;IL CONTE&amp;quot;, &amp;quot;LA CONTESSA&amp;quot;, &amp;quot;SUSANNA&amp;quot;, &amp;quot;FIGARO&amp;quot;, &amp;quot;CHERUBINO&amp;quot;, &amp;quot;MARCELLINA&amp;quot;, &amp;quot;BARTOLO&amp;quot;,  &amp;quot;BASILIO&amp;quot;, &amp;quot;DON CURZIO&amp;quot;, &amp;quot;BARBARINA&amp;quot;, &amp;quot;ANTONIO&amp;quot;, &amp;quot;DUE DONNE&amp;quot;, &amp;quot;TUTTI&amp;quot;, &amp;quot;CORO&amp;quot;, &amp;quot;CONTADINELLE&amp;quot;) 
figchardf &amp;lt;- data.frame(figcharacters)

# Now use regular expression to identify when the row is a character name (TRUE) or not (FALSE)
# FALSE then means that we are looking at a line row
figchargrepl &amp;lt;- grepl(paste(figchardf$figcharacters, collapse = &amp;quot;|&amp;quot;), figaroDF$text)
# This says go through each row and if something in that row is the same as something in our characters data frame (the &amp;quot;|&amp;quot; is working as an &amp;#39;or&amp;#39;, telling R that it could be Dorabella or Fiordiligi or Don Alfonso, etc.) then put &amp;#39;TRUE&amp;#39;

# The output is a list so we have to turn it into a data frame to merge it back with our data
figchargreplDF &amp;lt;-as.data.frame(figchargrepl)

# To merge something, you have to have a common key. Create an id row for each data frame you want to combine and that will be your key! 
figaroDF$id &amp;lt;- 1:nrow(figaroDF)
figchargreplDF$id &amp;lt;- 1:nrow(figchargreplDF)

# Now merge!
figtext_df &amp;lt;- merge(figaroDF,figchargreplDF,by=&amp;quot;id&amp;quot;)

# Now things get complicated. We are going to &amp;#39;group&amp;#39; things according to names. We have name rows represented by &amp;#39;TRUE&amp;#39; and lines represented by &amp;#39;FALSE&amp;#39;. So we can create a group by telling R to make a new id row that starts anew each time it sees &amp;#39;TRUE&amp;#39;.

# Make groups of lines associated with names (help from here https://stackoverflow.com/questions/29376178/count-changes-to-contents-of-a-character-vector)
figtest_df &amp;lt;- figtext_df %&amp;gt;% mutate( 
   try_1 = cumsum(ifelse(figchargrepl == TRUE, 1, 0)) 
   )

# Now we want to copy our text column so we can eventually have two columns in which one has all of the lines and the other has the characters speaking those lines
# Create a duplicate column of the text
figtest_df$text2 &amp;lt;- figtest_df$text

# Let&amp;#39;s get rid of superfluous info for our character column (so, in this case, lines)
# Get rid of lines in one column
figtest_df$text &amp;lt;- ifelse(figtest_df$figchargrepl == TRUE, figtest_df$text, &amp;quot;&amp;quot;)
# This command says look through the text column and if the corresponding chargrepl column (our logical vector) is TRUE leave it alone but if it is else (&amp;#39;ifelse&amp;#39;) replace with &amp;quot;&amp;quot; (nothing).  

# Do the same for the other text column, but opposite
figtest_df$text2 &amp;lt;- ifelse(figtest_df$figchargrepl == FALSE, figtest_df$text2, &amp;quot;&amp;quot;)

# Now let&amp;#39;s aggregate things
figresult &amp;lt;- aggregate(text ~ try_1, data = figtest_df, paste, collapse = &amp;quot;&amp;quot;)
figresult2 &amp;lt;- aggregate(text2 ~ try_1, data = figtest_df, paste, collapse = &amp;quot;&amp;quot;)
# These commands are creating two data frames. The first one is just character names, the second is the lines associated with those characters. The collapse is also helping us out in that it&amp;#39;s putting each series of phrases in one cell. That&amp;#39;s because we are collapsing by the group id we created a few steps back (&amp;#39;try_1&amp;#39;). 

# Now we merge these data frames together by their shared column of group id
new_figaro &amp;lt;- merge(figresult, figresult2, by=&amp;quot;try_1&amp;quot;)

# Let&amp;#39;s get rid of that id!
new_figaro &amp;lt;- new_figaro[,c(&amp;quot;text&amp;quot;,&amp;quot;text2&amp;quot;)]&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Transform this into something we can work with: tidy text!
tidy_figaro &amp;lt;- new_figaro %&amp;gt;%
  unnest_tokens(word, text2)

figtribble &amp;lt;- tidy_figaro %&amp;gt;%
  anti_join(itstopwords)

figfreq &amp;lt;- figtribble %&amp;gt;%
  count(word, sort = TRUE) &lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;wordcloud(figfreq$word,figfreq$n, min.freq=7, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-20-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let’s look at some bigrams / trigrams&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;fig_bigrams &amp;lt;- new_figaro %&amp;gt;%
  unnest_tokens(bigram, text2, token = &amp;quot;ngrams&amp;quot;, n = 2)

figbigrams_separated &amp;lt;- fig_bigrams %&amp;gt;%
  separate(bigram, c(&amp;quot;word1&amp;quot;, &amp;quot;word2&amp;quot;), sep = &amp;quot; &amp;quot;)

figbigrams_filtered &amp;lt;- figbigrams_separated %&amp;gt;%
  filter(!word1 %in% itstopwords$word) %&amp;gt;%
  filter(!word2 %in% itstopwords$word)

# new bigram counts:
figbigram_counts &amp;lt;- figbigrams_filtered %&amp;gt;% 
  count(word1, word2, sort = TRUE)

figbigram_counts&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 1,763 x 3
##      word1    word2     n
##      &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
##  1  signor    conte     6
##  2   conte  susanna     5
##  3   conte      olà     4
##  4  saggio   signor     4
##  5 ballare   signor     3
##  6   conte    ebben     3
##  7     olà silenzio     3
##  8    pace     pace     3
##  9   parlo     amor     3
## 10 perdono  perdono     3
## # ... with 1,753 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Interestingly for an opera (which might suggest certain phrases being repeated multiple times) we have few bigrams. We see some repeats for the chorus “Bella Vita Militar”, for example.&lt;/p&gt;
&lt;p&gt;What about trigrams?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;new_figaro %&amp;gt;%
  unnest_tokens(trigram, text2, token = &amp;quot;ngrams&amp;quot;, n = 3) %&amp;gt;%
  separate(trigram, c(&amp;quot;word1&amp;quot;, &amp;quot;word2&amp;quot;, &amp;quot;word3&amp;quot;), sep = &amp;quot; &amp;quot;) %&amp;gt;%
  filter(!word1 %in% itstopwords$word,
         !word2 %in% itstopwords$word,
         !word3 %in% itstopwords$word) %&amp;gt;%
  count(word1, word2, word3, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 708 x 4
##       word1    word2    word3     n
##       &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
##  1  ballare   signor  contino     3
##  2    conte      olà silenzio     3
##  3     vuol  ballare   signor     3
##  4   amanti costanti  seguaci     2
##  5     amor    donne   vedete     2
##  6      bel    fiore     almo     2
##  7      bel guadagno    colla     2
##  8    conte    gente    gente     2
##  9 costanti  seguaci     onor     2
## 10    donne   vedete     s&amp;#39;io     2
## # ... with 698 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Again, trigrams aren’t THAT helpful when looking at things like an opera, because part of the structure of opera (at least at this time) was to have a specific structure to arias, in which certain phrases would be repeated. This might be more insightful if you want to look at bigrams in a varied, larger data set, like a bunch of tweets or maybe a corpus of newspaper data. Or we could combine all of our operas and look at bigrams to see if there are any instances of two words happening together. First though, let’s compare at how ‘love’ is discussed in each of these operas.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;figbigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;amore&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 3 x 2
##     word2     n
##     &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1 ch&amp;#39;oggi     1
## 2 l&amp;#39;abito     1
## 3   unita     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Sometimes this is shortened

figbigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;amor&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 4 x 2
##       word2     n
##       &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1     donne     2
## 2      pace     1
## 3  sognando     1
## 4 vegliando     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Sometimes &amp;#39;amore&amp;#39; is the second word, not the first (flexible word order here)

figbigrams_filtered %&amp;gt;%
  filter(word2 == &amp;quot;amore&amp;quot;) %&amp;gt;%
  count(word1, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6 x 2
##      word1     n
##      &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1 cherubin     1
## 2      man     1
## 3  mentono     1
## 4      ove     1
## 5   sforza     1
## 6   spagna     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;figbigrams_filtered %&amp;gt;%
  filter(word2 == &amp;quot;amor&amp;quot;) %&amp;gt;%
  count(word1, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 10 x 2
##        word1     n
##        &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
##  1     parlo     3
##  2  adoncino     1
##  3    antico     1
##  4    felice     1
##  5 intendete     1
##  6      nomi     1
##  7   piaceri     1
##  8     porgi     1
##  9      solo     1
## 10    tenero     1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;See how it takes a few tries to get something that makes more sense with the data? In this instance, we see that looking at where the shortened ‘amor’ pops up as the second word gives us a little more information about how love is discussed in Le Nozze di Figaro.&lt;/p&gt;
&lt;p&gt;Let’s look at Don Giovanni now…&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;don_bigrams &amp;lt;- new_giovanni %&amp;gt;%
  unnest_tokens(bigram, text2, token = &amp;quot;ngrams&amp;quot;, n = 2)

donbigrams_separated &amp;lt;- don_bigrams %&amp;gt;%
  separate(bigram, c(&amp;quot;word1&amp;quot;, &amp;quot;word2&amp;quot;), sep = &amp;quot; &amp;quot;)

donbigrams_filtered &amp;lt;- donbigrams_separated %&amp;gt;%
  filter(!word1 %in% itstopwords$word) %&amp;gt;%
  filter(!word2 %in% itstopwords$word)

# Let&amp;#39;s check out &amp;#39;love&amp;#39;!

# donbigrams_filtered %&amp;gt;%
#   filter(word1 == &amp;quot;amore&amp;quot;) %&amp;gt;%
#   count(word2, sort = TRUE)
# No instances for this one

# Sometimes this is shortened

donbigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;amor&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 1 x 2
##       word2     n
##       &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1 consiglio     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Sometimes &amp;#39;amore&amp;#39; is the second word, not the first (flexible word order here)

donbigrams_filtered %&amp;gt;%
  filter(word2 == &amp;quot;amore&amp;quot;) %&amp;gt;%
  count(word1, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 2 x 2
##     word1     n
##     &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1 fingere     1
## 2   parla     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;donbigrams_filtered %&amp;gt;%
  filter(word2 == &amp;quot;amor&amp;quot;) %&amp;gt;%
  count(word1, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 4 x 2
##       word1     n
##       &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1      fido     2
## 2 innocente     1
## 3     sorte     1
## 4    tenero     1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Again we see the last one is more effective in picking up examples. Already we can see a shift in tone from Figaro to Giovanni – “fingere” for example means to pretend or fake.&lt;/p&gt;
&lt;p&gt;How about Cosi?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;bigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;amore&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6 x 2
##      word2     n
##      &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1    altri     1
## 2   d&amp;#39;esca     1
## 3       en     1
## 4   esempi     1
## 5 singolar     1
## 6 terminar     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Sometimes this is shortened

bigrams_filtered %&amp;gt;%
  filter(word1 == &amp;quot;amor&amp;quot;) %&amp;gt;%
  count(word2, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 7 x 2
##           word2     n
##           &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1        andate     1
## 2 contentissimi     1
## 3          lega     1
## 4     secondate     1
## 5        simile     1
## 6      virtuoso     1
## 7         vista     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Sometimes &amp;#39;amore&amp;#39; is the second word, not the first (flexible word order here)

bigrams_filtered %&amp;gt;%
  filter(word2 == &amp;quot;amore&amp;quot;) %&amp;gt;%
  count(word1, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 7 x 2
##      word1     n
##      &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1  balsamo     1
## 2 cangiano     1
## 3    desio     1
## 4   giusto     1
## 5  intatto     1
## 6    speme     1
## 7  trattar     1&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;bigrams_filtered %&amp;gt;%
  filter(word2 == &amp;quot;amor&amp;quot;) %&amp;gt;%
  count(word1, sort = TRUE)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 11 x 2
##        word1     n
##        &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
##  1   alfonso     1
##  2  antidoto     1
##  3       dio     1
##  4    madame     1
##  5    mangia     1
##  6 pennacchi     1
##  7     senza     1
##  8      sola     1
##  9    tenero     1
## 10   vezzosi     1
## 11      voci     1&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s combine all of these data frames together to look at everything at once. Let’s build a mini Mozart opera corpus.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s first add columns so we know which line belongs to which opera
# Let&amp;#39;s also add an &amp;#39;id&amp;#39; -- this will be helpful later when we want to look at sentiment changes over time 
new_data$opera &amp;lt;- &amp;quot;COSI&amp;quot; 
new_data$id &amp;lt;- 1:nrow(new_data)
new_giovanni$opera&amp;lt;-&amp;quot;GIOVANNI&amp;quot;
new_giovanni$id &amp;lt;- 1:nrow(new_giovanni)
new_figaro$opera&amp;lt;-&amp;quot;FIGARO&amp;quot;
new_figaro$id &amp;lt;- 1:nrow(new_figaro)

mozart &amp;lt;- rbind.fill(new_data, new_giovanni, new_figaro)

#Let&amp;#39;s play around with this and look at a word cloud representation
tidy_mozart &amp;lt;- mozart %&amp;gt;%
  unnest_tokens(word, text2)

moztribble &amp;lt;- tidy_mozart %&amp;gt;%
  anti_join(itstopwords)

mozartfreq &amp;lt;- moztribble %&amp;gt;%
  count(word, sort = TRUE) 

# Because we have more words, let&amp;#39;s up the minimum frequency to 10
wordcloud(mozartfreq$word,mozartfreq$n, min.freq=10, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-26-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We don’t have the best sentiment dictionary for Italian (especially this kind of Italian, which is 200+ years old!) nor do we have a ‘built in’ sentiment dictionary from a package. If we were working with English, we could use the tidytext package to download the Bing sentiment corpus and use that, etc. But we will have to make our own! I’ll be using the sentiment dictionary Andrea Cirillo put together to look at Italian tweets, available &lt;a href=&#34;https://github.com/AndreaCirilloAC/TweetIT/tree/master/lexicon/IT&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# First let&amp;#39;s load in some text files, which are just the words Andrea has collected that I copy-pasted into a word doc and &amp;#39;saved as&amp;#39; a txt file.
pos &amp;lt;- readLines(&amp;quot;itpositivewords.txt&amp;quot;)
neg &amp;lt;- readLines(&amp;quot;itnegativewords.txt&amp;quot;)

# Turn these into data frames
posDF &amp;lt;- data_frame(text = pos)
negDF &amp;lt;- data_frame(text = neg)

# Create a new column for each with a marker of what kind of sentiment we are dealing with
posDF$sentiment &amp;lt;- &amp;quot;positive&amp;quot;
negDF$sentiment &amp;lt;- &amp;quot;negative&amp;quot;

# Combine these into a data frame
sentiment &amp;lt;- rbind.fill(posDF, negDF)

# We have lots of empty cells, get rid of those
# Mark empty cells as NA
sentiment$text[sentiment$text==&amp;quot;&amp;quot;] &amp;lt;- NA

# Get rid of NA rows (help from here: https://stackoverflow.com/questions/12763890/exclude-blank-and-na-in-r)
sentimentDF &amp;lt;- na.omit(sentiment)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now that we have our own new sentiment dictionary, let’s evaluate the sentiment of our operas! Code adapted from &lt;a href=&#34;http://tidytextmining.com/sentiment.html&#34;&gt;Silge and Robinson&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We have to change the name again
names(sentimentDF)[names(sentimentDF)==&amp;quot;text&amp;quot;] &amp;lt;- &amp;quot;word&amp;quot;

## Sentiment analysis
senti_word_counts &amp;lt;- tidy_mozart %&amp;gt;%
  inner_join(sentimentDF) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

# senti_word_counts

# The first time I ran this, I didn&amp;#39;t get too many matches. Then I went back to the text file, added some words to both postive and negative text files and now we have a semblance of a result. Hopefully you&amp;#39;ll have more luck with your more contemporary texts!

# Now we can graph these
senti_word_counts %&amp;gt;%
  filter(n &amp;gt; 15) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-28-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;You will most likely have to fiddle with this a bit, depending on your data and ESPECIALLY if you’ve ‘made’ your own sentiment dictionary. For example, the first time I ran this, it was counting ‘caro’ as both positive and negative. This is because ‘caro’ means ‘dear’ – which in operas is usually used as a term of endearment, whereas in modern Italian it means expensive! So I had to go back to the negative sentiment list and delete it.&lt;/em&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s see how each opera compares
# Cosi
cosisenti_word_counts &amp;lt;- tribble %&amp;gt;%
  inner_join(sentimentDF) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

cosisenti_word_counts %&amp;gt;%
  filter(n &amp;gt; 5) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-29-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Giovanni
donsenti_word_counts &amp;lt;- tidy_giovanni %&amp;gt;%
  inner_join(sentimentDF) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

donsenti_word_counts %&amp;gt;%
  filter(n &amp;gt; 5) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-29-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Figaro
figsenti_word_counts &amp;lt;- tidy_figaro %&amp;gt;%
  inner_join(sentimentDF) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

figsenti_word_counts %&amp;gt;%
  filter(n &amp;gt; 5) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-29-3.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;As we might guess, the opera seria (‘serious’ or melodramatic opera) Don Giovanni seems to be skewed on the negative side, however the two opera buffas (comedic operas) do not follow this pattern. Cosi Fan Tutte has a lot of negative sentiment, more than positive sentiment especially compared to Le Nozze di Figaro. This makes sense when we consider the plots (a lot of the opera is made up of pining; confusion and farewells) but it’s kind of interesting to see it scaled back a bit especially when we think about the tone of an opera overall and how words fit into that.&lt;/p&gt;
&lt;p&gt;Something really cool to do would be to take advantage of opera as music drama by noting which key each of these words are being sung in (i.e. whether or not we are working in a major or minor key) to add an extra layer to our analysis, but that is a little too much work for the time being! This would be especially cool to try with Wagner, but I digress.&lt;/p&gt;
&lt;p&gt;Let’s look at how sentiment has changed over the course of each opera! (Again, adapted from &lt;a href=&#34;http://tidytextmining.com/sentiment.html&#34;&gt;Silge and Robinson&lt;/a&gt;)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;mozartsenti &amp;lt;- moztribble %&amp;gt;%
  inner_join(sentimentDF) %&amp;gt;%
  count(opera, index = id %/% 1, sentiment) %&amp;gt;%
  spread(sentiment, n, fill = 0) %&amp;gt;%
  mutate(sentiment = positive - negative)

# Graph
ggplot(mozartsenti, aes(index, sentiment, fill = opera)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~opera, ncol = 2, scales = &amp;quot;free_x&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-30-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# That previous example was looking at line by line, but we can group these together a little more to see larger trends (you can modify this depending on your data)
mozartsenti &amp;lt;- moztribble %&amp;gt;%
  inner_join(sentimentDF) %&amp;gt;%
  count(opera, index = id %/% 10, sentiment) %&amp;gt;%
  spread(sentiment, n, fill = 0) %&amp;gt;%
  mutate(sentiment = positive - negative)

# Graph
ggplot(mozartsenti, aes(index, sentiment, fill = opera)) +
  geom_col(show.legend = FALSE) +
  facet_wrap(~opera, ncol = 2, scales = &amp;quot;free_x&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-30-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Let’s check out some more things. The following examples and code comes from &lt;a href=&#34;http://tidytextmining.com/tfidf.html&#34;&gt;Silge and Robinson’s Text Mining with R book&lt;/a&gt; which I have adapted for our data set that we’ve been working with.&lt;/p&gt;
&lt;p&gt;Julia Silge’s &lt;a href=&#34;https://juliasilge.com/blog/life-changing-magic/&#34;&gt;blog&lt;/a&gt; also has a lot of cool examples of how to play around with these tools and how to get different visualizations.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Use representation of text that has stop words removed
mozart_words &amp;lt;- moztribble %&amp;gt;%
  count(opera, word, sort = TRUE) %&amp;gt;%
  ungroup()

total_words &amp;lt;- mozart_words %&amp;gt;% 
  group_by(opera) %&amp;gt;% 
  summarize(total = sum(n))

mozart_words &amp;lt;- left_join(mozart_words, total_words)

mozart_words&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6,441 x 4
##     opera     word     n total
##     &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;
##  1 FIGARO    conte   233  4838
##  2 FIGARO contessa   134  4838
##  3 FIGARO  susanna    49  4838
##  4 FIGARO   signor    46  4838
##  5   COSI      cor    29  4200
##  6   COSI      due    27  4200
##  7 FIGARO   madama    25  4838
##  8 FIGARO   figaro    24  4838
##  9   COSI   amanti    22  4200
## 10   COSI    bella    21  4200
## # ... with 6,431 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;As Silge and Robinson explain (adapted to fit our example), now we have “one row in this mozart_words data frame for each word-opera combination; n is the number of times that word is used in that opera and total is the total words in that opera. The usual suspects are here with the highest n, “the”, “and”, “to”, and so forth (‘il’, ‘la’, ‘che’, etc.) … let’s look at the distribution of n/total for each opera, the number of times a word appears in an opera divided by the total number of terms (words) in that opera. This is exactly what term frequency is.”&lt;/p&gt;
&lt;p&gt;Term frequency is an important concept in text mining. Let’s see some examples and explore what this term means and why it is useful.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;ggplot(mozart_words, aes(n/total, fill = opera)) +
  geom_histogram(show.legend = FALSE) +
  xlim(NA, 0.0009) +
  facet_wrap(~opera, ncol = 2, scales = &amp;quot;free_y&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-32-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;What this is trying to show is that “there are many words that occur rarely and fewer words that occur frequently”, a tenant of Zipf’s Law. Zipf’s Law is probably something you’ve heard of if you have taken corpus linguistics – it means “the frequency that a word appears is inversely proportionally to its rank”. We can see this with our own data (all code take from Silge and Robinson):&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq_by_rank &amp;lt;- mozart_words %&amp;gt;% 
  group_by(opera) %&amp;gt;% 
  mutate(rank = row_number(), 
         `term frequency` = n/total)

# Better to plot to see relationship
freq_by_rank %&amp;gt;% 
  ggplot(aes(rank, `term frequency`, color = opera)) + 
  geom_line(size = 1.2, alpha = 0.8) + 
  scale_x_log10() +
  scale_y_log10()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-33-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Kind of cool right? Anyway, the main idea that is useful for us out of this is thinking about the term-frequency / inverse document frequency representation of our data. As Silge and Robinson explain, “the idea of tf-idf is to find the important words for the content of each document by decreasing the weight for commonly used words and increasing the weight for words that are not used very much in a collection or corpus of documents” – basically, that if a word like ‘amor’ is really common across all of our librettos, it’s not really informative as to the unique character of each opera. If we are using text mining and analysis as a way to evaluate trends in our data, the tf-idf is really useful. It’s a way of seeing what are unique features in groupings of data. For example, if you have a collection of interviews on the same topic, what are some different features that come up for each person / interview session? If you are looking at a corpus of tweets collected by hashtag and have grouped them by time frame (let’s say grouped in terms of weeks) what are some unique terms for each time period? etc. etc.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Let&amp;#39;s get this td-idf started
mozart_words2 &amp;lt;- mozart_words %&amp;gt;%
  bind_tf_idf(word, opera, n)
mozart_words2&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6,441 x 7
##     opera     word     n total          tf       idf      tf_idf
##     &amp;lt;chr&amp;gt;    &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;       &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;       &amp;lt;dbl&amp;gt;
##  1 FIGARO    conte   233  4838 0.048160397 1.0986123 0.052909604
##  2 FIGARO contessa   134  4838 0.027697396 1.0986123 0.030428699
##  3 FIGARO  susanna    49  4838 0.010128152 1.0986123 0.011126912
##  4 FIGARO   signor    46  4838 0.009508061 0.0000000 0.000000000
##  5   COSI      cor    29  4200 0.006904762 0.0000000 0.000000000
##  6   COSI      due    27  4200 0.006428571 0.0000000 0.000000000
##  7 FIGARO   madama    25  4838 0.005167425 0.0000000 0.000000000
##  8 FIGARO   figaro    24  4838 0.004960728 1.0986123 0.005449916
##  9   COSI   amanti    22  4200 0.005238095 0.4054651 0.002123865
## 10   COSI    bella    21  4200 0.005000000 0.0000000 0.000000000
## # ... with 6,431 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Look at high tf idf (i.e. terms that are most particular to certain operas)
mozart_words2 %&amp;gt;%
  select(-total) %&amp;gt;%
  arrange(desc(tf_idf))&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6,441 x 6
##       opera      word     n          tf      idf      tf_idf
##       &amp;lt;chr&amp;gt;     &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;       &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;       &amp;lt;dbl&amp;gt;
##  1   FIGARO     conte   233 0.048160397 1.098612 0.052909604
##  2   FIGARO  contessa   134 0.027697396 1.098612 0.030428699
##  3   FIGARO   susanna    49 0.010128152 1.098612 0.011126912
##  4 GIOVANNI   masetto    21 0.006485485 1.098612 0.007125033
##  5 GIOVANNI leporello    19 0.005867820 1.098612 0.006446459
##  6   FIGARO    figaro    24 0.004960728 1.098612 0.005449916
##  7     COSI   despina    19 0.004523810 1.098612 0.004969913
##  8     COSI  guilelmo    15 0.003571429 1.098612 0.003923615
##  9   FIGARO    paggio    16 0.003307152 1.098612 0.003633278
## 10     COSI   alfonso    13 0.003095238 1.098612 0.003400467
## # ... with 6,431 more rows&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Plot it 
plot_mozart &amp;lt;- mozart_words2 %&amp;gt;%
  arrange(desc(tf_idf)) %&amp;gt;%
  mutate(word = factor(word, levels = rev(unique(word))))

plot_mozart %&amp;gt;% 
  top_n(20) %&amp;gt;%
  ggplot(aes(word, tf_idf, fill = opera)) +
  geom_col() +
  labs(x = NULL, y = &amp;quot;tf-idf&amp;quot;) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-34-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;We see that character names are the most common, which makes sense! This is also similar to &lt;a href=&#34;http://tidytextmining.com/tfidf.html&#34;&gt;the results that Silge and Robinson find&lt;/a&gt; in their analysis of Jane Austen’s novels.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Plot these individually
plot_mozart %&amp;gt;% 
  group_by(opera) %&amp;gt;% 
  top_n(15) %&amp;gt;% 
  ungroup %&amp;gt;%
  ggplot(aes(word, tf_idf, fill = opera)) +
  geom_col(show.legend = FALSE) +
  labs(x = NULL, y = &amp;quot;tf-idf&amp;quot;) +
  facet_wrap(~opera, ncol = 2, scales = &amp;quot;free&amp;quot;) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-10-19-intro-to-text-mining_files/figure-html/unnamed-chunk-35-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Intro to Social Media Mining</title>
      <link>/portfolio/2017-10-23-Intro-to-Social-Media-Mining/</link>
      <pubDate>Thu, 23 Jul 2015 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-10-23-Intro-to-Social-Media-Mining/</guid>
      <description>&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;packs = c(&amp;quot;twitteR&amp;quot;,&amp;quot;RCurl&amp;quot;,&amp;quot;RJSONIO&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;,&amp;quot;devtools&amp;quot;,&amp;quot;DataCombine&amp;quot;,&amp;quot;ggmap&amp;quot;,&amp;quot;topicmodels&amp;quot;,&amp;quot;slam&amp;quot;,&amp;quot;Rmpfr&amp;quot;,&amp;quot;tm&amp;quot;,&amp;quot;stringr&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,&amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;, &amp;quot;readr&amp;quot;)
# lapply(packs, install.packages, character.only=T)
lapply(packs, library, character.only=T)&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;getting-data-from-twitter&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Getting data from Twitter&lt;/h2&gt;
&lt;p&gt;After you have set up your developer account on Twitter, copy paste the key, secret, tok (token) and tok secret (token secret). Take care to keep these private – you don’t want someone else using these credentials to collect data (they could abuse the limit of tweets you are allowed to get and lock you out of accessing the API for a while, etc.)&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# key = &amp;quot;YOUR KEY HERE&amp;quot;
# secret = &amp;quot;YOUR SECRET HERE&amp;quot;

# tok = &amp;quot;YOUR TOK HERE&amp;quot;
# tok_sec = &amp;quot;YOUR TOK_SEC HERE&amp;quot;

twitter_oauth &amp;lt;- setup_twitter_oauth(key, secret, tok, tok_sec)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now you have set up your ‘handshake’ with the API and are ready to collect data. I’ll go over two examples, one searching by coordinate (with no specific search term) and the other searching by a specific hashtag.&lt;/p&gt;
&lt;p&gt;I’m interested in the Mission District neighborhood in San Francisco, California. I obtain a set of coordinates using Google maps and plug that into the ‘geocode’ parameter and then set a radius of 1 kilometer. I know from experience that I only get around 1,000 - 2,000 posts per time I do this, so I set the number of tweets (n) I would like to get from Twitter at ‘7,000’. If you are looking at a more ‘active’ area, or a larger area (more about this later) you can always adjust this number. The API will give you a combination of the most “recent or popular”&amp;quot; tweets that usually extend back about 5 days or so. If you are looking at a smaller area, this means to get any kind of decent tweet corpus you’ll have to spent some time collecting data week after week. Also if you want to look at a larger area than a 3-4 kilometer radius, a lot of times you’ll get a bunch of spam-like posts that don’t have latitude and longitude coordinates associated with them. A work around I thought of (for another project looking at posts in an entire city) is to figure out a series of spots to collect tweets (trying to avoid overlap as much as possible) and stiching those data frames all together and getting rid of any duplicate posts you picked up if your radii overlapped.&lt;/p&gt;
&lt;p&gt;Luckily for the Mission District, however, we are interested in a smaller area and don’t have to worry about multiple sampling points and rbind’ing data frames together, and just run the searchTwitter function once:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geo &amp;lt;- searchTwitter(&amp;#39;&amp;#39;,n=7000, geocode=&amp;#39;37.76,-122.42,1km&amp;#39;,
                     retryOnRateLimit=1)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now you have a list of tweets. Lists are very difficult to deal with in R, so you convert this into a data frame:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geoDF&amp;lt;-twListToDF(geo)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Chances are there will be emojis in your Twitter data. You can ‘transform’ these emojis into prose using this code as well as a &lt;a href=&#34;https://github.com/lyons7/emojidictionary&#34;&gt;CSV file&lt;/a&gt; I’ve put together of what all of the emojis look like in R. (The idea for this comes from &lt;a href=&#34;http://opiateforthemass.es/articles/emoticons-in-R/&#34;&gt;Jessica Peterka-Bonetta’s work&lt;/a&gt; – she has a list of emojis as well, but it does not include the newest batch of emojis, Unicode Version 9.0, nor the different skin color options for human-based emojis). If you use this emoji list for your own research, please make sure to acknowledge both myself and Jessica.&lt;/p&gt;
&lt;p&gt;Load in the CSV file. You want to make sure it is located in the correct working directory so R can find it when you tell it to read it in.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# library(readr)
# Solution for weird encoding issue on Windows. The &amp;quot;trim_ws&amp;quot; argument is really important because it ensures the spaces are kept in 
between emojis so you can count each one individually.

emojis &amp;lt;- read_csv(&amp;quot;Emoji Dictionary 2.1.csv&amp;quot;, col_names=TRUE, 
                    trim_ws=FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To transform the emojis, you first need to transform the tweet data into ASCII:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geoDF$text &amp;lt;- iconv(geoDF$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, sub = &amp;quot;byte&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To ‘count’ the emojis you do a find and replace using the CSV file of ‘Decoded Emojis’ as a reference. Here I am using the DataCombine package. What this does is identifies emojis in the tweets and then replaces them with a prose version. I used whatever description pops up when hovering one’s cursor over an emoji on an Apple emoji keyboard. If not completely the same as other platforms, it provides enough information to find the emoji in question if you are not sure which one was used in the post.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;geodata &amp;lt;- FindReplace(data = geoDF, Var = &amp;quot;text&amp;quot;, 
                            replaceData = emojis,
                       from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                       exact = FALSE)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now might be a good time to save this file, perhaps in CSV format with the date of when the data was collected:&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# write.csv(geodata,file=paste(&amp;quot;ALL&amp;quot;,Sys.Date(),&amp;quot;.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Let’s try searching by hashtag. I’ll search by the tag ‘#GamesForAnimals’. Note I’ve changed the limit to 1,000 because I probably will get more results. I’ve also gotten rid of the geocode parameter.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tag &amp;lt;- searchTwitter(&amp;#39;#GamesForAnimals&amp;#39;,n=1000,retryOnRateLimit=1)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now do all of the steps above: convert to data frame, then identify emojis.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;tagDF&amp;lt;-twListToDF(tag)

tagDF$text &amp;lt;- iconv(tagDF$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, sub = &amp;quot;byte&amp;quot;)

tagdata &amp;lt;- FindReplace(data = tagDF, Var = &amp;quot;text&amp;quot;, 
                            replaceData = emojis,
                       from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                       exact = FALSE)
# Check it out
# View(tagdata)&lt;/code&gt;&lt;/pre&gt;
&lt;/div&gt;
&lt;div id=&#34;getting-data-from-youtube&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Getting data from Youtube&lt;/h2&gt;
&lt;p&gt;Let’s look at some Youtube comments. To do this, you need to set up some things on Google in order to access their API. The &lt;a href=&#34;https://github.com/soodoku/tuber&#34;&gt;package documentation&lt;/a&gt; for the package we will use to get Youtube data has information on how to do this.&lt;/p&gt;
&lt;p&gt;Had some issues with authorization, got help from &lt;a href=&#34;https://stackoverflow.com/questions/42818602/unable-to-connect-r-with-youtube-api/42934996#42934996&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# To install the tuber package use the following command
# First you have to install devtools (if you haven&amp;#39;t already)
# install.packages(&amp;quot;devtools&amp;quot;)
# devtools::install_github(&amp;quot;soodoku/tuber&amp;quot;, build_vignettes = TRUE)
library(tuber)&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Input your information from your account you set up 
# yt_oauth(&amp;quot;app_id&amp;quot;, &amp;quot;app_password&amp;quot;, token=&amp;quot;&amp;quot;)

# Let&amp;#39;s get comments from a Youtube video of a weird guy talking about raven ownership
res2 &amp;lt;- get_comment_threads(c(video_id=&amp;quot;izpdLM4VOfY&amp;quot;), max_results = 300)

# Let&amp;#39;s save those as a CSV
# write.csv(res2,file=paste(&amp;quot;WeirdRavenDudeComments2&amp;quot;,Sys.Date(),&amp;quot;.csv&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It works! You can try this with any video – just copy paste the number that comes after the ‘v=’ in the video address. For example, a video with the web link ‘&lt;a href=&#34;https://www.youtube.com/watch?v=AUM99UXMbow&#34; class=&#34;uri&#34;&gt;https://www.youtube.com/watch?v=AUM99UXMbow&lt;/a&gt;’ has the video id ‘AUM99UXMbow’.&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Intro to Analyzing Social Media Data</title>
      <link>/portfolio/2017-11-05-analyzing-social-media-data/</link>
      <pubDate>Sun, 05 Nov 2017 21:13:14 -0500</pubDate>
      
      <guid>/portfolio/2017-11-05-analyzing-social-media-data/</guid>
      <description>&lt;p&gt;Today we are going to talk about applying text mining techniques to social media data. You can download this data yourself (if you have everything set up with the Twitter and YouTube APIs) or you can access the data we’ll be discussing here in R data format.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# packs = c(&amp;quot;twitteR&amp;quot;, &amp;quot;stringr&amp;quot;,&amp;quot;ggplot2&amp;quot;,&amp;quot;devtools&amp;quot;,&amp;quot;DataCombine&amp;quot;,&amp;quot;ggmap&amp;quot;,&amp;quot;tm&amp;quot;,&amp;quot;wordcloud&amp;quot;,&amp;quot;plyr&amp;quot;,&amp;quot;tuber&amp;quot;,&amp;quot;tidytext&amp;quot;,&amp;quot;dplyr&amp;quot;,&amp;quot;tidyr&amp;quot;,&amp;quot;readr&amp;quot;,&amp;quot;ggrepel&amp;quot;,&amp;quot;emoGG&amp;quot;,&amp;quot;lubridate&amp;quot;,&amp;quot;corpus&amp;quot;, &amp;quot;purrr&amp;quot;, &amp;quot;broom&amp;quot;)
# lapply(packs, library, character.only=T)

# You might have to install some of these -- check the &amp;#39;Packages&amp;#39; tab in R Studio to see which ones you already have. For &amp;#39;emoGG&amp;#39; you need to download it directly from github, using the following:
# devtools::install_github(&amp;quot;dill/emoGG&amp;quot;)

# Make sure to set the right working directory
# setwd(&amp;quot;/Users/katelyons/Documents/Workshop&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;div id=&#34;twitter-geographic-analyses&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;Twitter + Geographic Analyses&lt;/h2&gt;
&lt;p&gt;First we will look at some Twitter data of the Wicker Park neighborhood in Chicago, IL.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# key = &amp;quot;YOUR KEY HERE&amp;quot;
# secret = &amp;quot;YOUR SECRET HERE&amp;quot;

# tok = &amp;quot;YOUR TOK HERE&amp;quot;
# tok_sec = &amp;quot;YOUR TOK_SEC HERE&amp;quot;

# twitter_oauth &amp;lt;- setup_twitter_oauth(key, secret, tok, tok_sec)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If you don’t have the Twitter API set up, you can access the data in R data format &lt;a href=&#34;https://www.dropbox.com/s/nzh8d6fdwcdra9d/sampledata.Rda?dl=0&#34;&gt;here&lt;/a&gt;. If you get the data this way, start from the ‘twListToDF’ step.&lt;/p&gt;
&lt;p&gt;Now you have set up your ‘handshake’ with the API and are ready to collect data. We will search by coordinate for all tweets that have occurred within a 1 kilometer radius of a central point in the Wicker Park neighborhood in Chicago, IL.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# geo &amp;lt;- searchTwitter(&amp;#39;&amp;#39;,n=7000,geocode=&amp;#39;41.908602,-87.677417,1km&amp;#39;,retryOnRateLimit=1)
# Save tweet data (if you want)
# save(geo,file=paste(&amp;quot;sampletweetdata.Rda&amp;quot;))

# If you need to load that data (make sure you are in the right directory)
load(&amp;#39;sampletweetdata.Rda&amp;#39;)

# Convert to data frame
geoDF&amp;lt;-twListToDF(geo)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have a data frame. We will now identify emojis, select just those tweets that come from Instagram and clean this data (get rid of links, special characters, etc.) so we can do text analyses. You can access the emoji dictionary &lt;a href=&#34;https://www.dropbox.com/s/orpj7lmh5ueapo1/Emoji%20Dictionary%202.1.csv?dl=0&#34;&gt;here&lt;/a&gt;. The code for cleaning the tweets comes from &lt;a href=&#34;http://tidytextmining.com/twitter.html&#34;&gt;Silge and Robinson’s book&lt;/a&gt; – this is special and awesome code to do this because it keeps hashtags and @ mentions. Other methods of cleaning text will count ‘#’ and ‘@’ as special characters and will get rid of them.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Convert the encoding so you can identify emojis
geoDF$text &amp;lt;- iconv(geoDF$text, from = &amp;quot;latin1&amp;quot;, to = &amp;quot;ascii&amp;quot;, sub = &amp;quot;byte&amp;quot;)

# Load in emoji dictionary. The &amp;#39;trim_ws&amp;#39; argument is super important -- you need those spaces so the emojis aren&amp;#39;t all squished together!
# emojis &amp;lt;- read_csv(&amp;quot;Emoji Dictionary 2.1.csv&amp;quot;, col_names=TRUE, trim_ws=FALSE)
# If you don&amp;#39;t get weird encoding issue just use read.csv
emojis &amp;lt;- read.csv(&amp;quot;Emoji Dictionary 5.0.csv&amp;quot;, header=T)


# Go through and identify emojis
geodata &amp;lt;- FindReplace(data = geoDF, Var = &amp;quot;text&amp;quot;, 
                            replaceData = emojis,
                       from = &amp;quot;R_Encoding&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                       exact = FALSE)

# Just keep those tweets that come from Instagram
wicker &amp;lt;- geodata[geodata$statusSource == 
        &amp;quot;&amp;lt;a href=\&amp;quot;http://instagram.com\&amp;quot; rel=\&amp;quot;nofollow\&amp;quot;&amp;gt;Instagram&amp;lt;/a&amp;gt;&amp;quot;, ]

# Get rid of stuff particular to the data (here encodings of links and such)
# Most of these are characters I don&amp;#39;t have encodings for (other scripts, etc.)
wicker$text = gsub(&amp;quot;Just posted a photo&amp;quot;,&amp;quot;&amp;quot;, wicker$text)
wicker$text = gsub( &amp;quot;&amp;lt;.*?&amp;gt;&amp;quot;, &amp;quot;&amp;quot;, wicker$text)

# Now time to clean your posts. First let&amp;#39;s make our own list of stop words again, adding additional stop words to the tidy text stop word list from the tm package stop word list.
# This makes a larger list of stop words combining those from the tm package and tidy text -- even though the tm package stop word list is pretty small anyway, just doing this just in case
data(stop_words)
mystopwords &amp;lt;- c(stopwords(&amp;#39;english&amp;#39;),stop_words$word, &amp;quot;im&amp;quot;)

# Now for Silge and Robinson&amp;#39;s code. What this is doing is getting rid of 
# URLs, re-tweets (RT) and ampersands. This also gets rid of stop words 
# without having to get rid of hashtags and @ signs by using 
# str_detect and filter! 
reg &amp;lt;- &amp;quot;([^A-Za-z_\\d#@&amp;#39;]|&amp;#39;(?![A-Za-z_\\d#@]))&amp;quot;
tidy_wicker &amp;lt;- wicker %&amp;gt;% 
  filter(!str_detect(text, &amp;quot;^RT&amp;quot;)) %&amp;gt;%
  mutate(text = str_replace_all(text, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, text, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% mystopwords,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have cleaned posts in tidy format. Let’s adapt some of &lt;a href=&#34;http://tidytextmining.com/twitter.html&#34;&gt;Silge and Robinson’s code&lt;/a&gt; to look at frequent terms and then map concentrations of frequent terms. Note how I’ve grouped these by latitude and longitude. This will help us later on when we want to group them by coordinate.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq &amp;lt;- tidy_wicker %&amp;gt;% 
  group_by(latitude,longitude) %&amp;gt;% 
  count(word, sort = TRUE) %&amp;gt;% 
  left_join(tidy_wicker %&amp;gt;% 
              group_by(latitude,longitude) %&amp;gt;% 
              summarise(total = n())) %&amp;gt;%
  mutate(freq = n/total)

# Look at most frequent terms
freq&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The n here is the total number of times this term has shown up, and the total is how many terms there are present in a particular coordinate.&lt;/p&gt;
&lt;p&gt;Cool! Now we have a representation of terms, their frequency and their position. Now I might want to plot this somehow… one way would be to try to plot the most frequent terms (n &amp;gt; 3) (Some help on how to do this was taken from &lt;a href=&#34;http://blog.revolutionanalytics.com/2016/01/avoid-overlapping-labels-in-ggplot2-charts.html&#34;&gt;here&lt;/a&gt; and &lt;a href=&#34;http://stackoverflow.com/questions/14288001/geom-text-not-working-when-ggmap-and-geom-point-used&#34;&gt;here&lt;/a&gt;). Depending on the size of your data set, the base line for ‘most frequent’ is subject to change – because my data set is relatively small, I’m just going to say words that appear more than three times.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;freq2 &amp;lt;- subset(freq, n &amp;gt; 3) 

# Let&amp;#39;s get a map!
# This will take a couple tries to make sure you have a nice map for your data
map &amp;lt;- get_map(location = &amp;#39;Damen and North, Chicago, Illinois&amp;#39;, zoom = 14)

freq2$longitude&amp;lt;-as.numeric(freq2$longitude)
freq2$latitude&amp;lt;-as.numeric(freq2$latitude)
lon &amp;lt;- freq2$longitude
lat &amp;lt;- freq2$latitude

mapPoints &amp;lt;- ggmap(map) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freq2, aes(x = lon, y = lat, label = word),size = 3) 

mapPoints&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/Map%20Tweet%20Frequency-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Now we have a representation of coordinates in which frequent terms are being used. A lot of these are associated with restaurants or bars (see discussion of Piece pizzeria, Emporium Arcade Bar). We can also go back and investigate certain groupings of terms – for example, the ‘#inktober’ concentration is from an artist who posts a picture of their art everyday. To check things like the concentration of ‘face with tears of joy’, we can go back to our original ‘wicker’ data frame and search for the coordinate that is in our freq2 data frame to find the links to the orginal Instagram posts or just search that coordinate on Google maps. For the coordinate where we find lots of ‘face with tears of joy’ emojis, we can see from Google maps that there is a comedy club, ‘The Comedy Clubhouse’ at that location – guess it’s a good comedy club!&lt;/p&gt;
&lt;p&gt;How about sentiment analysis? What are the most common positive and negative words? This time we’ll just use the sentiment dictionary available in the tidytext package from the BING sentiment corpus.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We can also look at counts of negative and positive words
bingsenti &amp;lt;- sentiments %&amp;gt;%
  filter(lexicon ==&amp;quot;bing&amp;quot;)

bing_word_counts &amp;lt;- tidy_wicker %&amp;gt;%
  inner_join(bingsenti) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

# bing_word_counts

# Now we can graph these
# Change &amp;#39;filter&amp;#39; parameter depending on the size of your data set
bing_word_counts %&amp;gt;%
  filter(n &amp;gt; 2) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/Tweet%20Sentiments-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Unsurprisingly, our Instagram posts are mostly positive, with negative sentiments related to cold (Chicago!) or swear words or difficulty. We see how sentiment analysis is not always infallible, as ‘fall’ is counted as negative when really people are talking about the season and see to be happy about it.&lt;/p&gt;
&lt;p&gt;How about a word cloud?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;wordcloud(freq$word,freq$n, min.freq=3, 
          colors=brewer.pal(1, &amp;quot;Dark2&amp;quot;))&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/Tweet%20Word%20Clouds-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;It’s interesting people talk about Logan Square a lot, but Logan Square is another neighborhood! This could be people talking about Logan Sqaure but tagging a coordinate in Wicker Park or maybe our radius was a little too big and got some Logan Square tweets (it’s pretty close). And yes, the ‘aubergine’ is the eggplant emoji. And yes, the posts are not about the vegetable.&lt;/p&gt;
&lt;p&gt;Speaking of emojis, what are the most common emojis? What if we want to map JUST emojis? We can do this, but it involves a few steps.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# First let&amp;#39;s &amp;#39;tidy&amp;#39; our posts again, but DON&amp;#39;T make everything lowercase. This way we can isolate what are emojis and what are not emojis.
emotidy_wicker &amp;lt;- wicker %&amp;gt;%
  unnest_tokens(word, text, to_lower = FALSE)

# Let&amp;#39;s also tidy our emoji dataframe so we can use it to filter out non-emojis in our data

emojis$Name &amp;lt;- as.character(emojis$Name)

tidy_emojis &amp;lt;- emojis %&amp;gt;%
  unnest_tokens(word, Name, to_lower = FALSE)

# Now we will use inner_join to keep matches and get rid of non-matches
emojis_total &amp;lt;- tidy_emojis %&amp;gt;%
  inner_join(emotidy_wicker)

# What is the most frequent emoji?
freqe &amp;lt;- emojis_total %&amp;gt;% 
   count(word, sort = TRUE)

head(freqe)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6 x 2
##                             word     n
##                            &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1             FACEWITHTEARSOFJOY    12
## 2 SMILINGFACEWITHHEARTSHAPEDEYES     6
## 3                           FIRE     5
## 4                      AUBERGINE     4
## 5          DOUBLEEXCLAMATIONMARK     4
## 6                           EYES     4&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Now we have our most common emojis. Note skin color is not separate -- you can go into the CSV file to change this if you want.

# Map it
# left_join to keep all instances, not just one
test &amp;lt;- freqe %&amp;gt;%
  left_join(emojis_total, freqe, by = &amp;quot;word&amp;quot;)

# Get rid of duplicates
freqe2 &amp;lt;- subset(test, n &amp;gt; 3) 
# Get rid of duplicates
freqe2 &amp;lt;- freqe2[!duplicated(freqe2$id),]

freqe2$longitude&amp;lt;-as.numeric(freqe2$longitude)
freqe2$latitude&amp;lt;-as.numeric(freqe2$latitude)

mapPoints2 &amp;lt;- ggmap(map) + geom_jitter(alpha = 0.1, size = 2.5, width = 0.25, height = 0.25) +
  geom_label_repel(data = freqe2, aes(x = longitude, y = latitude, label = word),size = 3) 

mapPoints2&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/Identify%20Emoji%20Posts%201-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;This is more impressive with a larger data set, but you can see the kinds of things you can look at with these tools. Let’s look at another visualization technique, actually graphing &lt;em&gt;with&lt;/em&gt; emojis using the &lt;a href=&#34;https://github.com/dill/emoGG&#34;&gt;‘emoGG’ package&lt;/a&gt;. Once again, this takes a few steps to set up.&lt;/p&gt;
&lt;p&gt;Let’s do the top six: 😂 😍 🔥 🍆 ✨ and 👀.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# We will use emoGG to find the right code for each one. You can cross-check this with the unicode codepoint listed in your emoji dictionary (usualy they are the same, with all letters in lowercase).
# How to search:
# Identifier for each one for mapping
# &amp;#39;face with tears of joy&amp;#39; 1f602
# &amp;#39;heart eyes&amp;#39; 1f60d
# &amp;#39;fire&amp;#39; 1f525
# &amp;#39;aubergine&amp;#39; 1f346
# &amp;#39;sparkles&amp;#39; 2728
# &amp;#39;eyes&amp;#39; 1f440

tearsofjoygrep &amp;lt;- grepl(paste(&amp;quot;FACEWITHTEARSOFJOY&amp;quot;), freqe2$word)
tearsofjoyDF&amp;lt;-as.data.frame(tearsofjoygrep)
freqe2$ID7 &amp;lt;- 1:nrow(freqe2)
tearsofjoyDF$ID7 &amp;lt;- 1:nrow(tearsofjoyDF)
freqe2 &amp;lt;- merge(freqe2,tearsofjoyDF,by=&amp;quot;ID7&amp;quot;)
tearsofjoy &amp;lt;- freqe2[freqe2$tearsofjoygrep == &amp;quot;TRUE&amp;quot;, ]

sparkgrep &amp;lt;- grepl(paste(&amp;quot;SPARKLES&amp;quot;), freqe2$word)
sparkDF&amp;lt;-as.data.frame(sparkgrep)
sparkDF$ID7 &amp;lt;- 1:nrow(sparkDF)
freqe2 &amp;lt;- merge(freqe2,sparkDF,by=&amp;quot;ID7&amp;quot;)
spark &amp;lt;- freqe2[freqe2$sparkgrep == &amp;quot;TRUE&amp;quot;, ]

egggrep &amp;lt;- grepl(paste(&amp;quot;AUBERGINE&amp;quot;), freqe2$word)
eggDF &amp;lt;-as.data.frame(egggrep)
eggDF$ID7 &amp;lt;- 1:nrow(eggDF)
freqe2 &amp;lt;- merge(freqe2,eggDF,by=&amp;quot;ID7&amp;quot;)
aubergine &amp;lt;- freqe2[freqe2$egggrep == &amp;quot;TRUE&amp;quot;, ]

firegrep &amp;lt;- grepl(paste(&amp;quot;FIRE&amp;quot;), freqe2$word)
fireDF&amp;lt;-as.data.frame(firegrep)
fireDF$ID7 &amp;lt;- 1:nrow(fireDF)
freqe2 &amp;lt;- merge(freqe2,fireDF,by=&amp;quot;ID7&amp;quot;)
fire &amp;lt;- freqe2[freqe2$firegrep == &amp;quot;TRUE&amp;quot;, ]

eyesgrep &amp;lt;- grepl(paste(&amp;quot;EYES&amp;quot;), freqe2$word)
eyesDF&amp;lt;-as.data.frame(eyesgrep)
eyesDF$ID7 &amp;lt;- 1:nrow(eyesDF)
freqe2 &amp;lt;- merge(freqe2,eyesDF,by=&amp;quot;ID7&amp;quot;)
eyes &amp;lt;- freqe2[freqe2$eyesgrep == &amp;quot;TRUE&amp;quot;, ]

heartgrep &amp;lt;- grepl(paste(&amp;quot;SMILINGFACEWITHHEARTSHAPEDEYES&amp;quot;), freqe2$word)
heartDF&amp;lt;-as.data.frame(heartgrep)
heartDF$ID7 &amp;lt;- 1:nrow(heartDF)
freqe2 &amp;lt;- merge(freqe2,heartDF,by=&amp;quot;ID7&amp;quot;)
hearteyes &amp;lt;- freqe2[freqe2$heartgrep == &amp;quot;TRUE&amp;quot;, ]


# Map this
# Some stuff we have to do first
tearsofjoy$latitude &amp;lt;- as.numeric(tearsofjoy$latitude)
tearsofjoy$longitude &amp;lt;- as.numeric(tearsofjoy$longitude)
spark$latitude &amp;lt;- as.numeric(spark$latitude)
spark$longitude &amp;lt;- as.numeric(spark$longitude)
aubergine$latitude &amp;lt;- as.numeric(aubergine$latitude)
aubergine$longitude &amp;lt;- as.numeric(aubergine$longitude)
fire$latitude &amp;lt;- as.numeric(fire$latitude)
fire$longitude &amp;lt;- as.numeric(fire$longitude)
eyes$latitude &amp;lt;- as.numeric(eyes$latitude)
eyes$longitude &amp;lt;- as.numeric(eyes$longitude)
hearteyes$latitude &amp;lt;- as.numeric(hearteyes$latitude)
hearteyes$longitude &amp;lt;- as.numeric(hearteyes$longitude)

# Get a new map that is zoomed in a bit more
map2 &amp;lt;- get_map(location = &amp;#39;Damen and North, Chicago, Illinois&amp;#39;, zoom = 15)


emomap &amp;lt;- ggmap(map2) + geom_emoji(aes(x = longitude, y = latitude), 
                                     data=tearsofjoy, emoji=&amp;quot;1f602&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=spark, emoji=&amp;quot;2728&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=aubergine, emoji=&amp;quot;1f346&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=fire, emoji=&amp;quot;1f525&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=eyes, emoji=&amp;quot;1f440&amp;quot;) +
                              geom_emoji(aes(x=longitude, y=latitude),
                                     data=hearteyes, emoji=&amp;quot;1f60d&amp;quot;)

emomap&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/Identify%20Emoji%20Posts-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Again, this is more impressive when you have more data. You can see some examples of this &lt;a href=&#34;https://lyons7.github.io/portfolio/2017-03-10-emoji-maps/&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;/div&gt;
&lt;div id=&#34;youtube-diachronic-sentiment-analysis&#34; class=&#34;section level2&#34;&gt;
&lt;h2&gt;YouTube + Diachronic Sentiment Analysis&lt;/h2&gt;
&lt;p&gt;Onwards to YouTube data! We’ll be doing some similar analyses, like sentiment analysis, but instead of looking at things in terms of coordinates, we’ll look at things in terms of time. I wanted to choose something that had a lot of comments, so let’s look at the comments from the ill-advised (and ill-fated) ‘Emoji Movie’ trailer. This also has a lot of varying sentiment (one of the comments is “The movie is a such disgrace to the animation film industry.”😹😹😹).&lt;/p&gt;
&lt;p&gt;If you don’t have the YouTube API set up, you can access the data &lt;a href=&#34;https://www.dropbox.com/s/3zwmn7vsvo9ihpv/sampletubedata.Rda?dl=0&#34;&gt;here&lt;/a&gt;.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Connect to YouTube API
# Leave token blank
# yt_oauth(&amp;quot;app_id&amp;quot;, &amp;quot;app_password&amp;quot;, token=&amp;#39;&amp;#39;)

 # emojimovie &amp;lt;- get_comment_threads(c(video_id=&amp;quot;o_nfdzMhmrA&amp;quot;), max_results = 101)

# Save data (if you want)
# save(emojimovie,file=paste(&amp;quot;sampletubedata.Rda&amp;quot;))

# If you need to load that data (make sure you are in the right directory)
load(&amp;#39;sampletubedata.Rda&amp;#39;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now we have some (~10,300) comments to play with. Due to the subject matter, emojis are likely to be frequent in our data set, so let’s follow the same procedure as with our tweets to identify those emojis and label them.&lt;/p&gt;
&lt;p&gt;Help figuring out the emoji encoding issue from &lt;a href=&#34;https://stackoverflow.com/questions/47243155/get-r-to-keep-utf-8-codepoint-representation/47243425#47243425&#34;&gt;Patrick Perry&lt;/a&gt; – thanks Patrick! 😊&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;emojis &amp;lt;- read.csv(&amp;#39;Emoji Dictionary 5.0.csv&amp;#39;, stringsAsFactors = FALSE)
# change U+1F469 U+200D U+1F467 to \U1F469\U200D\U1F467
emojis$escapes &amp;lt;- gsub(&amp;quot;[[:space:]]*\\U\\+&amp;quot;, &amp;quot;\\\\U&amp;quot;, emojis$Codepoint)

# convert to UTF-8 using the R parser
emojis$codes &amp;lt;- sapply(parse(text = paste0(&amp;quot;&amp;#39;&amp;quot;, emojis$escapes, &amp;quot;&amp;#39;&amp;quot;),
                      keep.source = FALSE), eval)

emojimovie$text &amp;lt;- as.character(emojimovie$textOriginal)

# Go through and identify emojis
emoemo &amp;lt;- FindReplace(data = emojimovie, Var = &amp;quot;text&amp;quot;, 
                            replaceData = emojis,
                       from = &amp;quot;codes&amp;quot;, to = &amp;quot;Name&amp;quot;, 
                       exact = FALSE)
# This might take some time, we have a big data set. &lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now let’s clean our data using the same code as we used for our tweets (in case there are any hashtags or @ mentions).&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;emoemo$text = gsub( &amp;quot;&amp;lt;.*?&amp;gt;&amp;quot;, &amp;quot;&amp;quot;, emoemo$text)

reg &amp;lt;- &amp;quot;([^A-Za-z_\\d#@&amp;#39;]|&amp;#39;(?![A-Za-z_\\d#@]))&amp;quot;
tidy_tube &amp;lt;- emoemo %&amp;gt;% 
  filter(!str_detect(text, &amp;quot;^RT&amp;quot;)) %&amp;gt;%
  mutate(text = str_replace_all(text, &amp;quot;https://t.co/[A-Za-z\\d]+|http://[A-Za-z\\d]+|&amp;amp;amp;|&amp;amp;lt;|&amp;amp;gt;|RT|https&amp;quot;, &amp;quot;&amp;quot;)) %&amp;gt;%
  unnest_tokens(word, text, token = &amp;quot;regex&amp;quot;, pattern = reg) %&amp;gt;%
  filter(!word %in% mystopwords,
         str_detect(word, &amp;quot;[a-z]&amp;quot;))

freqtube &amp;lt;- tidy_tube %&amp;gt;% 
  group_by(publishedAt) %&amp;gt;% 
  count(word, sort = TRUE) %&amp;gt;% 
  left_join(tidy_tube %&amp;gt;% 
              group_by(publishedAt) %&amp;gt;% 
              summarise(total = n())) %&amp;gt;%
  mutate(freq = n/total)

# Look at most frequent terms
head(freqtube)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6 x 5
## # Groups:   publishedAt [5]
##                publishedAt               word     n total       freq
##                     &amp;lt;fctr&amp;gt;              &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt;      &amp;lt;dbl&amp;gt;
## 1 2017-10-23T13:00:06.000Z facewithtearsofjoy  2163  2163 1.00000000
## 2 2017-10-12T11:42:22.000Z          pileofpoo   412  1133 0.36363636
## 3 2017-10-07T07:49:00.000Z facewithtearsofjoy   309  1133 0.27272727
## 4 2017-10-15T21:17:26.000Z               film   309  3708 0.08333333
## 5 2017-10-06T18:22:11.000Z              cheap   206   927 0.22222222
## 6 2017-10-06T18:22:11.000Z              knock   206   927 0.22222222&lt;/code&gt;&lt;/pre&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;bing_word_countstube &amp;lt;- tidy_tube %&amp;gt;%
  inner_join(bingsenti) %&amp;gt;%
  count(word, sentiment, sort = TRUE) %&amp;gt;%
  ungroup()

# bing_word_counts

# Now we can graph these
# Change &amp;#39;filter&amp;#39; parameter depending on the size of your data set
bing_word_countstube %&amp;gt;%
  filter(n &amp;gt; 100) %&amp;gt;%
  mutate(n = ifelse(sentiment == &amp;quot;negative&amp;quot;, -n, n)) %&amp;gt;%
  mutate(word = reorder(word, n)) %&amp;gt;%
  ggplot(aes(word, n, fill = sentiment)) +
  geom_bar(alpha = 0.8, stat = &amp;quot;identity&amp;quot;) +
  labs(y = &amp;quot;Contribution to sentiment&amp;quot;,
       x = NULL) +
  coord_flip()&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/YouTube%20Sentiment-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;Perhaps unsurprisingly, it seems most of our comments are negative. We can see here some of the biggest contributors to negative sentiment, terms like ‘disgrace’ or ‘hate’, ‘cheap’ or ‘cringe’.&lt;/p&gt;
&lt;p&gt;Let’s look at things over time. Code from &lt;a href=&#34;http://tidytextmining.com/twitter.html&#34;&gt;Silge and Robinson&lt;/a&gt;. This example is looking at the words that have changed the most over time in terms of frequency.&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Need to change some formatting
tidy_tube$created &amp;lt;- as.character(tidy_tube$publishedAt)
tidy_tube$created &amp;lt;- as.POSIXct(tidy_tube$created)

words_by_time &amp;lt;- tidy_tube %&amp;gt;%
  filter(!str_detect(word, &amp;quot;^@&amp;quot;)) %&amp;gt;%
  mutate(time_floor = floor_date(created, unit = &amp;quot;1 day&amp;quot;)) %&amp;gt;%
  count(time_floor, word) %&amp;gt;%
  ungroup() %&amp;gt;%
  group_by(time_floor) %&amp;gt;%
  mutate(time_total = sum(n)) %&amp;gt;%
  group_by(word) %&amp;gt;%
  mutate(word_total = sum(n)) %&amp;gt;%
  ungroup() %&amp;gt;%
  rename(count = n) %&amp;gt;%
  filter(word_total &amp;gt; 300)

# words_by_time
nested_data &amp;lt;- words_by_time %&amp;gt;%
  nest(-word) 

# nested_data

nested_models &amp;lt;- nested_data %&amp;gt;%
  mutate(models = map(data, ~ glm(cbind(count, time_total) ~ time_floor, ., 
                                  family = &amp;quot;binomial&amp;quot;)))

# nested_models


slopes &amp;lt;- nested_models %&amp;gt;%
  unnest(map(models, tidy)) %&amp;gt;%
  filter(term == &amp;quot;time_floor&amp;quot;) %&amp;gt;%
  mutate(adjusted.p.value = p.adjust(p.value))

top_slopes &amp;lt;- slopes %&amp;gt;% 
  filter(adjusted.p.value &amp;lt; 0.1)

# top_slopes

# Graph
words_by_time %&amp;gt;%
  inner_join(top_slopes, by = c(&amp;quot;word&amp;quot;)) %&amp;gt;%
  ggplot(aes(time_floor, count/time_total, color = word)) +
  geom_line(size = 1.3) +
  labs(x = NULL, y = &amp;quot;Word frequency&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/YouTube%20Comments%20Over%20Time-1.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Graph a subset of just slopes of emojis
# Have to make them lowercase for this 
emojis2 &amp;lt;- emojis %&amp;gt;%
  unnest_tokens(word, Name)

sub_slopes &amp;lt;- top_slopes %&amp;gt;%
  inner_join(emojis2)
  
  
words_by_time %&amp;gt;%
  inner_join(sub_slopes, by = c(&amp;quot;word&amp;quot;)) %&amp;gt;%
  ggplot(aes(time_floor, count/time_total, color = word)) +
  geom_line(size = 1.3) +
  labs(x = NULL, y = &amp;quot;Word frequency&amp;quot;)&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;img src=&#34;/portfolio/2017-11-05-analyzing-social-media-data_files/figure-html/YouTube%20Comments%20Over%20Time-2.png&#34; width=&#34;672&#34; /&gt;&lt;/p&gt;
&lt;p&gt;What are the most common emojis in comments about the emoji movie?&lt;/p&gt;
&lt;pre class=&#34;r&#34;&gt;&lt;code&gt;# Same steps as above
emotidy_tube &amp;lt;- emoemo %&amp;gt;%
  unnest_tokens(word, text, to_lower = FALSE)

emojis$Name &amp;lt;- as.character(emojis$Name)
tube_tidy_emojis &amp;lt;- emojis %&amp;gt;%
  unnest_tokens(word, Name, to_lower = FALSE)

# Now we will use inner_join to keep matches and get rid of non-matches
tube_emojis_total &amp;lt;- tube_tidy_emojis %&amp;gt;%
  inner_join(emotidy_tube)

# What is the most frequent emoji?
tube_freqe &amp;lt;- tube_emojis_total %&amp;gt;% 
   count(word, sort = TRUE)

head(tube_freqe)&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code&gt;## # A tibble: 6 x 2
##                             word     n
##                            &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;
## 1             FACEWITHTEARSOFJOY  2987
## 2                      PILEOFPOO   824
## 3                   GRINNINGFACE   412
## 4 SMILINGFACEWITHHEARTSHAPEDEYES   412
## 5                      AUBERGINE   309
## 6    GRINNINGFACEWITHSMILINGEYES   309&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;So, our most frequent emojis in the comments of the Emoji Movie trailer are 😂, 💩, 😀, 😍, 😁 and 🍆. Read into that what you will! 😂&lt;/p&gt;
&lt;/div&gt;
</description>
    </item>
    
    <item>
      <title>Hello world</title>
      <link>/post/2017-03-08-hello-world/</link>
      <pubDate>Wed, 08 Mar 2017 00:00:00 +0000</pubDate>
      
      <guid>/post/2017-03-08-hello-world/</guid>
      <description></description>
    </item>
    
    <item>
      <title>A Plain Markdown Post</title>
      <link>/post/2016-12-30-hello-markdown/</link>
      <pubDate>Fri, 30 Dec 2016 21:49:57 -0700</pubDate>
      
      <guid>/post/2016-12-30-hello-markdown/</guid>
      <description>&lt;p&gt;This is a post written in plain Markdown (&lt;code&gt;*.md&lt;/code&gt;) instead of R Markdown (&lt;code&gt;*.Rmd&lt;/code&gt;). The major differences are:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;You cannot run any R code in a plain Markdown document, whereas in an R Markdown document, you can embed R code chunks (&lt;code&gt;```{r}&lt;/code&gt;);&lt;/li&gt;
&lt;li&gt;A plain Markdown post is rendered through &lt;a href=&#34;https://gohugo.io/overview/configuration/&#34;&gt;Blackfriday&lt;/a&gt;, and an R Markdown document is compiled by &lt;a href=&#34;http://rmarkdown.rstudio.com&#34;&gt;&lt;strong&gt;rmarkdown&lt;/strong&gt;&lt;/a&gt; and &lt;a href=&#34;http://pandoc.org&#34;&gt;Pandoc&lt;/a&gt;.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;There are many differences in syntax between Blackfriday&amp;rsquo;s Markdown and Pandoc&amp;rsquo;s Markdown. For example, you can write a task list with Blackfriday but not with Pandoc:&lt;/p&gt;

&lt;ul class=&#34;task-list&#34;&gt;
&lt;li&gt;&lt;input type=&#34;checkbox&#34; checked disabled class=&#34;task-list-item&#34;&gt; Write an R package.&lt;/li&gt;
&lt;li&gt;&lt;input type=&#34;checkbox&#34; disabled class=&#34;task-list-item&#34;&gt; Write a book.&lt;/li&gt;
&lt;li&gt;&lt;input type=&#34;checkbox&#34; disabled class=&#34;task-list-item&#34;&gt; &amp;hellip;&lt;/li&gt;
&lt;li&gt;&lt;input type=&#34;checkbox&#34; disabled class=&#34;task-list-item&#34;&gt; Profit!&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Similarly, Blackfriday does not support LaTeX math and Pandoc does. I have added the MathJax support to this theme (&lt;a href=&#34;https://github.com/yihui/hugo-lithium-theme&#34;&gt;hugo-lithium-theme&lt;/a&gt;) but there is a caveat for plain Markdown posts: you have to include inline math expressions in  a pair of escaped parentheses &lt;code&gt;\( \)&lt;/code&gt; instead of dollar signs &lt;code&gt;$ $&lt;/code&gt;, e.g. &lt;code&gt;\(S_n = \sum_{i=1}^n X_i\)&lt;/code&gt;. For R Markdown posts, you can use &lt;code&gt;$ $&lt;/code&gt; to write inline math expressions.&lt;/p&gt;

&lt;p&gt;When creating a new post, you have to decide whether the post format is Markdown or R Markdown, and this can be done via the &lt;code&gt;rmd&lt;/code&gt; argument of the function &lt;code&gt;blogdown::new_post()&lt;/code&gt;, e.g.&lt;/p&gt;

&lt;pre&gt;&lt;code class=&#34;language-r&#34;&gt;blogdown::new_post(&amp;quot;Post Title&amp;quot;, rmd = FALSE)
&lt;/code&gt;&lt;/pre&gt;
</description>
    </item>
    
  </channel>
</rss>