Technical note

Creating a correspondence map for Tsukuba plans and census tracts

Alan Engel
Last update: 9 October 2020

Objective: Using R, create maps for comparing census tracts to Tsukuba planning maps.

This technical note describes a procedure for visualizing and comparing census tracts and planning maps using shapefiles downloaded from https://www.e-stat.go.jp/ (described in Original shape files, and a planning map from Tsukuba City, using a classification described by Kawanaka and Kaneko (2015).

1. Data and reference sources

1.1 Ibaraki Prefecture census tract shapefiles

Shapefile data for all prefectures of Japan and for each census can be downloaded from e-Stat. Those for Ibaraki Prefecture are included in the R package AlanInTsukuba/jpucd on Github.

library(remotes) # or library(devtools)
install_github("AlanInTsukuba/jpucd")

Shapefiles downloaded from e-Stat are in KMZ format and were converted to GeoJSON format using a utility written in C#.

> # read census tract file for 2015 general census
> library(jpucd)
> library(sf)
> shppath <- system.file("extdata",package="jpucd")
> list.files(shppath)
> # read in previously converted geojson shapefile
> gjsn2015 <- paste(datapath , "JPGen2005CTgenlCY2015P08Ibaraki.geojson",sep="/")
> ib2015gjsn <- st_read(gjsn2015 )
> head(ib2015gjsn)
Simple feature collection with 6 features and 36 fields
geometry type:  MULTIPOLYGON
dimension:      XY
bbox:           xmin: 140.406 ymin: 36.35057 xmax: 140.5438 ymax: 36.39631
geographic CRS: WGS 84
            jpid    KEY_CODE PREF CITY S_AREA PREF_NAME CITY_NAME     S_NAME KIGO_E HCODE
1 10820100200100 08201002001   08  201 002001    茨城県    水戸市 赤塚1丁目         8101
2 10820100200200 08201002002   08  201 002002    茨城県    水戸市 赤塚2丁目         8101
3 10820100300000   082010030   08  201 003000    茨城県    水戸市     圷大野         8101
4 10820100400000   082010040   08  201 004000    茨城県    水戸市       曙町         8101
5 10820100500002   082010050   08  201 005000    茨城県    水戸市     朝日町     E2  8101
6 10820100500001   082010050   08  201 005000    茨城県    水戸市     朝日町     E1  8101
        AREA  PERIMETER H27KAxx_ H27KAxx_ID KEN KEN_NAME SITYO_NAME GST_NAME CSS_NAME KIHON1
1 563757.188 3603.69897     3237       3236  08   茨城県              水戸市            0020
2 335358.969 2711.66504     1332       1331  08   茨城県              水戸市            0020
3 1070799.75 5474.47607     2179       2178  08   茨城県              水戸市            0030
4 79852.6094 1244.79102     1330       1329  08   茨城県              水戸市            0040
5 17083.2852  643.72699     2388       2387  08   茨城県              水戸市            0050
6 109892.008 1943.10596     2389       2388  08   茨城県              水戸市            0050
  DUMMY1 KIHON2  KEYCODE1  KEYCODE2 AREA_MAX_F KIGO_D N_KEN N_CITY KIGO_I       MOJI KBSUM JINKO
1      -     01 201002001 201002001          M                            赤塚1丁目    34  2876
2      -     02 201002002 201002002          M                            赤塚2丁目    12  1063
3      -     00 201003000   2010030          M                                圷大野     4   182
4      -     00 201004000   2010040          M                                  曙町    11   677
5      -     00 201005000   2010050                                           朝日町     0     0
6      -     00 201005000   2010050          M                                朝日町    18   529
  SETAI     X_CODE    Y_CODE  KCODE1                       geometry
1  1354 140.415543 36.384178 0020-01 MULTIPOLYGON (((140.4213 36...
2   459 140.416122 36.388619 0020-02 MULTIPOLYGON (((140.4135 36...
3    55 140.534119 36.357349 0030-00 MULTIPOLYGON (((140.5414 36...
4   372 140.443985 36.394661 0040-00 MULTIPOLYGON (((140.4438 36...
5     0   140.4841  36.35981 0050-00 MULTIPOLYGON (((140.4847 36...
6   241 140.487564 36.359901 0050-00 MULTIPOLYGON (((140.488 36....

The column codes are defined in origshapefiles.html Tsukuba City is CITY 220, which can be used to select only Tsukuba census tracts.

> # get Tsukuba tracts
  > tsukubaMP2015<- ib2015gjsn[which(ib2015gjsn$CITY == 220),c("jpid","CITY_NAME","CITY","MOJI" ,"S_AREA"     
,"KIGO_E", "AREA" , "PERIMETER" ,"geometry")]

For the shape outlines to be visualized on https://maps.gsi.go.jp, drawing attributes must be added. (Specification)

> # add properties for maps.gsi.go.jp
> tsukubaMP2015$X_name <- paste(tsukubaMP2015$MOJI,"2015",sep="-")
> tsukubaMP2015$X_color <- "#ff0000" # names starting with underscore not allowed in R
> tsukubaMP2015$X_opacity <- 1
> tsukubaMP2015$X_weight <- 3
> tsukubaMP2015$X_fillColor <- "#666600"
> tsukubaMP2015$X_fillOpacity <- 0
> # change the attribute names to match maps.gsi.go.jp
> colnames(tsukubaMP2015) <- c("jpid","CITY_NAME","CITY","MOJI" ,"S_AREA"     
,"KIGO_E", "AREA" , "PERIMETER" ,"geometry","name"         
, "_color" ,"_opacity" ,"_weight","_fillColor","_fillOpacity")

Finally write this data frame as a geojson file.

> dsnyko <- "TsukubaMasterPlan-2015.geojson"
> st_write(tsukubaMP2015, dsnyko, delete_dsn=TRUE,layer="CensusTracts")

This file can then be dragged and dropped onto the maps.gsi.go.jp base map.

1.2 Reference planning maps

Tsukuba City map of peripheral towns
Tsukuba City map of revitalization towns

Tsukuba City makes available its urban planning map at higher resolution than the census tract shapefiles and can be used to visually compare tract shapefiles to urban plans. For the task at hand, comparing census tracts to the grand plans, the map at left of Tsukuba's outlying towns provides comparable detail in a more convenient format. Figure 3-4 in Kawanaka and Kaneko (2015) is essentially the same map but is simplified by major plan and omits detail within the respective plans. The 5 major plans are the following:

The white areas are general nonurban areas (GNL) not included in any of the above plans.

1.3 Geospatial Information Authority of Japan (GSI) web map

GSI base map with census tract overlay
GSI base map with census tract overlay

The GSI web map is detailed at the structure level. The map at left was created by dragging the file TsukubaMasterPlan-2015.geojson that was created with the R code at top onto the GSI web map. The red lines outline the census tracts. Clicking on a tract brings up a properties panel that lists the attributes of the data frame tsukubaMP2015. The GSI base map can be selected from a standard colors map, a grayed structures map, a white map, and a satellite photo. Each of these can be modified with a grayscale, transparency level, or both.

2. Correspondence table for census tracts and plans

Tract-plan correspondence table (selected rows)
jpid USE1Code USE1Desc USE2Code USE2Desc USE3Code USE3Desc
10822000100001 PTR Yatabe (R8) GNL Yatabe <NA> <NA>
10822000500002 IND Katata <NA> <NA> <NA> <NA>
10822013900000 TEXP [Nakane-Kontadai] Sakuranomori PTR Sakai GNL Nakane
10822014500000 TEXP [Nakane-Kontadai] Sakuranomori/Ryuseidai PTR Sakai GNL Konda
10822002000000 TEXP [Kamikawarazaki-Nakanishi] GNL Shimana <NA> <NA>
10822002310000 TEXP [Kamikawarazaki-Nakanishi] GNL Kamikawarazaki <NA> <NA>
10822002320000 TEXP [Kamikawarazaki-Nakanishi] GNL Kamikawarazaki <NA> <NA>
10822002700000 TEXP [Kamikawarazaki-Nakanishi] GNL Shimokawarazaki <NA> <NA>
10822021500100 TEXP [Katsuragi] Gakuen no Mori 1 <NA> <NA> <NA> <NA>

The tract-plan correspondence table maps 338 shapefiles from the 2015 general census to Tsukuba planning mapes. Some tracts overlapped as many as 2 plans as well as general nonurban land. Place names in brackets are those listed in Inui (2017) for the Tsukuba Express development plan. If Tsukuba City has designated newer place names, these are appended after the original plan name, e.g., [Katsuragi] Gakuen no Mori 1.

3. Additional features

The correspondence map in Section 4 is enhanced with accessory maps extracted from files downloaded from the GIS Home Page of the Ministry of Land, Infrastructure, and Tourism.

The R package jpucd contains copies of the geojson files produced in this section.

3.1 Municipal boundaries

Shapefiles for Japanese municipal boundaries are available by prefecture for 1920, every five years between 1950 and 2005, and annually thereafter. This technical note shows the shapefiles for Ibaraki municipalities for 2015 (Heisei 27).


## background_maps is local subfolder
ibdsn2015 <- paste(getwd(),"background_maps","N03-20150101_08_GML",sep="/")
ib2015admin <- st_read(ibdsn2015)
head(ib2015admin)
plot(ib2015admin["N03_007"])

## just Tsukuba
tsukuba2015outline <- with(ib2015admin, ib2015admin[N03_007 == "08220",])
plot(tsukuba2015outline["N03_007"])

## can also do following
plot(ib2015admin[ib2015admin$N03_007 == "08220",]["N03_007"])
## export
writepath <- "IbarakiMunicipalBoundaries2015.geojson"
st_write(ib2015admin, writepath , delete_dsn=TRUE)

3.2 Railways

Maps of Japanese railways are available since 2005 and are updated annually. This technical note uses file N02-18_GML.zip from 2018.


### Extract railway maps
library(sf)
## Tsukuba Express
mappath <- "N02-18_GML" # unzipped GML files from MLIT
gmlmap <- st_read(mappath, layer="N02-18_RailroadSection")
head(gmlmap)
Encoding(gmlmap$N02_003) <- "unknown"
Encoding(gmlmap$N02_004) <- "unknown"
head(gmlmap)
txLine <- with(gmlmap, gmlmap[N02_003 =="常磐新線",]) ## Tsukuba Express
plot(txLine["N02_003"])
writepath <- "TsukubaExpressLine.geojson"
st_write(txLine , writepath , delete_dsn=TRUE,layer="RailroadSection")

## Stations
mappath <- "N02-18_GML"
gmlmap <- st_read(mappath, layer="N02-18_Station")
head(gmlmap)
Encoding(gmlmap$N02_003) <- "unknown"
Encoding(gmlmap$N02_004) <- "unknown"
Encoding(gmlmap$N02_005) <- "unknown"
head(gmlmap)
txStations <- with(gmlmap, gmlmap[N02_003 =="常磐新線",])
plot(txStations["N02_003"])
writepath <- "TsukubaExpressStations.geojson"
st_write(txStations , writepath , delete_dsn=TRUE,layer="Station")

## Joban Line
mappath <- "N02-18_GML"
gmlmap <- st_read(mappath, layer="N02-18_RailroadSection")
head(gmlmap)
Encoding(gmlmap$N02_003) <- "unknown"
Encoding(gmlmap$N02_004) <- "unknown"
head(gmlmap)
jobanLine <- with(gmlmap, gmlmap[N02_003 =="常磐線",])
plot(jobanLine["N02_003"])
writepath <- "JobanLine.geojson"
st_write(jobanLine , writepath , delete_dsn=TRUE,layer="RailroadSection")

## Stations
mappath <- "N02-18_GML"
gmlmap <- st_read(mappath, layer="N02-18_Station")
head(gmlmap)
Encoding(gmlmap$N02_003) <- "unknown"
Encoding(gmlmap$N02_004) <- "unknown"
Encoding(gmlmap$N02_005) <- "unknown"
head(gmlmap)
jobanStations <- with(gmlmap, gmlmap[N02_003 =="常磐線",])
plot(jobanStations["N02_003"])
writepath <- "JobanLineStations.geojson"
st_write(jobanStations , writepath , delete_dsn=TRUE,layer="Station")        
      

3.3 Expressways

Maps of Japanese railways are available since 2011 and are updated annually. This technical note uses file N06-15_GML.zip from 2015. Field N06_001 is the fiscal year in which the section was opened and can be used to match stage of completion to census year.


### Extract expressway maps
library(sf)
## Joban Expressway
mappath <- "N06-15_GML"
gmlmap <- st_read(mappath, layer="N06-15_HighwaySection")
head(gmlmap)
jobanExpressway <- with(gmlmap, gmlmap[N06_007 =="常磐自動車道",])
plot(jobanExpressway["N06_007"])
writepath <- "JobanExpressway.geojson"
st_write(jobanExpressway , writepath , delete_dsn=TRUE,layer="HighwaySection")

## Kenodo
mappath <- "N06-15_GML"
gmlmap <- st_read(mappath, layer="N06-15_HighwaySection")
head(gmlmap)
kenodo2015 <- with(gmlmap, gmlmap[N06_007 =="首都圏中央連絡自動車道" & N06_001 < 2016,])
plot(kenodo2015["N06_007"])
writepath <- "Kenodo.geojson"
st_write(kenodo2015, writepath , delete_dsn=TRUE,layer="Station")

4. Correspondence map for census tracts and plans

This section shows the R code for combining the correspondence table from Section 2 with the shapefiles from Section 1.

The first task is to look for errors and omissions in the table. Comparing the correspondence table to data frame tsukubaMP2015 found that 6 tracts were missing. A left outer merge followed by selecting the subset without USE1Code entries produced a subset that could be written as a geojson file and imported into the GSI web map.


# The tract-plan correspondence table is in a csv file
tract_plan2015 <- read.csv("TsukubaMasterPlan-Tract-Plan-2015.csv",
  header = TRUE, sep = ",",strip.white = TRUE,na.strings = "NA",
  colClasses = c("character","character","character","character","character","character","character"))
head(tract_plan2015)

summary(tract_plan2015)
summary(tsukubaMP2015)
## tract_plan2015 has 332 rows, tsukubaMP2015 has 338. 6 are missing from 
## tract_plan2015

## do left outer merge
temp <- merge( x = tsukubaMP2015,y = tract_plan2015, by = "jpid", all.x=TRUE)
head(temp)
temp[1:338,c("USE1Code")]
plot(temp[is.na(temp$USE1Code),]["jpid"])
tempmissing <- temp[is.na(temp$USE1Code),]

dstemp <- "TsukubaMasterPlan-2015-missing.geojson"
st_write(tempmissing , dstemp , delete_dsn=TRUE,layer="CensusDistricts")

The correspondence table can then be merged with the master plan data frame. This is then plotted using a palette that matches that of Figure 3-4 in Kawanaka and Kaneko (2015)


##############################################
## now with all tracts coded merge and plot
tract_plan2015 <- read.csv("TsukubaMasterPlan-Tract-Plan-2015.csv",
  header = TRUE, sep = ",",strip.white = TRUE,na.strings = "NA",
  colClasses = c("character","character","character","character","character","character","character"))
head(tract_plan2015)

## redo tsukubaMP2015
tsukubaMP2015 <- ib2015gjsn[which(ib2015gjsn$CITY == 220),c("jpid","CITY_NAME","CITY","MOJI" ,"S_AREA"     
,"KIGO_E", "AREA" , "PERIMETER" ,"geometry")]
tsukubaMP2015 <- merge( x = tsukubaMP2015,y = tract_plan2015, by = "jpid", all.x=TRUE)

## make palette to match Kawanaka & Kaneko 2015
kawanakapalette <- c("#4798C7","#FFFFFF","#A73986","#F6D9E4","orange","#FFEF43","#AEB534")

plot(tsukubaMP2015["USE1Code"],main="Census tracts and Tsukuba urban plans 2015",
pal=kawanakapalette ,
key.pos=3,
lwd=1, reset=FALSE)

Several tracts overlap two or even three plan use classes. Those with second and third uses are marked with a circle or triangle respectively.


####### try plotting points at centroids of tracts with USE2Code
tsukubaMP2015.centroids <- st_centroid(tsukubaMP2015[!is.na(tsukubaMP2015$USE2Code),c("USE2Code","USE3Code","geometry")],of_largest_polygon=TRUE)

png("Census tracts and Tsukuba urban plans 2015.png",width=600,height=900)
plot(tsukubaMP2015["USE1Code"],main="Census tracts and Tsukuba urban plans 2015",
pal=kawanakapalette ,
key.pos=1,key.width=0.05,key.length=.6,
lwd=1, reset=FALSE)
plot(tsukubaMP2015.centroids["USE2Code"],
pal=kawanakapalette , pch=19,cex = 2,
key.pos=NULL,
lwd=1, add=TRUE)
plot(tsukubaMP2015.centroids["USE3Code"],
pal=kawanakapalette , pch=17,cex = 1,
key.pos=NULL,
lwd=1, add=TRUE)
plot(ib2015admin["N03_001"],col="#00000000",border="black",lwd=2,key.pos=NULL,  add=TRUE)
plot(txLine["N02_002"], col="red",key.pos=NULL,  lwd=2,lty=1, axes=FALSE, add=TRUE)
plot(txStations["N02_002"], col="red", key.pos=NULL,  lwd=12, add=TRUE)
plot(jobanLine["N02_002"],key.pos=NULL,   col="blue",lwd=4,lty=1, axes=FALSE, add=TRUE)
plot(jobanExpressway["N06_002"], col="purple",key.pos=NULL,  ,lwd=4,lty=1, axes=FALSE, add=TRUE)
plot(kenodo2015["N06_002"], col="purple",key.pos=NULL,  ,lwd=4,lty=1, axes=FALSE, add=TRUE)
text(139.988,36.07,"Tsukuba\nExpress", col="red",font=2,pos=2)
arrows(139.988,36.07,140.06,36.07,col="red",lwd=2)
text(140.14,35.97,"Joban Railway", col="blue",font=2,pos=4)
text(139.98,35.99,"Joban\nExpressway", col="purple",font=2, pos=4)
text(140.147,35.985,"Kenodo\nExpressway", col="purple",font=2, pos=4)
dev.off()

Plot with ggplot


  ## plot with ggplot
  library("ggplot2")
  library("ggrepel")
  
  featurelabels <- data.frame(feature = c("Tsukuba\nExpress","Joban\nRailway",
    "Joban\nExpressway","Kenodo\nExpressway"), 
    lat=c(35.987,35.96,35.988, 35.992), lng=c(140.035,140.1400,140.021, 140.16))
  (featurelabels <- st_as_sf(featurelabels, coords = c("lng", "lat"), remove = FALSE, 
      crs = 4326, agr = "constant"))
  
  png("Census tracts and Tsukuba urban plans 2015.png",width=600,height=900)
  ggplot(tsukubaMP2015) +
    geom_sf( aes(fill = USE1Code) ) + theme_void() +
    scale_fill_manual(values = kawanakapalette ) +
    geom_sf(data=tsukubaMP2015.centroids[!is.na(tsukubaMP2015.centroids$USE2Code),], 
    size = 5, shape = 21, aes(fill = USE2Code)) +
    geom_sf(data=tsukubaMP2015.centroids[!is.na(tsukubaMP2015.centroids$USE3Code),], size = 3, 
          shape = 24, aes(fill = USE3Code)) +
    geom_sf(data = ib2010admin, fill = NA, color = "black", size = 1.5 ) + 
    geom_sf(data = txLine , color = "red", size = 1.5,linetype = "solid") +
    geom_sf(data = txStations, color = "red", size = 4,shape = 23) +
    geom_sf(data = jobanLine, color = "blue", size = 1.5,linetype = "solid") +
    geom_sf(data = jobanExpressway, color = "purple", size = 1.5,linetype = "solid") +
    geom_sf(data = kenodo, color = "purple", size = 1.5,linetype = "solid") +
    geom_text_repel(data = featurelabels, aes(x = lng, y = lat, label = feature), 
          size = 3.5, col = "black" , fontface = "bold", 
    nudge_x = c(0.020, -0.09, -0.00, 0.1), 
    nudge_y = c(-0.005, 0.005, 0.025, -0.015)) +
    geom_text_repel(data = citylabels2005 , aes(x = lng, y = lat, label = city), 
          size = 6.0, col = "black" , fontface = "bold", segment.size=2, arrow=arrow(ends="last"),
    nudge_x = c(0.03), 
    nudge_y = c(0.01)) +
    coord_sf(xlim = c(139.99,140.18), ylim = c(35.94,36.24), expand = FALSE) +
    labs(fill='Plan Code') +
    ggtitle("Census tracts and Tsukuba urban plans", subtitle = "(2015 general census)")
  dev.off()
  
Census tracts and Tsukuba urban plans - 2015
Census tracts and Tsukuba urban plans - 2015

Footnotes

References

Kawanaka, Takashi, Hiroshi Kaneko (2015) Tsukuba Science City's problems on formation process through a present state and subjects, Technical Note of NILIM, No. 815, January 2015, http://www.nilim.go.jp/lab/bcg/siryou/tnn/tnn0815.htm

Inui, Yasuyo (2017) The Details and Characteristics of Residential Development alongside Tsukuba Express Railway – Examination of 21 Century’s Suburban Development, Bulletin of the Faculty of Education, Ibaraki University. Humanities and social sciences, Vol 66, pp 35-50, 30 Mar 2017, http://hdl.handle.net/10109/13324

R citations

Engel, Alan 2020. jpucd: JP unified census districts. R package version 0.2.0.9002. https://github.com/AlanInTsukuba/jpucd

Pebesma, E., 2018. Simple Features for R: Standardized Support for Spatial Vector Data. The R Journal 10 (1), 439-446, https://doi.org/10.32614/RJ-2018-009