Socrata was acquired by Tyler Technologies in 2018 and is now the Data and Insights division of Tyler. The platform is still powered by the same software formerly known as Socrata but you will see references to Data & Insights going forward.

Forecasting with RSocrata

stuagano

stuagano

    Ever since the City of Chicago team built the RSocrata Connector I have been dying to put together this simple tutorial for showing the ability for forecasting data in R on top of Socrata datasets. Most recently the City of Chicago updated the RSocrata connector to include write.socrata which will have a number of interesting uses.

    This example is pulling a dataset in from the City of Austin Open Data Portal containing EMS Incidents by Month and forecasting the next two years of EMS Incidents.

    Three dependencies you will need are:

    devtools [needed for getting the GitHub version of RSocrata and not through CRAN]
RSocrata [reading and writing to and from Socrata]
forecast [values generator]

    Step 0: getting RSocrata from GitHub

    install_github("Chicago/RSocrata")

    The first real step is to import the dataset as an R dataframe.

    # API Endpoint for EMS-Ambulence Responses by Month
EMSIncidents <- read.socrata("https://data.austintexas.gov/resource/bptg-ndvw.json")

    The next step is to create a time series variable based off of the response column in the dataset.

    # Create time series variable based off of "count_responses_all"
myts <- ts(EMSIncidents$count_responses_all, start=c(2010, 1), end=c(2015,2), frequency=12)
plot(myts)

    Time Series Plot

    The next component is to control for seasonality that exists within the data.

    # Seasonal Decomposition#
fit <- stl(myts, s.window = "period")
plot(fit)

    Seasonality

    Next we can forecast the dataset out a number of periods.

    # Projected Forecast
forecast(fit)
plot(forecast(fit))

    Projected Forecast

    Let’s save these forecasted values in their own data frame

    projected <- forecast(fit) # stores it as a list 
projected.DF <- as.data.frame(projected) #converts list to data frame

    Next lets write this to Socrata

    # Store user email and password
socrataEmail <- Sys.getenv("SOCRATA_EMAIL", "XXX@socrata.com")
socrataPassword <- Sys.getenv("SOCRATA_PASSWORD", "XXXXXXX")

datasetToAddToUrl <- "https://opendata.socrata.com/resource/evnp-32vr.json" # dataset

write.socrata(projected.DF,datasetToAddToUrl,"UPSERT",socrataEmail,socrataPassword)

    Let’s use Socrata to visualize this data as well.

    Check out the Code on Github

    RSocrata saves the Day!