The Use and Limitations of Advanced Algorithms in Real Estate
The Use and Limitations of Advanced Algorithms in Real Estate
Tim Savage PhD, Chief Economic Advisor

INTRODUCTION
The advent of large language models (LLMs) has thrust the idea of artificial intelligence (AI) into the popular lexicon. AI is not new. It draws on nearly 250 years of human exploration of an obvious question: how do we examine conjectures about the way the world works? For commercial real estate (CRE), these conjectures have historically taken a specific form: how will market fundamentals (rents, vacancy, and cap rates) vary in the future based on current market conditions? Salient questions I would like to address for my students at NYU Schack are: whether “statistical learning” (my preferred term for AI) is novel to CRE, and what are, with high probability, the future contours of its impacts on our industry.
IS AI NOVEL TO CRE?
Let’s start with the first question, to which I answer a resounding “no.” CRE is dominated by two things: location and human relationships. More generally, finance is motivated by the search for the highest and best use of capital, ideally with the highest risk-adjusted returns. In finance, the use of statistical learning goes back centuries with both failure and success. The first success was a concise mathematical statement of the problem laid out by Pierre Simon LaPlace, who created the two forms of statistical learning that we now call classical and Bayesian. At the time, we had the formulas but no data or computational power. As a result, statistical learning remained largely a mathematical discipline based on the idea that “experiments” were a set of repeated but independent coin flips. As a side note, “games of chance” became a descriptor of statistical learning at the time. Students know that when I teach risk aversion, I always ask who has been to a casino. I am typically disappointed by the number of hands I see raised.
The next major practical and successful innovation came from the Russian mathematician, Andrey Markov, whose name we now give to Markov processes. Markov asked a simple question about statistical learning: what if experiments (or more broadly, events) where not independent over time? In modern finance, this was a revolution. Asset price movements are not independent events unless markets are highly liquid and informationally efficient. Today’s asset price may be correlated with yesterday’s asset price, suggesting that returns may too autocorrelate through time. Today’s rents may be correlated with yesterday’s (or last quarter’s) rents. Today’s vacancy may be correlated with last quarter’s vacancy. As cap rates are simply yields, they too may be correlated. All of this to say, the application of Markovian processes to prediction in commercial real estate should be clear.
Roughly at the same time, the next noteworthy innovation in finance would arise from Louis Bachelier in the early 1900’s. Bachelier focused directly on the idea of asset price changes over time. He laid out a framework that we now call Brownian motion to value options, a precursor to the Black-Scholes-Merton framework. (As a side note, the word “stochastic” simply means random.) The Bachelier approach was to focus on asset prices, following Markov, whose approach was more general, rather than returns and surely risk-adjusted returns.
As a side note, at this time, there was no such thing as “real estate finance” or even of “commercial real estate”. None of the iconic buildings in New York City existed when Markov and Bachelier were their most productive. Indeed, the modern electric elevator, without which the New York City skyline would not exist, had yet to be deployed. Moreover, the idea of land valuation was driven simply by the idea of agricultural production (see, for example, Geltner et. al, Chapter 4). Happily, for New York City, the idea of urban planning was non-existent, other than the 1822 plan that laid out the city’s network of roads.
The next evolution of classical statistical learning would come with a formalized framework that we now call classical hypothesis testing developed by the mathematical statisticians Neyman, Fisher and Pearson. The “NFP” framework lays out two exhaustive and mutually exclusive conjectures that are then subject to further evidence. In commercial real estate, this framework would ask: if I lower my rents will I increase my occupancy? This conjecture can easily be explored using historic rent and vacancy data together with the “linear regression” algorithm but may give an answer that makes no sense from the perspective of real estate professional.
By the 1970’s, we had algorithms and a framework to examine conjectures. How much more should I charge for the 10th floor than for the 4th floor? No algorithm, whether simple linear regression or complicated deep learning, can work without data. And the industry lacked data. At the time, we had difficulty even measuring net operating income (NOI). Further, it was unclear whether CRE was even an asset class, compared to equities, bonds and cash. A major innovation for the industry was Blake Eagle’s National Council of Real Estate Investment Fiduciaries (NCREIF), which was, to quote, “Established over 40 years ago, the National Council of Real Estate Investment Fiduciaries (NCREIF) serves the institutional real estate investment community as its Data Central, representing the largest, most robust and diverse database of country-specific real estate assets in the world. NCREIF produced the first property level return index, the NCREIF Property Index (NPI), dating back to 1978 to capture investment performance records that meet the rigorous scrutiny and review of major investors and academia.” Now the CRE industry had data and algorithms but limited computing power.
In the late 1980’s, two prominent Boston-based real estate economists met (perhaps randomly on a street corner, though the story remains shrouded in mythology). Raymond Torto (Ray) and William Wheaton (Bill) decided that they should create the Torto-Wheaton Research Group (TWR), whose function was to develop forecasting algorithms to predict future rents and vacancies. But they, too, lacked data because NCREIF data were not sufficiently granular. What did they do? They ultimately turned to the largest commercial real estate services that I name “the house.” They first followed the split model common with brokerages: if they got a client, they earned 50%, and 50% went to the house. In turn, the house would provide them access their revenue management system, in which the details of all brokered deals were settled.
An early creation was the TWR rent index, followed by simple models of forecasting based on the book, “Urban Economics and Real Estate Markets” by Denise DiPasquale and William C. Wheaton. In this process, TWR became the ground truth to the CRE industry in the late 1990’s and 2000’s: it combined of the use of algorithms and data to provide the industry with actionable intelligence. Other prominent CRE entities emerged at this time, the data of which students may use in their jobs or in class. For example, the Commercial Real Estate Finance Counsel (CREFC) is an industry organization focused on the (considerable) debt markets used to finance development, acquisitions, and dispositions in our industry. MSCI Real Cap Analytics provides data and forecasts of transactional cap rates.
FUTURE CONTOURS
In my opinion, the industry, from the perspective of the use of algorithms and data, is more fragmented than my days of running TWR. We have much more data and better algorithms, but there is no ground truth. This reflects the complexities of our industry, which is comprised of a leveraged and highly-heterogenous asset that cannot be physically moved. To emphasize this point, sit in Bryant Park with your lunch and look around.
As to the future, I can only speak as a Bayesian, which means probabilistically. With high probability, the use of advanced algorithms will arise when we think about CRE finance. Why? Because, in finance, they already have, of which CRE finance is a subset. This is why I stress to students that these are, in the best sense of the phrase, interesting times in CRE. An LLM can help you do a DCF, but it will prompt you for the necessary ingredients, such as discount rate or reversion cash flows. Your judgment is critical.
Algorithms will be accretive to deals, but they will not replace the relationships necessary in CRE. Trust remains a key component to any deal even if the parties are competing. Remember, this is a repeated game, and even generative AI does not quite get basic game theory, which is why it punts if it cannot answer a question. Moreover, there are limitations to advanced algorithms in finance. (See Savage and Vo, “Big Data, Simulation and Causation” and Savage and Vo, “Deep Learning and Finance: Here We Go Again”.)
Adapted from an NYU Schack Institute of Real Estate Publication.